14 May 2020 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by a Technical Team under the USAID/Nigeria Monitoring, Evaluation & Learning Activity. EVALUATION FINAL PERFORMANCE EVALUATION ELECTORAL EMPOWERMENT OF CIVIL SOCIETY PROGRAM 2 FINAL PERFORMANCE EVALUATION ELECTORAL EMPOWERMENT OF CIVIL SOCIETY PROGRAM (USAID Cooperative Agreement #AID-620-A-14-00003) Prepared by: Dr. Reuben Hermoso, Senior Evaluation Specialist and Team Leader Ms. Adaozo Umenwaliri, Mid-Level Evaluation Specialist Mr. Babajide Ojo, Local Evaluator Mr. Benson Adetona, Local Evaluator Submitted by: Mr. Paul DeLucco, Chief of Party, USAID MEL Activity, DevTech Systems, Inc Submitted to: Dr. Chidimma Anyanwu, COR, PRO Office, USAID/Nigeria Ms. Beth Brownson, Alt. COR, PRO Office, USAID/Nigeria Dr. Beatrice Reaud, Democracy Officer, PDG Office, USAID/Nigeria Cover Photo: Data Clerks During Simulation Exercise in Preparation for Anambra Election Observation, Youth Initiative for Advocacy Growth and Advancement (YIAGA). Disclaimer: The views expressed in this report do not necessarily reflect the views of the United States Agency for International Development. 3 ACRONYMS AAR After Action Review AMELP Activity Monitoring and Evaluation and Learning Plan CBO Community-Based Organization CCD Centre for Citizens with Disabilities CDD Ghana Center for Democratic Development-Ghana CHAD Center for Community Health and Development CLHEI Community Links and Human Empowerment Initiative CITAD Center for Information Technology and Development COP Chief of Party CPI Challenged Parenthood Initiative CSO Civil Society Organization DCOP Deputy Chief of Party DFID United Kingdom Department for International Development DSIEC Delta State Independent Electoral Commission ECES European Centre for Electoral Support EECSP Electoral Empowerment of Civil Society Program ET Evaluation Team FCT Federal Capital Territory FGD Focus Group Discussion FIDA International Federation of Women Lawyers FOMWAN Federation of Muslim Women Association of Nigeria GADA Gender and Development Action GAT Gender Awareness Trust GON Government of Nigeria GREP Girl Child Empowerment & Reproductive Health Initiative HFDI Hope for Family Development Initiative IDP Internally Displaced Person IFA Inclusive Friends Association IFES International Foundation for Electoral Systems INEC Independent National Electoral Commission IPAC Inter Party Advisory Council IRI International Republican Institute JDPC Justice and Development Peace Commission JONAPWD Joint National Association of Persons with Disabilities KADSICOM Kaduna State Independent Electoral Commission KATSIEC Katsina State Independent Electoral Commission KDI Kimpact Development Initiative KII Key Informant Interview LANW Legal Awareness for Nigerian Women LGA Local Government Area LHI Life Helpers Initiative MAJI Media Awareness and Justice Initiative MDA Ministry, Department, and Agency M&E Monitoring and Evaluation MEL Monitoring, Evaluation, and Learning NBA Nigeria Bar Association NDI National Democratic Institute NGO Non-Governmental Organization NIGAWD The New Generation Girls and Women Development NIPSS National Institute for Policy and Studies NISD New Initiative for Social Development NOA National Orientation Agency 4 NTYTR Not Too Young to Run NUPAS Non-US Organization Pre-Award Survey OPA One Project Afrika - Reach Out Foundation PDG USAID Peace and Democratic Governance Office PIA Participatory Institutional Analysis PIBCID Participation Initiative for Behavioral Change in Development PREO Pre-Election Observation PWD Persons With Disabilities PMI Pilot Media Initiative PVT Parallel Vote Tabulation REC Resident Electoral Commissioner SDA Save Democracy Africa SFCG Search for Common Ground SFP State Focal Point Stop-VAWIE Stop Violence Against Women in Election Stop-VAWIP Stop Violence Against Women in Politics TMG Transition Monitoring Group UNDP United Nations Development Programme USAID United States Agency for International Development VAPP Violence Against Persons Prohibition VNF Vote Not Fight VSA Voice, Space, and Accountability WACOL Women's Aid Collective WACPAF War Against Children and Physically Abuse Foundation WFD Westminster Foundation for Democracy WPA Women’s Policy Agenda WTV #Watching the Vote project WL World Learning YIAGA Youth Initiative for Advocacy, Growth & Advancement YDI Young Stars Development Initiative 5 TABLE OF CONTENTS ACRONYMS 2 TABLE OF CONTENTS 5 EXECUTIVE SUMMARY 6 1.1 ACTIVITY BACKGROUND, RATIONALE, AND OBJECTIVES 8 2. EVALUATION OBJECTIVES AND PURPOSE 11 2.1 EVALUATION OBJECTIVES 12 2.2 EVALUATION QUESTIONS 12 3. EVALUATION METHODOLOGY AND LIMITATIONS 12 3.1 EVALUATION DESIGN 12 4. DATA COLLECTION AND ANALYSIS 14 4.1 DOCUMENT REVIEW 14 4.2 KEY INFORMANT INTERVIEWS 15 4.3 FOCUS GROUP DISCUSSIONS 16 4.4 SURVEY OF PARTICIPANTS IN ACTIVITY TRAINING 16 4.5 DATA ANALYSIS 17 4.6 LIMITATIONS AND MITIGATION APPROACH 18 5. EVALUATION QUESTION NO. 1 19 5.1 FINDINGS 19 5.2 CONCLUSIONS 30 5.3 RECOMMENDATIONS 31 6. EVALUATION QUESTION NO. 2 31 6.1 FINDINGS 32 6.2 CONCLUSIONS 34 6.3 RECOMMENDATIONS 35 7. EVALUATION QUESTION NO. 3 36 7.1 WOMEN’S CAMPAIGN FINDINGS 37 7.2 POSITIVE FACTORS OF WOMEN’S CAMPAIGNS 39 7.3 BARRIERS FOR WOMEN’S GROUPS 40 7.4 CONCLUSIONS 41 7.5 RECOMMENDATIONS 42 7.6 FINDINGS ON YOUTH CAMPAIGNS 43 7.7 CONCLUSIONS 46 7.8 FINDINGS ON OTHER CAMPAIGNS 48 7.9 CONCLUSIONS 51 7.10 RECOMMENDATIONS 52 8.0 EVALUATION QUESTION NO. 4 53 8.1 SUSTAINABILITY OF INSTITUTIONAL CAPACITY OF PARTNERS BEYOND LIFE OF EECSP 53 8.2 SELF-SUSTAINABILITY WITHOUT DONOR SUPPORT 54 8.3 RECOMMENDATION FOR A NEW CIVIL SOCIETY ELECTORAL ACTIVITY 54 ANNEXES 57 ANNEX A: EVALUATION STATEMENT OF WORK 57 ANNEX B: OUTCOME MAPPING 68 ANNEX C: DATA COLLECTION INSTRUMENTS 69 ANNEX D: SOURCES OF INFORMATION/LIST OF RESPONDENTS 83 ANNEX E: DISCLOSURE OF ANY CONFLICT OF INTEREST 84 ANNEX F: SUMMARY INFORMATION ON THE EVALUATION TEAM 88 6 EXECUTIVE SUMMARY The evaluation team examined outcomes achieved by the Electoral Empowerment of Civil Society Program (EECSP), factors that led to their achievement, the role EECSP played, and the challenges that may have affected program performance. EECSP, a six-year activity to strengthen the capacity of civil society organizations (CSOs) to become integral parts of election activities in Nigeria, has three objectives: 1) To improve impartial voter observation in 2015 and beyond; 2) To improve management systems within Nigerian CSOs; and, 3) To increase the participation of marginalized groups in Nigerian political processes. The National Democratic Institute (NDI) is the lead implementer of the EECSP consortium that also includes World Learning (WL), Search for Common Ground (SFCG), and the Center for Democratic Development￾Ghana (CDD-Ghana) as partners. The evaluation report provides insights to assess the extent to which: 1) Parallel vote tabulations (PVTs) and pre-election observations (PREOs) influenced impartial observation, accurate reporting, and recording of vote totals; 2) NDI partners’ management skills improved PVT deployment and campaigns toward marginalized groups; 3) Marginalized groups participated in Nigerian political processes because of campaigns; and, 4) Institutional capacities can be sustained after the end of EECSP. The evaluation used a mixed-methods approach that consisted of a desk review, 61 key informant interviews (KIIs) with stakeholders, focus group discussions (FGDs) with 90 participants, and a survey of election observation trainees. The evaluation design applied the outcome-harvesting method to map EECSP outcomes identified by USAID/Nigeria and trace the factors that led to the outcomes using the various data collection tasks in the mixed-methods approach. The findings related to the first evaluation question revealed that effective PVT training-of-trainers activity, CSO commitment, organizational capacities built over the years, acceptance of PVT by the Nigerian elections commission (INEC), and spillover effects enhanced the credibility and integrity of election outcomes due to the rigorous methodology, extensive training, observer recruitment, and cascade training that led to the successful deployment of PVTs and PREOs. Impediments to successful PVT and PREO deployment included security threats around the country, infrastructure challenges, environmental factors that made access difficult to some areas, lack of adequate internet connectivity in some areas, poor INEC accreditation systems, lack of commitment from some partners, and insufficient outreach regarding PVTs. These findings led to the conclusions that although PVTs and PREOs served to increase the credibility of election results, they ought to have made stronger efforts to disseminate PVT methods for broader acceptance. Another conclusion was that NDI helped key CSOs, Transition Monitoring Group (TMG), and Youth Initiative for Advocacy, Growth & Advancement Africa (YIAGA Africa) to deploy PVTs and contribute to electoral law reforms and procedural changes within INEC. On the second evaluation question regarding improvements in management systems by NDI partners, the evaluation found that training and capacity building through EECSP improved partners’ management systems. The improvements were due to funding support, local collaboration and ownership, leadership buy-in among the partners, staff capabilities and openness to learning, presence of resource networks, and synergies with capacity building in advocacy formation and programming. The impediments to this particular EECSP objective included the timing and scheduling of training and follow-up activities, as well as perceived micromanagement among some grantees’ staff by the implementing partner. 7 In response to the third question on inclusiveness, the evaluation team explored strengthened participation of marginalized groups through EECSP-supported campaigns. For women’s groups, the evaluation found that NDI advocacy training sessions combined with capacity building in management systems were effective in strengthening CSO operations. CSOs also found that the technical assistance NDI provided in advocacy strategy and campaign development helped their programming efforts. CSOs coordinated actively among themselves, which helped sustain advocacy efforts. The evaluation found strong representation of women in local governance and public financial management at the state level, as well as individual champions, both men and women. Impediments to the Stop Violence Against Women in Politics (Stop-VAWIP) campaign and participation of women’s groups included the late delivery of funding and other support, the short timelines of support just around elections, and the persistent cultural practices that suppress women. For youth groups, the evaluation team found that the Not Too Young to Run (NTYTR) campaign contributed to the constitutional amendment on age reduction for elective office, although, some respondents felt the limitations of young people in the electoral process is due more to economics, that the high cost of elections excludes younger, less affluent people. The “Vote Not Fight” campaign contributed to youth’s increased confidence and interest in the elections, as they felt seen and valued. The evaluation also found that focused messaging had an impact on youth voter education and peaceful participation in elections. Other factors contributing to successful youth campaigns include the signing of peace pledge cards by candidates and young people across the country, social media use, and the formation of loose but committed organizations. These factors led to an increase in youth candidacy in the 2019 general elections and to the development of youth-policy agendas. Other marginalized groups have also benefited from EECSP initiatives: groups supporting persons with disabilities (PWD) observed greater advocacy and participation which contributed to providing quantitative evidence on PWD election day participation barriers; the INEC amendment of election day form 40H to disaggregate PWD data by type of disability; the INEC pilot of election day written instructions for deaf voters (Form EC 30D PWD) during the Ekiti 2018 governorship election; and, in the Osun 2018 governorship elections, Braille ballot guides employed for the first time in a Nigeria election. The sustained advocacy of IFA and other PWD groups led to the assent of the Discrimination against Persons with Disabilities (Prohibition) Act 2019 on January 23, 2019 by Nigeria’s President, Muhammadu Buhari. Groups supporting internally displaced people (IDPs) were also more committed to greater election participation, even though respondents observed that INEC fell short of providing full support to IDPs. All evaluation findings and conclusions point to the following recommendations: First, a new CSO activity should consider expanding partnerships to include informal institutions and community-based organizations (CBOs) for sustained impact. Second, a new CSO activity should expand local capacity through the use of local experts to build institutional capacity with local subgrantees where it is the best available option. Third, USAID/Nigeria should develop election and political sensitization interventions targeting youth and women. Future USAID activities should continue to support programming and campaigns advocating for democratizing political parties’ processes to ensure equity and fairness, especially for PWDs, youth, and women. Campaign support strategies should include tangible plans for the CSOs to deepen or scale impact post-donor support. USAID/Nigeria and all stakeholders should align with local traditional and religious structures to garner greater support, trust, and influence. 8 1. INTRODUCTION 1.1 ACTIVITY BACKGROUND, RATIONALE, AND OBJECTIVES 1.1.1 Activity Background On April 1, 2014, USAID awarded Cooperative Agreement No. AID-620-A-14-00003, with an initial budget of $24 million and a five-year period of performance, to the National Democratic Institute (NDI) as head of the consortium to implement the Electoral Empowerment of Civil Society Program (EECSP). The consortium, composed of NDI, World Learning (WL), Search for Common Ground (SFCG), and the Center for Democratic Development-Ghana (CDD-Ghana), is currently working nationwide in Nigeria with targeted interventions in selected states. The EECSP activity’s overall goal is to promote sustainable long-term civil society engagement in electoral and political processes. USAID extended EECSP for one year, from April 1, 2019 to March 31, 2020, for a total of six years, with an increase in funding of $6.6 million, bringing the total award to $30.6 million, of which $11.7 million was to be contributed by the United Kingdom’s Department for International Development (DFID). During this time period, EECSP has supported civil society organizations (CSOs) that have improved in capacity to the point of leading successful efforts in increasing participation in the Nigerian electoral process. These efforts include: empowering lawmakers to enact legislation to decrease the minimum legal age for Nigerians to run for office; promoting the use by the Independent National Electoral Commission (INEC) of tactile ballot guides to increase the participation of persons with disabilities (PWDs); and monitoring efforts in seven off-cycle gubernatorial elections as well as the 2019 general elections. EECSP is due to conclude March 31, 2020. The period of performance covered by this evaluation is from the EECSP’s start date of April 1, 2014 to the evaluation’s start date of August 28, 2019. 1.1.2 Activity Goals and Objectives The goal of the EECSP is to strengthen the capacity of CSOs to become integral parts of election activities in Nigeria. The objectives of the EECSP activity are: 1. Improve impartial observation, accurate reporting on, and recording of election vote totals in 2015 and beyond; 2. Improve management within Nigerian partner CSOs; and 3. Increase participation by marginalized groups in Nigerian political processes. EECSP’s goals and objectives are grounded on NDI’s voice, space, and accountability (VSA) theory of change that participation enhances the VSA in the democratization process. 1.1.3 Activity Strategy and Approach To respond to the immediate challenges of the 2015 election and promote sustainable long-term CSO engagement in electoral and political processes, the EECSP consortium has been implementing this six-year initiative to: 1. Improve impartial observation, accurate reporting on, and recording of election vote totals by domestic electoral observers during and after the 2015 election cycle; 2. Improve management within Nigerian partner CSOs; and 3. Increase participation by marginalized groups in Nigerian political processes. 9 NDI is the consortium lead and the primary implementer, responsible for all aspects of the program. NDI collaborates with specialized partner organizations, including CDD-Ghana for Objective 1 and WL and SFCG for Objective 3. The program is divided into four phases to promote iterative learning as the EECSP consortium decreases its role over time. Under Objective 1, the Transition Monitoring Group (TMG) successfully conducted a Parallel Vote Tabulation (PVT), or Quick Count, of the 2015 presidential election and independently verified INEC’s official results, bolstering citizen confidence in the electoral process. TMG trained and deployed 3,014 Quick Count observers to a representative random sample of 1,507 polling units and collected more than 125,000 individual data points on the election process via 20,000 coded text messages. With the 2015 Quick Count complete, TMG prepared a draft report on its election day PVT and pre-election observation (PREO) effort, which was finalized and distributed. TMG also hosted a roundtable with key media actors to discuss the Quick Count and possible areas for electoral reform. TMG carried out Quick Count observations of the 2015 Kogi and Bayelsa off-cycle gubernatorial elections and each state’s supplemental elections. Ahead of the elections, NDI and CDD-Ghana trained TMG and assisted the network to step down PVT training to its Quick Count project structures. NDI also supported TMG to deploy 34 roving observers and conduct general observation of the Federal Capital Territory (FCT) council elections. NDI also produced and began testing a guide for observing political party primaries ahead of the Edo State off-cycle gubernatorial elections. Following a national symposium on elections in September 2015, NDI issued an operational subgrant to Youth Initiative for Advocacy, Growth & Advancement (YIAGA) Africa to implement a PVT of the Ondo State off-cycle gubernatorial election on November 26, 2016. After the successful deployment in Ondo, NDI selected YIAGA Africa through a second competitive application process, held partnership-building activities, and collaboratively developed a new deployment and training approach for PVTs in Nigeria. NDI supported YIAGA Africa to employ the PVT election observation methodology during gubernatorial polls in Anambra (November 2017), Ekiti (July 2018), and Osun (September 2018). YIAGA Africa, under their #WatchingTheVote (WTV) project, also successfully constituted a nationwide election observation structure with a presence in all 774 local government areas (LGAs) observing the February 23, 2019 Presidential and National Assembly elections. Under Objective 2, and in coordination with NDI staff in Abuja, WL conducted a participatory institutional analysis (PIA) with TMG in 2015 and Non-U.S. Organization Pre-Award Survey (NUPAS) for Life Helpers Initiative (LHI), Youngstars Development Initiative (YDI), YIAGA Africa, Inclusive Friends Association (IFA), and Centre for Citizens with Disabilities (CCD) in 2016. The assessment uncovered major weaknesses in the partners’ operational systems and triggered WL’s necessary intervention to address weaknesses in the areas of: • Non-functional organizational boards of trustees; • Centralized decision-making processes driven mostly by the Executive Director; • Financial management policies and practices and procurement processes; and • Monitoring and evaluation (M&E) systems. In 2016, WL’s role was redefined to provide support on communications, while NDI handled financial management institutional capacity development for the sub-grantees to KPMG. KPMG is: • Conducting periodic oversight of EECSP sub-grantees, including unannounced visits to test the partners’ financial transactions and verify their compliance with NDI agreement terms, policies and procedures; 10 • Undertaking forensic verification of documentation to ensure the integrity and quality of sub￾grantee accounting information and fraud prevention; and, • Making recommendations to subgrantees to remedy gaps in partners’ financial management. Under Objective 3, NDI has been working with IFA, YIAGA Africa, YDI, and other state-level partners to: • Conduct advocacy and facilitate public discourse around electoral policy and political decision￾making processes of importance to the marginalized groups that they represent; • Advance their participation in elections; and • Integrate electoral violence-mitigation strategies and voter education into their activities to promote peaceful gubernatorial polls in Kogi (2015), Bayelsa (2015), Edo (2016), Ondo (2017), Anambra (2017), Ekiti (2018), and Osun (2018), and during the 2015 and 2019 general elections. NDI’s technical interventions were informed by its baseline assessment and grounded in NDI’s VSA theory of change. VSA rests on the intrinsic right of all citizens to be heard and the power of organized, informed citizen action to drive democratic change. NDI understands that the constraints for citizen participation may be affected by multiple and simultaneous dimensions, including age, class, ethnicity, religion, disability and sexual orientation, and that gender norms and inequities cut across these constraints. 1.1.4 Key Stakeholders The EECSP consortium, led by NDI, is expected to achieve its objectives through collaborations with CSOs in Nigeria and include the following. Under Objective 1: • National and off-cycle gubernatorial PVTs: TMG and YIAGA Africa Under Objective 2: • Organizational capacity building: WL, Development Alternatives and Resource Center, and KPMG Under Objective 3: • Access Nigeria campaign: IFA • Misinformation and hate speech: CDD-West Africa • Vote Not Fight (VNF) campaign: YDI • Countering dangerous speech: Center for Information Technology and Development (CITAD) • Youth Advocacy: KDI, Media Awareness and Justice Initiative, and One Project Afrika; • Stop Violence Against Women in Politics: Gender and Development Action (GADA) and Legal Awareness for Nigerian Women (LANW); • Internally displaced person (IDP) voter education: Community Links and Human Empowerment Initiative and Forward in Action Education Poverty and Malnutrition. Table 1 below shows the EECSP partners engaged under the three objectives of the EECSP activity. 11 TABLE 1: EECSP PARTNERS No. SUB-GRANTEE CAMPAIGN/PROGRAM LOCATION 1 Life Helpers Initiative (LHI) Women in political process Sokoto 2 Youngstars Development Initiative (YDI) Youth, VNF Abuja 3 YIAGA Africa PVT & PREO Abuja 4 Inclusive Friends Association (IFA) PWD, Access Abuja 5 Centre for Citizens with Disabilities (CCD) PWD, Access Lagos 6 Transition Monitoring Group (TMG) PVT & PREO Abuja 7 Hope for Family Development Initiative (HFDI) Women, Stop Violence against Women in Politics (Stop-VAWIP) Osun 8 Save Democracy Africa (SDA) Good governance Abuja 9 New Initiative for Social Development (NISD) Women in political process Ekiti 10 Kimpact Development Initiative (KDI) Youth, VNF Osun 11 The New Generation Girls and Women Development (NIGAWD) Youth, VNF Ekiti 12 Media Awareness and Justice Initiative (MAJI) Youth, VNF Rivers 13 One Project Afrika - Reach Out Foundation (OPA) Youth, VNF Kaduna 14 Gender and Development Action (GADA) Women, Stop-VAWIP Rivers 15 Legal Awareness for Nigerian Women (LANW) Women, Stop-VAWIP Kaduna 16 Center for Community Health and Development (CHAD) IDPs Gombe 17 Community Links and Human Empowerment Initiative (CLHEI) IDPs Benue 18 Pilot Media Initiative (PMI) Youth, VNF New York 19 Face Initiative VAPP, Youth, VNF Bayelsa 20 Women's Aid Collective (WACOL) VAPP Enugu 21 Gender Awareness Trust (GAT) VAPP Kaduna 22 Center for Democracy and Development (CDD-West Africa) Disinformation Abuja 23 Center for Information Technology and Development (CITAD) Hate Speech Kano 24 Participation Initiative for Behavioral Change in Development (PIBCID) Youth, VNF Kogi 25 Challenged Parenthood Initiative (CPI) Women, Stop-VAWIP Kogi 26 Girl Child Empowerment & Reproductive Health Initiative (GREP) Women, Stop-VAWIP Bayelsa 2. EVALUATION OBJECTIVES AND PURPOSE The evaluation examined outcomes EECSP achieved, the factors that led to their achievement, and the role EECSP played in catalyzing or contributing to those outcomes, as well as the challenges that may have affected performance in achieving outcomes. The ultimate purpose of the evaluation is to inform future programming. 12 2.1 EVALUATION OBJECTIVES Specifically, this performance evaluation developed a methodology that provided answers to the key evaluation questions (below) that focused on the following areas: • How effective the identified outcomes - and others that may have been identified in interviews - were in addressing the three objectives; • What contributions from EECSP, and outside of EECSP, led to those outcomes; • What key impediments to achieving the outcomes existed; • How sustainable the institutional management capacity of partners is (Objective 2) beyond the life of EECSP; and, • How self-sustainable the efforts undertaken by the CSOs will be without donor assistance. 2.2 EVALUATION QUESTIONS Evaluation Question No. 1: To what extent has the election observation mechanism (PVT and pre-election observations (PREOs)) adopted by the activity influenced impartial observation, accurate reporting, and recording of vote totals (April 1, 2014 to-date)? • What factors led to the election observation outcomes and what were the key impediments to achieving the outcomes? • What is the perception of stakeholders, particularly CSOs, regarding the utility of the PVT? What can be done differently? Evaluation Question No. 2: To what extent have NDI partners’ management skills (Objective 2) influenced their capacity to implement the PVT and campaigns towards increased participation by marginalized groups in Nigerian political processes (April 1, 2014 to-date)? • What factors led to the key outcomes achieved in improving management within partner CSOs, and what were the key impediments to achieving the outcomes? • Are partners (particularly the PVT subgrantee) implementing the new skills or recommendations from NDI’s capacity assessment? Evaluation Question No. 3: To what extent have the campaigns of marginalized groups’ (women, youth, PWDs, and IDPs) affected their participation in Nigerian political processes (April 1, 2014 to-date)? • Specifically, which of the marginalized groups’ campaigns have been successful and which were not? • What factors led to the campaign outcomes and what were the key barriers to achieving successful campaigns? Evaluation Question No. 4: To what extent can the developed institutional management capacity of partners be sustained beyond the life of the activity? • How self-sustainable are the efforts undertaken by the CSOs without donor assistance? • Make recommendations to USAID/Nigeria regarding the design of a new civil society elections activity. 3. EVALUATION METHODOLOGY AND LIMITATIONS 3.1 EVALUATION DESIGN This evaluation used a mix of qualitative and quantitative approaches to gather data on key program states where EECSP operates. The evaluation team (ET) conducted focus group discussions (FGDs) and key informant interviews (KIIs) in 12 states: Anambra, Bayelsa, Benue, Ekiti, Enugu, Kaduna, Kogi, Lagos, Ondo, Osun, Rivers, Sokoto, and the FCT. The ET selected these states based on the program 13 outcomes achieved by the EECSP partners and the roles CSOs played in achieving outcomes. The ET did not visit Bayelsa, Akwa Ibom, Ekiti, Kano, Sokoto, Gombe, and Adamawa for security reasons and the relatively lower levels of programming activity. Table 2 presents the distribution of KII and FGD participants by sex, region, and state. TABLE 2: DISTRIBUTION OF KII AND FGD PARTICIPANTS REGION STATE KIIs FGDs Male Female Male Female North Central Abuja/FCT 23 8 26 11 Benue 2 1 4 2 Kogi 3 1 5 1 North West Kaduna 1 3 1 3 Sokoto 1 0 1 4 South West Osun 2 0 4 8 Lagos 5 1 0 0 Ondo 2 0 0 0 Ekiti 2 1 0 0 South East Enugu 1 0 1 3 Anambra 1 0 0 0 South South Rivers 1 1 4 9 Bayelsa 0 1 0 3 The ET incorporated outcome harvesting within the overall evaluation design. Outcome harvesting is a particularly suitable method for this evaluation, because it provides an opportunity to understand past performance. It is a method that identifies the changes that have occurred in an activity to-date, and then helps track those changes back to their causes and related factors (Simister and Napier, 2017). For the outcome harvesting in this evaluation, the team received the 23 EECSP program outcomes identified by USAID/Nigeria and aligned each one with the appropriate evaluation question so that any information or finding the team collected can be used to validate how a particular outcome was achieved or adversely affected. Annex C presents the finalized questions for all KIIs, FGDs, and the survey, and details how each question is linked to the broader evaluation questions and to the 16 program outcomes. The ET decided not to include seven outcomes; Table 3 presents the rationale for doing so. TABLE 3: EECSP OUTCOMES EXCLUDED FROM THE EVALUATION DESIGN No. Outcome Excluded Note 1. Successful PVTs Considered redundant, as other outcomes would allow data collection to produce more detailed descriptions of successes and factors behind it. 2. National Orientation Agency (NOA) identified for underperformance in its voter education responsibilities leading to its increased performance. Does not explicitly help to understand what is intended and NOA did not come up significantly as a role player under the objective. It was noted that NOA may be interviewed if there is sufficient time. 14 3. Improved and diversified communications practices by CSO partners Considered ambiguous. Also, corresponding outcomes under this objective probed deeper into improved organizational practices like human resources, procurement, governance, etc.; all these also produce information on communication. 4. Increased youth participation in elections through peacemaking efforts, youth voter registration and PVC collection (VNF campaign) Increased and diversified PWD participation in elections through the Access Nigeria: Disability Votes Matter Campaign. Increased IDP voter participation in 2019 in at least one of two states of focus. Increased women’s participation in elections and politics (Stop-VAWIP campaign; VAPP Act campaign; and Sokoto women’s campaign) These four outcomes were merged into one, as they all focused on increased participation through the various campaigns: Increased participation of marginalized groups in elections and political process (Youth - VNF, PWDs, women and IDPs). More questions were framed to collect specific group data. 4. DATA COLLECTION AND ANALYSIS This section includes a detailed description of the data-collection methods used in this evaluation. 4.1 DOCUMENT REVIEW The ET conducted the document review in two phases: the pre-field documents and literature review (Phase I), and the continued review of documents collected during the field activities (Phase II). The goal of this two-phased approach was to make the document review as comprehensive as possible while anticipating that previously unidentified documents might become available during data collection in the field. The pre-field document and literature review included activity monitoring and evaluation and learning plans (AMELPs) as well as reviews of the literature on the grantees. The Phase II document review during field activities included documents that became available to the ET. The ET used much of the qualitative data reflected in these documents to analyze the effectiveness of the PVT and PREOs on election credibility, the effect of NDI and its partners’ intervention on the institutional capacity of the stakeholders involved in the activity, and the increased capacity of marginalized groups in electoral processes. The pre-field documents reviewed include: • EECSP AMELPs; • EECSP Quarterly and Annual Reports; • EECSP After Action Review (AAR) Reports; • EECSP Statement of Work; • All documents listed in the Pre-Fieldwork Literature Review; 1 1 The evaluation work plan the ET submitted to USAID/Nigeria provides a comprehensive list of all pre-field work documents. 15 • TMG and YIAGA Africa PVT statements for Ekiti 2014, National elections (March 2015 and February 2019), as well as gubernatorial elections for Kogi (2015), Bayelsa (2015), Ondo (2016), Anambra (2017), Ekiti (2018), Osun (2018), and national elections 2019 (February 2019); • TMG and YIAGA Africa pre-election observation statements 2015–2019; • Youths and the 2019 Elections in Nigeria, a February 2019 YIAGA Africa publication; • How Youth Fared in the 2019 Elections, a June 2019 YIAGA Africa report; and, • NDI’s publication on Parallel Vote Tabulations. 4.2 KEY INFORMANT INTERVIEWS KIIs served as the central data-collection method for this evaluation and were used both to deepen the ET’s understanding of the project development and activities and to obtain expert opinions on the success of activities and progress toward objectives. There were eight major groups of key informants that were interviewed for this evaluation: 1. USAID Peace and Democratic Governance Office (PDG) staff; 2. EECSP consortium program staff (NDI, CDD-Ghana, WL, SFCG); 3. Partner CSOs (e.g., TMG, YIAGA Africa, IFA, CDD-West Africa, YDI, CITAD, Kimpact Development Initiative (KDI), Media Awareness and Justice Initiative (MAJI), One Project Afrika (OPA), GADA, and Legal Awareness for Nigerian Women (LANW), Community Links and Human Empowerment Initiative (CLHEI), and Forward in Action Education Poverty and Malnutrition, among others); 4. Media; 5. CSOs such as PLAC/Situation Room and CDD-West Africa; 6. Donors such as DFID and USAID, implementing partners that include International Federation for Electoral Systems (IFES) and International Republican Institute (IRI); 2 7. Government of Nigeria (GON) key stakeholders, notably the INEC and the Salama Survival Center; and, 8. Labour Party members selected based on their availability and the Inter Party Advisory Council. The ET completed a total of 61 KIIs from the eight major respondent groups as presented in Table 4 below. The interviews were conducted from September 19 to October 3, 2019 and fieldwork was conducted outside Abuja from September 22 to 28, 2019. To conduct KIIs, the ET developed a semi-structured questionnaire with open-ended questions. The team structured the questionnaire to gather responses to the four evaluation questions, with sub￾questions to explore relevant issues, such as activity outcomes, successful factors that contributed to outcomes, impediments or barriers experienced, opportunities for sustainability, and suggested areas for future programming. All sub-questions were mapped to 16 out of the 23 EECSP outcomes collected by USAID/Nigeria. This ensured the outcome-harvesting approach was integrated into the KIIs by identifying the factors 2 For technical reasons, the ET only met with USAID and DFID among the donor groups, since they have been the two largest funding supporters of EECSP. For operational reasons, the ET had a limited field schedule to conduct in-country interviews. 16 and impediments that led to the activity outcome. Annex C1 provides the final version of the KII questionnaire used for this evaluation. TABLE 4: DISTRIBUTION OF KIIs BY STATE OR AREA State/Area KII Respondents State/Area KII Respondents Abuja/FCT 29 Kogi 4 Anambra 1 Lagos 6 Bayelsa 1 Ondo 2 Benue 3 Osun 2 Ekiti 3 Rivers 2 Enugu 1 Sokoto 1 Kaduna 4 International 2 Total 61 4.3 FOCUS GROUP DISCUSSIONS TABLE 5: DISTRIBUTION OF FGD RESPONDENTS BY STATE OR AREA State/Area FGD Participants State/Area FGD Participants Abuja/FCT 37 Kogi 6 Anambra – Lagos – Bayelsa 3 Ondo – Benue 6 Osun 12 Ekiti – Rivers 13 Enugu 4 Sokoto 5 Kaduna 4 International – Total 90 The ET completed 16 FGDs, which included 90 participants. The participant group had an even representation; 44 participants were female and 46 were male. The FGD discussion agenda followed the structure of the KII questionnaire, but the ET revised some questions depending on the group profile. For example, FGDs with political parties did not include inquiries on Objective 2 tasks or outcomes. Table 5 above presents the distribution of FGD by state; Annex C2 includes the finalized FGD discussion agenda. 4.4 SURVEY OF PARTICIPANTS IN ACTIVITY TRAINING A fourth data-collection source was an online survey of Objective 1 trainees from the WTV campaign for the 2019 elections. NDI provided the sample list of 90 respondents from their database of Objective 1 beneficiaries. Figure 1 shows the distribution of respondents by their group’s affiliation. This shows a significant representation of youth groups, with 81 percent of respondents 17 stating they belong to one. Respondents from groups supporting IDPs were the least-represented, with just a two-percent share of the survey sample. FIGURE 1: DISTRIBUTION OF RESPONDENTS BY GROUP AFFILIATION The ET distributed the questionnaire to the email addresses of the 90 respondents and received 42 responses. The ET would state the following considerations for this online survey, particularly in relation to what was envisioned in the evaluation work plan. First, following conversations with NDI, the list of 90 individuals is not an exhaustive register of Objective 1 trainees. It is only a list in the Apollo database of trainees that included email addresses. Second, even if complete, 42 completed surveys do not make representative results. Using a 95 percent confidence level and five percent confidence interval as standard parameters, the survey needed 74 responses to present representative findings. Therefore, references to survey results throughout this report should be considered to apply to the respondent sample only, i.e., the 42 respondents, and not as representative of any other group. Additional information on the online survey can be found in Section 4.6: Limitations and Mitigation Approach. 4.5 DATA ANALYSIS The team conducted an analysis of all qualitative data and relied on two primary methods: 1. Tabulation of qualitative results. The team constructed a matrix of main achievements of the activity based on different information sources and verified claims supported by cross￾referencing them against other available data sources. The ET used the matrix to determine frequencies of responses and the substance or context of this evidence to determine the degree to which each achievement could be proven. 2. Cross-verification of data. The ET anticipated that data analysis will present competing and even conflicting claims. For each statement derived from the qualitative data, the team triangulated evidence in each of the sources of qualitative data—literature/document review, KIIs, and secondary data sources—in addition to further probes or inquiries from the source(s) of the data. For the quantitative data collected from the survey, the ET conducted a range of analyses to obtain findings that answer the evaluation questions. The analyses largely ranged from statistical summaries of relevant questions/indicators to crosstabs of relevant variables. Given the data collected from the survey, the ET considered these methods to be the most appropriate for the analysis. Women, 30.95% Youth, 80.95% PWD, 14.29% IDP, 2.38% Other, 16.67% 18 4.6 LIMITATIONS AND MITIGATION APPROACH 1. Forming sub-teams to conduct field work. Given the broad coalition of sub-grantees in the identified states, the ET was divided into three sub-teams to cover the North Central, North West, South East, South West, and South-South regions to complete as many interviews and discussions as possible. The ET conducted data collection activities in the field outside Abuja/FCT over the period September 22–28, 2019. Each core team member was supported by Field Assistants from the DevTech MEL Activity office to provide logistics and data-collection support. This approach mitigated any concerns about reaching enough stakeholders, as 61 KIIs were completed out of the targeted 71 in the work plan. The 10 planned KIIs that were not conducted were due to various circumstances on the interviewees’ side, such as not responding to the team’s meeting requests and respondents double-booking appointments even after they confirmed meetings with the ET. 2. Changing online survey respondents’ sample. The limitation is that the ET, as stated in the work plan, proposed to draw a representative sample from a population list of Objectives 1 and 2 trainees, but was only able to survey WTV trainees, because NDI only provided the email addresses of WTV trainees. The ET thus did not receive all the email addresses of Objective 2 capacity assessment beneficiaries. It is important to note that the obtained sample is not what was envisioned in the work plan and is not representative of the total population of individuals that received EECSP assistance across all three activity objectives (instead, only those with Objective 1 support). Therefore, as mentioned previously, the survey results presented in this report should be weighed only in the context of the surveyed sample. The overall mixed-methods approach met the first evaluation objective to “develop a methodology that will provide answers to the evaluation questions.” All data-collection instruments also included inquiries to gather stakeholders’ perceptions regarding areas for future programming to meet the first evaluation objective (Figure 2 below). FIGURE 2: EVALUATION DESIGN 19 5. EVALUATION QUESTION No. 1 Evaluation Question No. 1: To what extent have the election observation mechanisms (PVT and PREO) adopted by the activity influenced impartial observation, accurate reporting, and recording of vote totals (April 1, 2014 to date)? To enhance electoral integrity, Nigerian CSOs observe elections and report electoral processes such as voters’ registrations, party primaries, campaigns, and the deployment of election materials and personnel. They also conduct voter education and sensitization activities. In terms of election observation, PREOs are undertaken by long- and short-term election observers to examine the pre￾election environment which includes political party activity, INEC preparedness, inclusion of marginalized populations, and incidents of electoral violence. In addition to PREO reports, election￾day observers are deployed to observe the various stages of voting from opening of the polls to collation. Traditional election observation involves deployment of thousands of accredited observers, some of whom are assigned as stationary observers in particular polling units and are complemented by roving observers. Some of the CSOs in Nigeria that conduct election observation include the TMG, Nigeria Civil Society Situation Room, CDD-West Africa, Nigeria Women Trust Fund (NWTF), and YIAGA Africa. Ahead of Nigeria’s 2011 General Election, NDI introduced a PVT known as Project Swift Count in 2010. NDI brought PVT as a new innovation because of its scientific, data-driven election-observation methodology. Building on traditional observation, PVT is a form of election observation that draws a random and representative sample of polling units which can make independent, generalizable statements about the overall process and results. As such, a PVT acts as an independent means of assessing an election. It can either independently validate the official election results and increase public confidence or identify issues on an objective basis and provide a path to peaceful resolution of complaints and improvements. In Africa over the last ten years, the PVT methodology has been deployed in Ghana, Kenya, Malawi, Uganda, and Zambia. NDI collaborated with TMG, Justice and Development Peace Commission (JDPC), Federation of Muslim Women Association of Nigeria (FOMWAN) and Nigeria Bar Association (NBA) to conduct the first PVT in Nigeria in 2011. 3 Under the EECSP activity, the only two organizations that NDI has supported to deploy PREO and PVT methodology are TMG from April 2014 to 2016 and YIAGA Africa from 2016 to date. 5.1 FINDINGS 5.1.1 Factors that Contributed to Outcomes Achieved under Objective 1. The evaluation team identified a number of factors that contributed to the election observation outcomes achieved under Objective 1. 1. Cascading PVT Training and Training of Trainers (TOT) model was effective. Both TMG and YIAGA Africa have adopted the cascading training model to ensure adequate training of their observers and to guarantee quality control. Following their own training, which includes an instructional development course, Master Trainers train newly recruited cadres of observers. For instance, YIAGA Africa has been running PVT through a pyramid structure. The organization has eight Board Members who give policy direction; 15 Watching the Votes Management Teams; 42 Master Trainers; 48 State Focal Points (SFPs)/Collation Observers; 774 Local Government 3 NDI has conducted PVTs and sample-based observations in over 52 countries. See here. 20 Supervisors; 3,030 Observers; and 46 observers for Voting Points. TMG also has similar pyramid structure for training of its observers. FIGURE 3A AND 3B: DISTRIBUTION OF NDI SUPPORT RECEIVED BY SURVEY RESPONDENTS AND PERCENTAGE OF PVT BENEFICIARY RESPONDENTS THAT HAVE ADOPTED NEW PRACTICES * “Other” includes coordinating the state team, logistics, peacebuilding and conflict resolution, and statistics. Most CSO participants who have received PVT training stated in KIIs that the approach was very effective. This finding is supported by online survey results that showed 97 percent of respondents received PVT training (Figure 3a). Of the group that received training, 72 percent stated that their organizations have started adopting the knowledge gained from PVT trainings (Figure 3b). Testifying to the effectiveness of this training model, one of the State Focal Persons for YIAGA Africa stated that; “the PVT was thorough, the training and materials provided the SFP the needed skills to recruit, train, and deploy observers during election day and during the pre-election observation.” 6% 6% 6% 15% 44% 44% 97% 15% 0% 20% 40% 60% 80% 100% Financial management HR Governance (board functions) Decision making Advocacy Communication Election Observation (PVT & PREO) Other (please specify)* a. Distribution of NDI support received by respondents Yes, 72% No, 28% b. Did Your Organization Adopt new Practices as a Result of this Support? 21 2. Organizational footprint was nationwide and successfully carried out the PVTs. TMG and YIAGA Africa conducted multiple PVTs that were executed according to the planned methodology and were therefore largely successful. PVTs that deviated from the methodology or were inconclusive were due to election-related issues (e.g., Osun 2018) and not from an inability by either partner to conduct the tabulation. TMG stated that 250 CSO members spread across the country effectively assisted deploying the PVT in the previous elections. The group has national, state, LGA, and ward structures. YIAGA Africa similarly stated that its network of 42 partner organizations helped to successfully deploy PVT under the EECSP activity. Factors that contributed to the organization’s ability to successfully deploy PVT included: • Sufficient capacity-building by NDI; this capacity was stepped down by the organization • Cooperation and support from INEC • The opportunity to contribute to the validity of outcomes in the “election market.” PVTs provided balance and contributed independent views. At the time of the PVT’s introduction, the trust and confidence of the electorate towards INEC was waning. PVT provided an alternative source of cross-checking INEC’s tabulation outcomes. 3. Capacity of organization built over the years by NDI and sustained by CSOs. TMG and YIAGA Africa succeeded in deploying PVT due to NDI’s ability to provide technical assistance to local groups to build national and state PVT structures and to conduct election-observation training. NDI’s insights and understanding of both the local context and international best practices in developing observer recruitment criteria was another key contributor. NDI’s development and deployment of the Apollo database to offer real-time collation and analysis of observation data also contributed to the capacities built as a management information resource. This capacity increase is reflected in the results of the PVT in the 2015 and 2019 general elections and several off-cycle gubernatorial elections, such as Ekiti 2014, Ondo 2016, Anambra 2017, and Osun 2018, where the results were consistent with INEC’s tabulation; most stakeholder sectors accepted the results of YIAGA Africa’s tabulations. Both YIAGA Africa and TMG affirmed that the institutional strengthening and capacity building support received from NDI assisted them to be able to successfully deploy PVT. According to a TMG interviewee, There was a deliberate attempt by NDI to step down the PVT knowledge to TMG. TMG had the structure to deploy election observers and had a presence in all the states and almost all the LGAs. There was the opportunity to continue learning on the job which enhanced the knowledge on PVT. The PVT training was an advancement for TMG member organizations who were used to traditional election observation. TMG and YIAGA Africa confirmed to the ET that one of the key strengths of EECSP is the ability to transfer skills to local partner organizations like theirs. The online survey suggests there is a demand for this type of training, showing 86 percent of respondents think that local CSOs can deploy PVTs for future elections. It is contingent, however, on having key components. Of the respondents who believe local CSOs can deploy, 16 percent stated it’s only possible if: 1) the technical capacity of internal staff exists; 2) funding is available; and 3) the CSO is large enough to have established communications networks nationwide or at least be able to network with other CSOs. PVT also succeeded due to the EECSP activity’s ability to support local groups to build effective communication and advocacy plans, including CDD Ghana’s support in building the capacity of YIAGA Africa’s Working Group to conduct PVT outreach to electoral stakeholders. 22 YIAGA Africa said its capacity has been built to independently deploy PVT election observation. According to a 2019 AAR on the PVT, NDI with funding support from USAID trained about 23 YIAGA staff during the PVT Academy; however, only about three of their staff claim they have the skills to draw a PVT sample and fully deploy a PVT because they showed commitment to internalize the skills provided by NDI. To show their ability to deploy a PVT, YIAGA Africa said it has started to use PVT ideas to observe the INEC Continuous Voters Registration exercise. On the other hand, TMG stated that despite having received the capacity building and experience spanning about six years in deploying PVT (2011–2016), it cannot single-handedly deploy PVT. 4. INEC’s willingness and acceptance of PVT. INEC’s lack of understanding of the PVT methodology led to its initial resistance to endorsing the election-observation model in 2011. However, over time the Electoral Management Body has endorsed and is now commending the model. All INEC KII respondents were unanimous in stating the value addition of the PVT election￾observation model. According to one respondent, “the PVT results have always matched INEC’s results; given INEC the required credibility, it helped to douse tension. The scientific methodology has helped the country to accept the result.” Another interviewee stated that the constant engagement from TMG and YIAGA Africa led to INEC’s eventual acceptance of the PVT election-observation methodology. According to him, “They engaged us, invited us to programs organized in their capacity, asked questions and made suggestions. The engagements and partnerships were robust and honest.” A Resident Electoral Commissioner (REC) shared that PVTs can address tension and be a source of reassurance. But the REC also stated it may also stir tensions, especially among the uninformed, if there is some degree of variance in its outcome with the official INEC result—even at statistically acceptable levels. Another INEC National Commissioner stated that: (T)here have been qualitative advancements in the methods applied in the electoral process. Before PVT, election observation was manual or traditional and PVTs and observations have impacted positively on elections in Nigeria. The political parties and politicians are careful of rigging in polling units where there are observers. Observers act as a deterrent to rigging and other electoral malpractices. The PVT has helped in stabilizing the electoral process. During the 2015 election, groups who organized PVTs had similar results to the announcement made by the electoral management body and both methods positively impacted the electoral process. However, the drawback is that most persons are still more familiar with the traditional or orthodox methods of election observation and are still trying to come to terms with PVT as a suitable election observation method or if both traditional and PVT methods should be applied. In traditional or orthodox observation methods, the INEC Commissioner stated there has been a “dignified aloofness” where the method is only concerned with the process, while PVT is concerned with the outcome. Although the above statement was made by an INEC National Commissioner, it is worth mentioning that PVT is one component of the YIAGA Africa Watch the Vote campaign that provided insight not only into election outcomes, but also the processes leading to the electoral outcome. PREO reports supplement PVT observation statements, while the PVT checklists used on election day also record the opening of polls, accreditation of voters, sorting and counting, as well as announcement of results. In addition, PVT also tracks a wide range of issues such as security deployment, electoral violence, vote buying, voter turnout, and professionalism of INEC staff. 23 5. Spillover effects include INEC changes to guidelines and regulations. YIAGA Africa’s PVT also contributed to areas beyond election tabulation and into INEC policy formulation. Two INEC National Commissioners stated that YIAGA Africa contributed to some changes in their regulations and guidelines. “YIAGA Africa’s input led to at least 20 percent of the changes made to the regulations and the guidelines.” PVTs have also provided insights on the participation of key marginalized groups (discussed in the context of evaluation questions 2 and 3). 5.1.2 Impediments to Achieving the Outcomes The evaluation team also identified the following impediments – external and internal - to achieving the outcomes. A. External 1. Security: Many PREO and PVT reports by YIAGA either indicted or admonished the security agents on their lack of professionalism in election security. For instance, YIAGA Africa in its preliminary Process Statement on the Anambra 2017 Gubernatorial Election made the following observation: The secrecy of the ballot remains a cardinal feature of democratic elections. Evidence abound that the secrecy of the ballot in this election was undermined in some polling units. This created an opportunity for vote buying across some polling units observed. There were also cases of bribery of election officials. These illegal acts occurred in the presence of security officials who made no arrests or attempts to abate the illegality. YIAGA Africa further observed in its Pre-election Press Statement on the 2018 Ekiti Gubernatorial Election; YIAGA AFRICA has observed instances of overzealous behaviour on the part of security officials deployed for the election. For instance, two of YIAGA Africa's observers were arrested and detained on July 11, 2018, in Ado-Ekiti for no justifiable reason. All security agents on election duty must uphold the principles of non-partisanship, transparency, civil policing and professionalism in the conduct of their affairs. YIAGA Africa calls on security agencies deployed for the election to respect the rights of citizens and observers, including the right to freedom of movement on Election Day for duly accredited observers. From the Verification Statement on the 2018 Ekiti Gubernatorial Election, YIAGA Africa advised: “The Security agencies should devise new strategies for detecting and apprehending individuals or groups involved in vote buying and selling during elections.” YIAGA Africa’s Pre-Election Press Statement on the 2018 Osun Governorship admonished: Watching the Vote would like to reiterate its call that security agents on election duty must uphold the principles of non-partisanship, transparency, civil policing and professionalism in the conduct of their affairs. Watching the Vote calls on security agencies deployed for the election to respect the rights of citizens and observers, including the right to freedom of movement on Election Day for duly accredited observers. Based on the reviews of election and pre-election statements, as well as on the findings by observers from TMG, YIAGA Africa, NDI, INEC, the media, and CSOs, security threats - while manageable for the administration of the PVT - were a major impediment to PVT election observation. A key informant reported that violence and insecurity posed a serious challenge to her group’s ability to deploy PVT. According to her; “The situation was so bad that [colleagues] were calling for insurance coverage in order to be able to be part of the PVT deployment.” An 24 Executive Director of a leading CSO agreed that the security threat posed a serious challenge to the organization’s ability to deploy PVT. Another key informant said; “During the Bayelsa 2015 gubernatorial election, the PVT only provided reports on election-day processes and could not verify the outcome of the election. This was due to widespread security challenges during the conduct of the elections. In some Bayelsa communities, observers were refused access to the polling units. The PVT fails when observers are not allowed access to polling units.” While the Bayelsa 2015 and Osun 2018 elections had security conditions that affected PVT deployment, most observations since 2011 were conducted successfully. 2. Infrastructure Challenges: In a meeting with CSOs on October 30, 2019 at INEC headquarters, the INEC Chairman acknowledged that infrastructure conditions, notably the unreliable power supply and poor road conditions, negatively affected the administration of the election. This finding was also echoed by Professor Ibeanu, INEC Commissioner on Electoral Operations and Logistics, in a roundtable discussion. 3. Environmental Factors: TMG, YIAGA Africa, and NDI affirmed that some environmental factors affected deployment of the PVT. There are some hard-to-reach riverine areas, mountainous locations, or forests where polling units are located, the PVT observers’ access to these places can be problem. According to an INEC National Commissioner, two of the five polling units where the Osun State supplementary governorship election was held were in the forest. In the opinion of an Executive Director for one of the partners that implemented the PVT, the recruitment of observers from some of these hard-to-reach areas only adds to the challenge. 4. Technology, e.g., internet connectivity: While protocols for data transfer in the PVT were devised to address this reality, poor internet connectivity in various locations affected real-time transfer of information from deployed PVT observers to the National Data Centre, according to key informants from TMG, YIAGA Africa, NDI, and INEC. Indeed, INEC said its independent research showed that only about 65 percent of Nigeria has “seamless telecommunication coverage.” 5. INEC Observer Accreditation Policy: According to an NDI respondent; Based on INEC guidelines, observers need to be fully accredited and kitted to observe elections. The deployment of observation kits and tags continued to be a logistics nightmare for the PVT as INEC distributed the materials late and expected every observer to personally collect their accreditation tags and badges from the INEC office. This can be challenging for the PVT deploying 500 observers for state gubernatorial elections and approximately 3,000 observers for presidential PVTs. The respondent further observed that sometimes INEC does not release the final list of polling units - and voting points, where elections will be conducted, on time. B. Internal 1. Staff Attitude and Commitment to Work: Beyond the capacity building provided by NDI to TMG and YIAGA Africa staff, there were internal challenges with knowledge management, staff commitment, and attitude to work. For instance, with YIAGA Africa, a key respondent noted that not all trained staff were willing and committed to utilizing the skills gained from the PVT Academy. Instead, many deferred to the few who had internalized the skills to implement the needed work on the PVT methodology. 25 2. Outreach and Communication: NDI’s 2019 AAR showed that YIAGA Africa took deliberate steps to publicize the PVT methodology and gain public confidence. The efforts included the production of videos and audio on the PVT methodology to show data, what the PVT can do, the capacity of the PVT and projecting the PVT as a reliable source of information. The efforts also included workshops and conferences with media executives on the PVT and equal access to the state and national information centers for political party members, journalists, and candidates. Still, however, there was not enough state-level awareness on the PVT. A key respondent noted that the radio programs were too short and should have been increased beyond the 30-minute slot and suggested more engagement should have been done in the local languages. The low level of awareness led to misunderstandings among some stakeholders and even the general public on what a PVT is about and led to incorrect proclamations about a PVT, such as false statements about how PVT sample sizes are too small to have valid findings. 5.1.3 Perceptions of Stakeholders 1. Regarding the Utility of the PVT: PVT relies on random and statistically representative sampling for its deployment with layers of authentication or verification. Its utility is in its ability to increase citizens’ confidence in the electoral process and reduce potential conflict. Findings from KIIs and FGDs show the PVT, through its statistical foundations, methodological rigor, dedication of observers, and in-depth analyses, contributed to the general acceptance of INEC’s results as it “doused tension.” However, there are some skeptics who still think the sample size is small and question the utility of the PVT. 2. Partner and CSO Perceptions: According to various stakeholders, the PVT and PREO helped to deepen election integrity, served as a conflict mitigation tool, and encouraged citizen participation. The acceptance of the PVT results by stakeholders was made possible by the value of the data generated through the rigorous PVT methodology. One key informant, speaking about the usefulness of PVTs, said: “The international community leveraged the PVT data in issuing statements on the election, the winners of the elections and the Electoral Commission used the data from the PVT. As a conflict-mitigation tool, the PREO helped to reduce incidence of electoral violence and served as an early warning mechanism. It provided the opportunity for the citizens to participate in the electoral process and not as voters alone, but as watchdogs.” NDI staff said PVTs helped detect electoral fraud during the 2015 presidential election. TMG’s Quick Count estimates revealed that turnout was inflated during the collation process in the South-South geopolitical zone by at least 10.8 percent (see Figure 4a). Based on this finding, a former INEC Chairman stated that TMG’s estimates in the 2015 presidential election prompted INEC to investigate collation in the zone. For 2019, YIAGA Africa WTV findings on turn-out estimates did not show any inflated turn-out in South-South compared to 2015 (see Figure 4b). 26 FIGURES 4a AND 4b: TURN-OUT ESTIMATES FOR 2015 AND 2019 IN SOUTH-SOUTH GEOPOLITICAL ZONE The 2019 presidential PVT observed possible voter suppression, based on high rates of cancelled votes (3.3 percent). This was four times higher than the rate in 2015. PVT therefore called on INEC to provide explanations for the high rates of cancelled ballots in Rivers (22.7 percent), Nasarawa (9.5 percent), Akwa Ibom (8.5 percent), Cross River (8.0 percent), Plateau (5.7 percent), Kogi (5.2 percent), Benue (4.9 percent), and Kaduna (4.7 percent) (Figure 5). In the case of Nasarawara State, YIAGA Africa’s February 27th statement on the 2019 Presidential election results concluded that the cancellation of ballots affected the election outcomes for that state, but not the national results. YIAGA Africa called on INEC to provide possible explanations on this outcome. FIGURE 5: YIAGA AFRICA WTV ESTIMATES ON VOTER SUPRESSION BASED ON HIGH RATE OF VOTE CANCELLATION IN THE 2019 PRESIDENTIAL ELECTION 22.70% 9.50% 8.50% 8.00% 5.70% 5.20% 4.90% 4.70% Rivers Nasarawa Akwa IbomCross River Plateau Kogi Benue Kaduna YIAGA Africa #WTV Findings on Vote Cancellation Figure 4a TMG’s Quick Count Turnout Estimates in South-South (2015) Figure 4b 27 According to another CSO interviewed during NDI’s 2019 AAR on the PVT, The PVT helped us in terms of the integrity of the election and also the fact that they were able to deploy both long-term and short-term observers and the fact they were able to use technology in monitoring the results and were able to predict to an extent the outcome of some of their results…What they did is fine in terms of the methodology and I’m not sure there is a problem, but a lot of people will not understand that it was more or less a sampling and it has its own advantages and disadvantages, when people don’t understand your methodology they tend to quote you out of context, especially if it is against whatever they believe. However, the ET noted five KIIs that used the insufficient-sample-size argument in criticizing PVTs. According to the Executive Director of a leading CSO in Nigeria; The PVT process is confusing. YIAGA Africa’s 2019 election report contradicted its position on the outcome of the election. The sample size is a limitation. It is not representative. Another major limitation with the process is that the methodology seeks to achieve two major objectives: the election process and the election outcome. NDI’s methodology needs to think through which of these objectives it wants to make a call on. In the opinion of a former PVT Program Manager, PVT is still seen as elitist. Popularizing PVT is still a challenge. Nigeria is used to big numbers of deployed election observers. Thus, when you say that you deploy 250 observers to predict electoral outcome, it is seen as not being representative enough. He went further to say that the PVT sometimes “over-promises,” speaks to only election day, and not pre-election or post-election. While this view is expressed, the PVT methodology deployed through EECSP is only meant to observe what happens on election day at the polling unit, which is why it is supplemented with PREOs that observe the pre-election environment and the final reports which incorporate review of information gathered before election day. The PVT provides information at the polling-unit level by design so that observers can observe voting at the most basic unit of analysis. YIAGA Africa also conducts traditional observation at collation centers and issued statements on collation from the 2019 March 9th governorship elections, for example. The insufficient-sample-size argument is inaccurate for several reasons. First, the PVT sampling method obtains the minimum representative sample size needed to be able to make generalizable observations through the use of statistics in a cost-effective way. The second related point is that increasing the sample size might increase the statistical power, but it may also magnify the potential to misinterpret results.4 The third related point is that given the first two arguments, increasing the sample size might not pass the cost-benefit test, especially given the large expenditures required for PVTs. The persistent use of the insufficient-sample-size argument points to three main factors. First, there is a lack of understanding regarding statistical methods, although PVTs have been deployed in elections since 2008 in six African countries, including Nigeria. Second, given the tendency for losing parties to malign even a well-executed PVT and winning parties to tout it, NDI and stakeholders did not conduct sufficient dissemination and information campaigns and messages to 4 Statistical tests are drawn from samples that are meant to be representative of populations. So, if samples are added to a statistical test, the analytical power also increases. But this implies an exaggerated tendency to reject null hypotheses with potentially negligible differences. In other words, what is insignificant might be significant (Faber and Fonseca, 2014). 28 dispel the public perception of this notion of insufficient sample sizes. A third factor is the reality that elections will always have aggrieved losing candidates who may seek convenient reasons to question an otherwise legitimate election rather than accept defeat. 3. Media and observer groups: The ET’s desk review of secondary data and reports showed that PVTs helped in building confidence in the electoral process as referenced by different election observation group reports and the media. Major findings of these observation and media groups are referenced below. • “Other indicators of attempts at manipulation are apparent from the findings of the citizen observer Transition Monitoring Group (TMG), which undertook a PVT (“Quick Count”). While this confirmed the overall accuracy of the official presidential election results, the data strongly suggests that turnout was inflated, by at least 10 percent, with results adjusted in favor of PDP during the collation in the South-South states of Akwa Ibom, Bayelsa, Delta and Rivers.” —2015 EU European Union Election Observation Mission Final Report, Federal Republic of Nigeria. • “These observations are corroborated by TMG’s Quick Count data which indicates that the official turnout figures were inflated in Rivers and other South-South states. TMG’s Quick Count results for parallel vote tabulation were consistent with INEC’s official turnout numbers for every geopolitical zone aside from South-South, in which official turnout in every state but Cross River (i.e. Akwa Ibom, Bayelsa, Delta, Edo and Rivers) was nearly 20 per cent higher than the estimated turnout based on TMG’s Quick Count data. TMG reports that its Quick Count estimates ’strongly suggest that turnout was inflated during the collation process in the South-South ... by at least 10.8 percent.’ However, TMG notes that the irregularities did not affect the final election result.” —International Republican Institute Observation Mission Nigeria 2015 Report (p.11). • “On 27 February, the INEC chairperson declared the APC candidate, President Buhari, as the re￾elected winner with 55.6 per cent of the valid votes. Atiku Abubakar, the PDP candidate who won 41.2 per cent of the valid votes, rejected the results almost immediately but committed to pursuing a legal petition. YIAGA Africa announced that the results were consistent with its parallel vote tabulation that independently projected the result based on a sample of 1,515 polling units. two leading parties won 96.8 per cent of the valid votes between them” —European Union Election Observation Mission NIGERIA 2019 Final Report.(Source: https://eeas.europa.eu/sites/eeas/files/nigeria_2019_eu_eom_final_report-web.pdf) • “The Situation Room notes with disappointment the impact of INEC’s logistical lapses on the elections. Several parts of the country report late opening of polls. According to YIAGA Africa, South East and South South recorded the lowest number of early poll openings. About 27 percent of polling units had opened as at 10 am in the South East. In the South South, it was only 21 percent. The average across other parts of the country was about 45 percent as at 10 am. The late arrival of materials and ad-hoc staff account for much of the delay in polls opening across the country. In some places, voting started around 11 am. The delay in the commencement of vote will certainly affect the collation process, which is now feared to extend well into the night, with attendant security risks and further logistic concerns.” —Nigeria Civil Society Situation Report on Nigeria 2019 general elections (Source: http://www.placng.org/situation_room/sr/wp-content/uploads/2019/09/Report-on-Nigerias￾2019-General-Elections.pdf) 4. Political Parties and Military: Among political parties and the military/security sector, stakeholders have also observed a reliance towards PVTs. According to YIAGA Africa, Kano PDP requested their PVT report after the 2019 general election. Similarly, the Nigerian military panel 29 investigating alleged interference and disruption by the men of the Nigerian Army in the 2019 elections wrote to YIAGA Africa for further information, which was provided. YIAGA Africa is now using the PVT statistical methodology to observe the Continuous Voters Registration exercise ahead of Anambra 2017 and Ekiti 2018 off-cycle governorship elections. YIAGA Africa also applied PVT principles in the audit of the Voters Register ahead of the November 16, 2019 Kogi and Bayelsa States’ off-cycle governorship elections. During the 2015 presidential election, only the Buhari Election Monitoring Committee visited the PVT data center hosted by TMG. During the 2019 presidential data center setup, the APC, PDP, and IPAC sent representatives to the data center hosted by YIAGA Africa. This development showed the growing acceptance of the PVT within the political class. The Buhari Centre also referenced YIAGA Africa’s PVT results-verification infographic in its official Twitter account (see Figure 6). FIGURE 6: THE BUHARI CENTRE’S TWEET REFERENCING YIAGA AFRICA PVT RESULTS 5.1.4 Major Findings on Pre-Election Observation PREO complements the administration of the PVT. The PREO findings guide the PVT implementation strategy overall. PREO reports were a major source of reference for the National Peace Committee chaired by General Abdulsalami Abubakar. The committee, composed of eminent Nigerians, monitors elections to ensure all parties comply with the Abuja Accord signed in January 2015. The PREO reports have not been as useful to the INEC as the PVT, according to two INEC National Commissioners the ET interviewed. They both observed that, although there were claims of PREO 30 deployment, none of the PREO reports disclosed that election materials were not at the INEC offices hours before the election—which led to the eventual postponement of the 2019 general election. TMG’s PREO provided initial data that led to subsequent campaigns by CSOs, such as Stop-VAWIP. Also, 2015 PVT and PREO data on violence against women provided part of the initial data used in producing NDI’s votes without violence toolkit. Ahead of the off-cycle Bayelsa governorship election in 2015, TMG’s PREO report pointed to the build-up of arms in the state, which was confirmed by the violence that marred the election. PREO findings helped inform the conduct of voter education and information campaigns around the gubernatorial and the 2019 presidential elections. Both INEC and the National Orientation Agency (NOA) stepped up conducting voter-education campaigns in the lead-up to the 2019 general election, based on WTV PREO reports. 5.2 CONCLUSIONS The findings the ET gathered relevant to Evaluation Question 1 provided information to validate the EECSP program outcomes at some level. Specifically, the relevant outcomes are 1 to 6. Outcomes 3 and 5 are thematically similar and are discussed together. The same holds for Outcomes 4 and 6. Outcome 1: PVTs served to increase credibility of election results PVTs increased the credibility of election results on two fronts. First, the application of statistical principles and methods behind a PVT, from sampling to analysis, show a rigor that makes it difficult to question its results, even though some still try to discredit the outcomes. Second, there has been acceptance among various stakeholder groups, including INEC, CSOs, national and international observers, political parties, and the security sector regarding the validity of PVT results. In politically charged contests, the losing parties tend to feel aggrieved and malign even a well-executed PVT while winning parties tend to tout it. That said, the EECSP activity fell short of sufficiently socializing and communicating PVTs to the general public, which has led to some misunderstandings about it such as the sample-size argument. Despite methodological foundations and broad stakeholder support, the EECSP should have addressed the issues raised by critics. Outcome 2: Growing evidence of local CSO capacity to deploy PVTs YIAGA Africa has developed its capabilities and established its reputation to effectively carry out PVTs. However, these capabilities were developed over time and informed by NDI’s extensive assessments of a partner’s technical and management strengths in carrying out a PVT. The ET did not find any evidence that other CSOs are gaining similar capacity to deploy PVTs, even toward the 2023 general elections. An interviewee from YIAGA Africa stated that they are willing to collaborate further with NDI and other CSOs to expand the deployment of PVTs throughout Nigeria. While the expressed willingness was encouraging, the ET did not find any evidence of existing CSO collaboration. Outcome 3: PVT contributions used to inform recommendations for electoral law reform; Outcome 4: PREO and PVT by TMG provided initial data that led to subsequent Stop￾VAWIP campaigns by CSOs; Outcome 5: Changes to INEC procedures due to PREO and PVT findings. There have been some contributions to electoral law reform and INEC procedures due to PREO and PVT findings, as discussed in its spillover effects (Page 16). Outcome 6: Women successfully recruited and deployed as 30 percent of total PVT observers. PREOs are conducted to understand initial conditions heading towards an election; PVTs are designed to tally votes. That said, there are opportunities to link PVTs to PREOs so that critical pre-election 31 issues can be resolved ahead of election day. The ET found moderate to weak evidence that PVTs and PREOs provided initial data that led to subsequent Stop-VAWIP campaigns by CSOs. The campaigns were largely due to already active programs run by CSOs, even relatively small ones at the LGA level. These CSOs have been serving communities before and after elections, and campaigns such as Stop￾VAWIP are just one dimension of a broader initiative to empower women. After discussions with NDI, the ET learned that there have been approximately 5,000 individuals trained as PVT observers, but the ET was unable to get the comprehensive list. As an alternative, the ET was able to determine that 35 percent of WTV master trainers and 43 percent of SFPs are women. 5.3 RECOMMENDATIONS 5.3.1 Improve PVT Communication and Outreach Strategies A strong finding from respondents is the need for better communication on the PVT. There is need for better sensitization on the PVT of critical stakeholders such as political parties, INEC, the media, and CSOs and the general public. 5.3.2 Diversify PVT Methodology The statistical principles of PVT could be used for wider governance issues such as budget tracking, in the area of health care delivery, constituent engagement, electoral reform advocacy, etc. PVT need not be very expensive, as many costs can be reduced through the way it is currently conducted. According to some KII respondents, organizations deploying PVT could explore the option of volunteers and can cut down on the staffing. 6. EVALUATION QUESTION No. 2 Evaluation Question No. 2: To what extent have NDI partners’ management skills (Objective 2) influenced their capacity to implement the PVT and campaigns towards increased participation by marginalized groups in Nigerian political processes (April 1, 2014 to date)? • What factors led to the key outcomes achieved in improving management within partner CSOs, and what were the key impediments to achieving the outcomes? • Are partners (particularly the PVT subgrantee) implementing the new skills or recommendations from NDI’s capacity assessment? WL conducted a participatory institutional analysis (PIA) with TMG in 2015 and NUPAS for LHI, YDI, YIAGA Africa, IFA, and Centre for Citizens with Disabilities (CCD) in 2016; major weaknesses were found in their operational systems. WL recommended necessary intervention in the areas of: • Non-functional organizational boards of trustees; • Centralized decision-making processes driven mostly by Executive Directors; • Weak financial management policies and practices; • Weak procurement processes; and • Weak M&E systems. These findings triggered WL’s capacity interventions for the partners before WL’s role was redefined to provide support on communications, while the financial management institutional capacity development for the sub-grantees were handed over to KPMG. In 2016, NDI disengaged TMG and KPMG took over the assignment of WL and provided to NDI forensic verification of documentation, integrity and quality assessments of sub-grantees’ accounting information, and fraud prevention reviews. In the process, two of the sub-grantees were dropped, 32 namely GAT and CCD, on the grounds of non-compliance with best practices on financial matters, while the remaining six sub-grantees continued with the process. 6.1 FINDINGS Evidence from the KIIs, FGDs, and desk review showed KPMG conducted quarterly financial assessments for the partners. Some of the issues addressed include: • Finance: Approval process, segregation of duties, maintenance of bank records, maintenance of financial records, maintenance of petty cash, bank reconciliation, treatment of exchange • Procurement: Vendor due diligence, vendor selection and contract award, vendor maintenance and delisting • Human Resources: Employee due diligence, ethics, training and awareness, disciplinary process • Information Technology: User access rights, multi-layer approval, and disaster recovery • Statutory Regulatory compliance: Tax payment and pension contributions KII and FGD respondents stated the technical assistance and capacity building support WL and KPMG provided under EECSP’s Objective 2 were successful because of the following factors. 6.1.1 Improved Partner Management Systems A total of six out of the eight Objective 2 sub-grantees5 agreed that before the NDI intervention, operational priorities were set on getting work done, without paying much attention to the process. This means sub-grantees focused largely on achieving work objectives, but with minimal considerations on issues such as efficiency, cost-effectiveness, governance and oversight, and optimizing business processes in achieving work objectives. With Objective 2 assistance, sub-grantees agreed that the decision-making process has gotten better as boards have been established within their organizations or have been expanded and assigned better-defined roles. Sub-grantee staff interviewed by the ET acknowledged that the human resources management, finance, accounting systems, and procurement systems have improved, and policies have been reviewed and updated along the suggested pattern of best practices recommended by KPMG and WL. Sub-grantee staff acknowledged that the improvements were not only beneficial, but also desirable, as they have eased conduct of business. They stated that the improvements were not entirely new to them, but they supported them to deepen what they had been practicing before in a lesser way. Sub￾grantees, such as YDI, are working on the expansion of board members, while YIAGA Africa and IFA have completed the process. Other organizations are still considering the best way to move forward. Key strengths of NDI’s support under the EECSP cited by sub-grantees include the institutional and capacity building support EECSP provided in programmatic areas, specifically media and advocacy strategy development, campaign communications strategy formulation, and mentoring. Sub-grantees also acknowledged that NDI’s training brought awareness to cross-cutting issues, notably the consideration of gender balance in selecting board members. For instance, IFA noted 60 percent of the board members are now females (along with a PWD member). This support overlaps with Objective 3 of EECSP. Specific assistance included providing technical inputs into partner presentations 5 LHI, YDI, YIAGA Africa, IFA, WACOL, and Face Initiative. 33 and agendas, training materials, talking points, press releases, and media appearances. NDI also facilitated power mapping and helped groups reflect and analyze on their campaigns. 6.1.2 Factors that Led to Improved Management Systems Most sub-grantee staff acknowledged the usefulness of the training they received under the EECSP. A sub-grantee Executive Director said, “The institutional building of EECSP is the strength of NDI.” Another Executive Director stated, “With the institutional capacity development that we received from NDI, we have moved from a local NGO to national and international.” Another KII respondent and beneficiary of capacity building assistance in fundraising stated, “The experience has developed our capacity and has been a bedrock for our activities.” In more specific ways, the sub-grantee staff respondents and KPMG interviewees acknowledged that the following factors accounted for outcomes achieved: 1. Funding: The sub-grantees recognized that the provision of funds to hire WL and KPMG was a major factor that contributed to their capacity building, as they do not see themselves having the required funds to engage these firms in years to come. 2. Collaboration and ownership: WL conducted the PIA and KPMG performed the needs assessment, and sub-grantees said it created a collaborative environment and built partnerships for the process improvements. The methodology supported the grantees’ ownership of the intervention and contributed to follow-up action to provide sustainability for the exercise. Specifically, the Executive Directors were satisfied that they were included in identifying training needs of their respective organizations. Another example of the ownership experience was the bottom-up approach of the PIA, which gave grantees the confidence to implement process improvements because they were based on their own input. 3. Leadership buy-in: Most Executive Directors stated that the recommended improvements were necessary steps for deepening their organizations’ institutional capacities. Most organizations still rely heavily on the opinion of the Executive Directors, so their willingness to accept the reforms helped not only to achieve the results desired but to also sustain them. 4. Staff openness and capabilities: The various sub-grantees had trainable and willing staff to participate in the Objective 2 trainings. Staff of the sub-grantees had been helped, largely because there was residual knowledge to build upon. For example, accountants at YDI and IFA were able to learn and apply the new QuickBooks accounting software and the human resource staff at YDI and YIAGA Africa improved recruitment processes to add value to the organization, moving away from a culture of recruiting “whoever was available.” There were also improvements in procurement systems to make them more transparent and efficient, which have been sustained since training. As one Executive Director noted, the organization is currently practicing “know your vendors.” 5. Resource networks: Resources for the implementation of KPMG’s recommendations were available to the sub-grantees as they were able to identify additional vendors and board members. 6. Synergies with Objective 3 support: For all EECSP grantees and partners, WL packaged Objective 2 capacity-building assistance with Objective 3 support in the areas of advocacy development, campaign mobilization, and power mapping, among others. The approach had spillover effects, as it helped grantees and partners build the capacities of CSOs that did not receive EECSP assistance. For example, one Executive Director stated that receiving both Objective 2 and 3 support allowed the CSO to deploy and manage strategies more effectively; 34 smaller CBOs they worked with gained knowledge in advocacy design, as well as best practices in managing campaigns. As another example, YDI, IFA, and YIAGA Africa have been training CSOs in their coalition who are not beneficiaries under EECSP. These capacity interventions are further explored under Objective 3. 6.1.3 Challenges Faced with Adopting new Skills and Enforcing the Change within Partner Organizations Except for CCD and GAT, all sub-grantees found the interventions necessary and it was not much of a problem to adopt them. However, the ET also collected information from sub-grantee staff on impediments or challenges under Objective 2. 1. Timing: Three sub-grantee staff members stated that the unannounced timing of the KPMG reviews or follow-up assessments was a source of disruption for already-scheduled programs of sub-grantees. While standard practice with reviews and audits of this nature tends to be unannounced visits (so the reviews show how the organization is actually implementing findings and routinely using the systems), according to a key informant the intervention was viewed as policing and a weakening of the (sub)partnership arrangement with NDI. 2. Grant micromanagement: Staff from three sub-grantees stated NDI and its partners micromanaged some aspects of the support. A senior staff member from another CSO stated NDI’s insistence on procuring services from their list of vetted vendors was challenging, because they preferred to work with their own proven vendors to ensure the quality of the services. While NDI implemented such procedures for better accountability to both their headquarters offices and, ultimately USAID, the sub-grantee staff said it limited their ability to perform their work effectively. 6.2 CONCLUSIONS The findings the ET gathered relevant to Evaluation Question 2 provided information to validate the EECSP program outcomes at some level. Specifically, the relevant outcomes are 7–9. Outcome 7: Financial and procurement management systems within local CSOs established and improved Outcome 8: CSO partners took steps to improve and diversify governance and advisory boards and executive management Outcome 9: Sequential KPMG assessments identified improved financial, procurement and HR management systems within local CSO partners Improvements in management capacities: All the current sub-grantees acknowledged the effectiveness of the training they received under the EECSP. The ET gathered data that showed NDI, WL, and KPMG took deliberate and careful steps to identify areas that needed strengthening to enhance effectiveness in the delivery of the sub-grantee programs, campaign mandates, and PVT deployment. Following NDL, WL, and KPMG’s support, six sub-grantees had improved in all areas of intervention and domesticated the respective changes in their financial and procurement policy documents, although the levels of improvement vary by sub-grantee. NDI also expressed satisfaction with the remaining five sub-grantees (LHI, YDI, YIAGA Africa, IFA, and WACOL), and stated that reasonable synergy had been built between NDI and the sub-grantees to facilitate future work with NDI until the close-out of EECSP. Notably, the sub-grantees gained greater capacity in financial management as they are able to keep petty cash books, keep separate accounts for different donors, and undertake monthly bank 35 reconciliation, among other functional improvements. In the area of governance, sub-grantees have expanded the number of board members and have assigned clearly defined roles for each member. In the area of human resource management, sub-grantees have developed a policy on recruitment, improved their work ethics, developed training policies for their staff, and are now paying pension benefits on behalf of their staff. Regarding procurement, they now have vetted vendors lists and are getting to know their vendors. Link to improvements in campaigns and PVT deployment: The financial management capacity building has started yielding benefits to the sub-grantees, as they all agreed they are more settled and stable in order to carry on with the business of participating in the election and electoral processes in Nigeria. Sub-grantees are sharing knowledge with other CSOs that did not participate in the EECSP activity. One sub-grantee that has been involved in annual capacity building for 20–25 CSOs before EECSP, participated in capacity-building activities to train an average of 15 CSOs and brought NDI to participate in the training of these non-beneficiary CSOs. Another sub-grantee had trained two notable PWD groups, namely Joint National Association of Persons with Disabilities and the Network for the Advancement of Persons with Visual Disabilities. At least two key informants from different sub￾grantees also noted they were able to attract and receive funding from other donors due to their improved capacity under the EECSP activity. A new activity recently launched its training on fundraising; this activity was rolled out under EECSP’s extension period. The first session was to be held around the time the ET was starting field data collection, so it was not possible to assess any outcome or measure of sustainability. The NDI-KPMG/sub-grantee relationship under the EECSP activity has been effective in bringing about the anticipated goal of improving the management skills of the beneficiaries to improve their management and operational systems around their campaigns and PVT deployment. This conclusion can be drawn for all currently operating sub-grantees interviewed. 6.3 RECOMMENDATIONS From the findings and conclusions gathered during the data-collection phase, the ET provides the following recommendations under Objective 2. 6.3.1 Continue Capacity-Strengthening Interventions The WL, NDI, and KPMG trainings have started yielding anticipated results and need to continue in the next project cycle to help sub-grantees move from a level of basic awareness and knowledge to that of experts, in order to mentor other growing in-country CSOs. The arrangements of partnership between NDI, KPMG, and sub-grantees should be retained, but refined to ensure that programs already planned are not disrupted or interrupted negatively. KPMG can inform sub-grantees on time periods during which a team would not likely visit the sub-grantee, while keeping a window of time that it may, to avoid disrupting sub-grantees’ operations while at the same time maintaining the oversight needed. 36 6.3.2 Flow-down Grant Policies to Address “Micromanagement” Issues To address sub-grantees’ perceptions of NDI’s micromanagement of vendors, NDI should require sub-grantees to flow down6—or pass along the NDI financial requirements—to vendors. NDI requires sub-grantees to provide documentation of all financial transactions, such as receipts and quotes; NDI should require sub-grantees to request this documentation from their preferred vendors as well. This requirement should be explicitly stated in the award agreement to establish this understanding from the very beginning. NDI should have more orientation and trainings for sub￾grantees regarding their financial management practices or protocols as established in the respective grants. These approaches might allow sub-grantees to work with vendors they prefer but will also ensure that transactions will meet NDI’s requirements for financial oversight. 7. EVALUATION QUESTION No. 3 Evaluation Question No. 3: To what extent have marginalized groups’ (women, youth, persons with disabilities and internally displaced persons) campaigns affected their participation in Nigerian political processes (April 1, 2014 to-date)? • Specifically, which of the marginalized groups campaigns have been successful and which were not? • What factors led to the campaign outcomes and what were the key barriers to achieving successful campaigns? NDI’s voice, space and accountability (VSA) baseline assessment of EECSP partners states that “voter and civic education campaigns to enhance electoral participation have in the past not adequately addressed the needs of marginalized groups.” Marginalized groups include women, youth, and PWDs. Later in the project, IDPs were added. Taken together, these marginalized groups are usually sidelined in dialogue and actions on matters concerning governance, election, political discussions—action, in general. To increase the capacity of these groups to demand representation and qualitative participation, NDI developed an intervention plan designed to encourage citizen-centered advocacy for inclusion in political and electoral dialogues. This plan included strategies to strengthen marginalized groups’ capacity to effectively influence and champion change. NDI’s 2016 VSA assessment states that Nigeria CSOs are transitioning from a service-delivery role to a strategic-advocacy function. It specifically mentions a weak understanding and application of advocacy tools as a constraint to successful advocacy. This was evident in CSOs’ shared and broad application of advocacy visits only as the main advocacy tool for influencing change. A 2016 strategy meeting with Nigerian CSOs showed that campaign projects and NDI partners were well-researched. It also showed an effective VSA vetting and selection process by NDI. Other observations from that strategy exercise indicated that the proposed campaign projects will stretch partner and staff skills but concluded that NDI’s partner-development strategy was adequate. 7 This background is critical to appreciating the progress made in advancing CSO roles and capacity within the EECSP project time frame. 8 6 Flow-down is a contracting safeguard whereby a vendor, in this case the sub-grantees’ sub-contractors, agree to a binding clause that ensures that the vendors’ obligations to the contractor, sub-grantees in this case, mirror the sub-grantees’ obligations to the owner, NDI in this case. 7 Wilfore, Kristina “Increasing Women’s Participation in Political and Local Government Processes, May 2016 8 April 1, 2014 to March 30, 2019 37 From the inception of the EECSP activity in 2014 to date, NDI has supported Nigerian advocacy campaigns listed in Table 6 below. TABLE 6: ADVOCACY CAMPAIGNS SUPPORTED BY NDI Campaign Focus Campaign Name Partner/Lead CSO Women ▪ Women in political process ▪ Stop-VAWIE/Stop-VAWIP ▪ VAPP ▪ Women Policy Agenda LHI, Sokoto FIDA, Kogi, Edo, Osun, Ekiti and Bayelsa WACOL, Enugu TMG, National Face Initiative. Bayelsa NISD, Osun Youth ▪ Vote not Fight (VNF) ▪ Not Too Young to Run (NTYTR) ▪ Youth Policy Agenda Youngstars Development Initiative (YDI), National Youth Initiative for Growth and Advancement (YIAGA Africa), National KDI, Osun Maji, Rivers PWDs Access Nigeria, formerly known as “Leave no one Behind” Centre for Citizens with Disabilities (CCD), National Inclusive Friends Association (IFA, National) IDP Voting Enfranchisement CLEHI, Benue and CHAD, Adamawa Others ▪ Disinformation ▪ Hate Speech CDD-West Africa, Abuja The ET assessed all Objective 3 campaigns and concluded the most successful ones were those that had: • National (and even international) presence; • Profound effect on the broader socio-economic conditions of Nigeria (e.g., NTYTR’s impact and the subsequent Constitutional amendment); and • Largest numbers in terms of individual participation. Using these criteria, the ET concluded that VNF, NTYTR, Violence Against Persons Prohibition (VAPP in three states) and Access Nigeria are considered successful marginalized-group campaigns. The IDP campaign was also successful, but its success is limited to Benue State. Though it is still early to assess the performance of the disinformation, hate speech, and women and youth policy agenda campaigns, there are already tangible outcomes from these campaigns as discussed below. 7.1 WOMEN’S CAMPAIGN FINDINGS 7.1.1 Women in the Political Process In 2016, NDI supported LHI to advocate for women to be represented in Sokoto State’s political space. At that time, Sokoto did not have a woman in an elected position and women were highly marginalized from government’s decision-making. LHI, with technical support from NDI, implemented a campaign in three Sokoto LGAs—Dange Shuni, Bodinga, and Yabo—to advance women’s political 38 inclusion. LHI advocated for the establishment of a women’s desk in the local council areas to provide an official avenue for integrating women’s voices into the local government’s decisions. LHI also conducted women’s voter education and worked with the State House of Assembly and the Ministry of Women Affairs on issues affecting women. 7.1.2 Stop Violence Against Women in Elections (Stop-VAWIE) In September 2015, NDI and the International Federation of Women Lawyers (FIDA) launched the Stop-VAWIE campaign to: 1) document and report data on incidents of violence against women; 2) engage with election officials, security services, women’s groups and others to mitigate violence; and 3) organize civic education around preventing violence against women in elections. NDI worked with TMG and FIDA throughout the 2015 off-cycle election process in Kogi and Bayelsa to collect data on incidents of gender-based violence, both through a PREO methodology and via a hotline that collected reports of violence against women from citizens. FIDA then disseminated this information to influence key government and security officials for immediate action and to inform future policies that address and mitigate violence against women. FIDA also implemented a broader campaign to educate voters on women’s political participation and the impact of violence against them and worked with women’s organizations to ensure women have access to resources they need in the areas where violence was reported. 7.1.3 Stop Violence Against Women in Politics (Stop-VAWIP) YIAGA Africa included a monitoring analysis about pre-election violence and hate speech directed at women in the PREO reports and statements for Anambra (2017), Ekiti (2018), Osun (2018), and the 2019 general elections. GADA and LANW are implementing the Stop-VAWIP campaign in Kaduna. The campaign aims to: increase awareness of violence faced by women and advocate for mitigating violence against women in the state; provide services to female survivors who faced gender-based election violence during the 2019 general election; and build synergy with other key stakeholders such as local CSO’s, women groups, and governmental agencies. 7.1.4 Women Policy Agenda In 2019, New Initiative for Social Development (NISD) launched a post-election women’s policy agenda (WPA) campaign in Ekiti state. Through the WPA campaign, NISD will promote the implementation of gender-based policies, advocate for greater inclusion of women in Ekiti State government through representative and appointive positions, and reestablish platforms for women to effectively engage key stakeholders on women’s policy priorities. 7.1.5 Violence Against Persons Prohibition Bill (VAPP) In 2017, NDI engaged in several consultative activities to develop a plan and identify partners to push for the adoption of the VAPP Act in three target states: Enugu, Adamawa and Bayelsa. By 2018, NDI selected Face Initiative (Bayelsa), WACOL (Enugu), and Gender Awareness Trust (GAT) (Kaduna) as partners. NDI, through its partners, organized strategic meetings that led to the establishment of a steering committee for each state’s campaign. Each steering committee was composed of members of FIDA, other civil society, and media partners, all who will play a role in implementing the campaign. NDI also facilitated advocacy trainings for Face Initiative, GAT, WACOL, and the VAPP steering committees, and offered technical and strategic support to the advocacy outreach activities targeting local stakeholders. A critical support offered by Face Initiative and WACOL in May and June 2018 respectively was the “Inclusive drafting of the VAPP bills,” where they worked with the legislatures to revise the VAPP Act draft for adoption by the state assemblies. In December 2019, the Kaduna State House of Assembly passed the VAPP Act. The Enugu State House of Assembly, following a successful 39 advocacy by WACOL, passed the Act in April 2019, and both have been assented to by the State Governors. 7.2 POSITIVE FACTORS OF WOMEN’S CAMPAIGNS 7.2.1 NDI Training As mentioned in the Objective 2 section, the training provided by NDI on developing campaign strategies and media/communications plans built the capacities of CSOs in these technical areas. Packaging this assistance along with the management skills technical assistance in Objective 2 further amplified the effects of NDI’s assistance. Most CSO members stated that the technical assistance they received from NDI on how to develop community-level campaigns were helpful. Respondents stated that the design and delivery of advocacy strategies covering multiple key areas improved their capabilities overall as a CSO. The strategies may have varied by CSO, but it generally included: 1) technical assistance in advocacy; 2) a cascading training-of-trainer model; 3) campaign organization; 4) coaching/mentoring; and 5) media and communications strategy development. The ET gathered information that gave some indication of strong and active coordination among CSOs, particularly the relatively small ones, after individual engagements with NDI that helped sustain their advocacy programs. For example, in Osun, small CSOs and CBOs attended NDI (refresher) trainings and received materials on the existing state policies and laws signed, which helped them to develop a targeted advocacy agenda and conduct more effective grassroots activities, such as door-to-door campaigns. The ET also gathered information on CSO/local government coordination following the campaigns. In Sokoto State, the campaigns led to pilot initiatives for women desk officers that further led to the implementation of the State Gender Policy under the Minister of Women’s Affairs. Women were also appointed as heads of government agencies and special advisers to the Governor. In Ekiti, CSOs have achieved various milestones for local women’s groups, including the State Women’s Agenda and the inclusion of women’s causes in the state budgeting process. 7.2.2 Representation of Women at LHI The push for the set-up of a women’s desk at all LGAs was adopted and implemented at all LGAs in Sokoto State. The creation of the office has helped institutionalize the participation and representation of women’s issues. LHI also advocated for making every ministry, department, agency, and LGA have a gender desk responsible for implementing the state gender policy. Women were appointed as heads of government agencies and special advisors to the government. LHI attributes these changes to their advocacy. 7.2.3 Increased Women’s Voice in Sokoto State’s Budgetary Process Sokoto State’s annual budgeting process did not include a formal process of collecting and harmonizing input from women. LHI advocated for a change to include women’s input in the state’s budget process. Currently, there is a formal process for consulting with women to collect and set priorities. These priorities are then communicated to the LGAs and thereafter to the state for approval and adoption. 7.2.4 Identifying Champions CSOs such as LHI and NISD identified influencers, both men and women, engaged these champions, and leveraged their space and voice to gather support and push for change. LHI employed community￾influencing tools by identifying women leaders who stepped down trainings and knowledge to their 40 groups. This helped to expedite acceptance and create a sense of ownership, which in turn helped in mobilization and facilitating action. NISD identified key influencers, such as the wife of the Governor of Ekiti State and the head of the market women popularly known as “Iyaloja,” who with others led advocacy meetings with top government officials such as the Governor, commissioners, directors of ministries, departments, and agencies, and other top government officials in Ekiti State to push for budgetary allocation for their advocacy asks. 7.3 BARRIERS FOR WOMEN’S GROUPS Based on KII and FGD data, the ET identified the barriers discussed below. 7.3.1 Timing and Timeline For several CSOs with whom the ET engaged, the key impediments to the effective delivery of the VAWIE campaigns were the timing and timelines of the delivery of the technical and financial support. The timing was an impediment particularly on the provision of grant support; CSO leaders believe it should have been provided sooner to be more effective in their program area. According to one CSO Executive Director in Osogbo, for example, their organization started the campaign two weeks to the off-cycle election in Osun (2018) after the collection of PVC cards in 2018 had closed. The implication was, then, that women who had not registered or collected their PVCs had no voting opportunity. If the advocacy campaign was conducted earlier, it was highly possible that more women would have registered and collected their PVCs—and therefore voted. In this case, even if the sensitization campaign was effective and more women in the community decided to participate in elections, the opportunity to put their intentions into action was lost; the voter registration period was over and the window for the collection of PVCs had closed. For other women’s groups, the funding delay forced them to start Stop-VAWIP after the political parties started their own rallies and other activities. The group members in these cases stated they had to explain their campaigns were non-partisan, to avoid confusion among women in the community and to mitigate conflicts with political parties. According to one KII respondent, “Like I earlier said, we started the sensitization exercise about two weeks to the election when PVC collection has closed. The sensitization when registration of voters had closed…(T)he sensitization started when political parties were campaigning and that was a threat to us. Because there are some places we get to that, with our posters, political party agents saw us as (an) opposing party…(W)e have to humble ourselves (to) persuasively let them know we are not political parties; we are NGOs. We are only trying to encourage everybody to vote and even to vote for your own party.” One Director stated that starting early would have made their messages to distinguish themselves as non-political more effective. The Executive Director in Osogbo referenced above noted that the two-week period their CSO had during the 2018 off-cycle election was insufficient; the consensus among respondents was to start from six months to one year before a state or general election. Respondents also felt the Stop-VAWIP assistance their group received was planned too closely around the 2019 general elections, either before or after. This situation led to activities being discrete and either did not allow for enough preparation before elections, or it limited the opportunities for follow￾through activities after the elections. Despite the limited support from the state governments and other CSOs implementing advocacy campaigns after the elections, however, some CSOs took it upon themselves to continue with activities to support women’s causes in general but used some of the election-related resources from NDI and other donors. For example, almost all women’s groups in states the ET visited said that the telephones they received to report election-related incidents against women were “repurposed” after 41 the 2019 elections to establish and maintain a hotline to receive incidents of violence towards women and children in the community and report them to the local police. The women currently serve as online desk officers and maintain the hotline as part of their advocacy functions. 7.3.2 Cultural Barriers The overwhelming feedback among respondents from women’s groups was that the broader cultural issue of a lack of women’s empowerment extended to the area of elections and voting. Prior to the 2018 off-cycle elections in Osun, one CSO conducted a baseline survey covering 30 LGAs in the state. The results showed an expected low participation rate for women in the election. Another CSO, also from Osun, conducted a baseline survey at selected public and private secondary schools before the 2019 elections and found incidents of children and relatives “suffering in silence.” While the survey targeted secondary students, it also captured information on violence against wives and young women in the home. KII respondents also provided consistent feedback on broad, deep-rooted practices of the marginalization of women both by individuals and established institutions. Examples include: • Persistent “advisories” by individuals from political parties, local government officials, and the security sector for women not to vote during the elections because polling units were high￾risk areas and they could be exposed to dangerous situations; • Pressure from husbands not to participate in campaigns because it would take the women away from their responsibilities at home and their duties to go to market (either to run the business or make the daily purchases); • One KII respondent stated, “Even if I am a political officer now, I am still marginalized.” She still meets resistance in doing her duties to go to public offices and gather information on women’s experiences in the household and community. While most CSO leaders the ET interviewed stated an active and growing interest in elections among women in their communities, participation of women in politics does not seem as strong. In terms of holding political positions, an INEC official stated the percentage share of women has been going down since 1999. In the House or Representatives of the current 9th Assembly of Congress, only 22 out of 360 members are women, while the Senate has seven women out of 109 senators. 9 Some women have assumed management roles within the political parties. 7.4 CONCLUSIONS The findings the ET gathered relevant to Evaluation Question 3 provided information to validate the EECSP program outcomes at some level. Specifically, the relevant outcomes are 12, 14, and 16. Outcome 12: Deepened capacity of local CSOs to conduct and sustain policy and advocacy initiatives Synergies in programming or combining Objectives 2 and 3 activities in one package were effective. The findings on NDI trainings and the advocacy strategies showed that synergies in programming support provided more impact to some CSOs compared with those who only received discrete forms of assistance. 9 Data were collected from an INEC official KII and the Policy and Legal Advocacy Centre. 42 The evaluation team also found evidence of more engagement from women in the communities through CSOs and not just through EECSP. There was help from NDI, but a lot of CSOs have been doing such work before and plan to continue without NDI support. Outcome 14: Increased participation of marginalized groups in elections and political process The participation of women at the community or grassroots level is evident, as CSOs observed a growing trend of individuals joining advocacy campaigns during the 2019 elections. Post-election campaigns have continued with the sustained operations of the hotline operated by desk officers and the regular media spots, mostly in radio. However, the participation of women has not been as strong further up the chain of the political process. The declining trend in women holding political positions has continued, which suggests that women do not have, and might not be receiving, the financial and non-financial capital needed to run for public offices. Outcome 16: Fostered CSO-government partnerships to independently sustain state￾level efforts to combat violence against women in elections and politics CSO/government collaborations were evident in many initiatives but not all, as noted earlier in the Findings section. These collaborations have already led to the codification of women’s causes within the state policies and budgets. In Ekiti, for example, CSO advocacy helped push the Ministry of Women’s Affairs, along with other ministries, to allocate funding for women’s causes in the work planning and state budget allocation following the 2019 general elections. Initiatives specific to women’s advocacy were included as line items in the medium-term plan budget meeting. While CSOs must remain vigilant over a five- to seven-month cycle to ensure the plans lead to actual funding disbursement, these developments are already considered significant steps forward compared to previous years. 7.5 RECOMMENDATIONS 7.5.1 Strengthen Campaigns Women-focused CSOs have made significant accomplishments that have empowered women beyond election participation. USAID should support the momentum that has been built by CSOs by continuing to provide technical and/or financial support for increased women’s participation in elections and political processes. 7.5.2 Expand Partnerships to Include Informal Institutions and CBOs for Deeper and more Sustained Impact While working with the current sub-grantees, USAID should also engage new CSOs and CBOs to develop their institutional capacity and help them to implement programs and campaigns effectively. CSOs, in this context, refer to formal organizations working at a national or subnational level on a range of issues. CBOs, on the other hand, refers to formal and informal groups working at the local or grassroot level on a range of issues. For instance, a CBO could be a local youth group or market women’s group. Engaging a broader coalition of stakeholders—from large, established NGOs to smaller CBOs—may provide USAID with a deeper understanding of the cultural needs and gaps that must be addressed to ultimately make programming more responsive. This approach may require more time and resources from USAID from the activity design to grant award, but it will allow the Agency to contribute more effectively in the VSA framework by opening the space for grassroots stakeholders to leverage 43 opportunities from USAID in addressing deep seated cultural values and impediments to address some of the developmental issues”. For example, there is the emerging concept of community rigging in which a whole community is induced through its community leadership to vote in a certain way. This may never be addressed with deployment of PVT or the social media voter education campaigns but could be addressed through dialogue with the informal structures within that community. 7.5.3 Establish Better Performance Management Systems for Improved Data The marginalization of women is an undeniably deep-rooted social issue not just in elections, but in daily life in Nigeria. Incidents of violence and abuse are often heard of but are largely accepted as a way of life. At the same time, however, data collected from the field have shown that CSOs have made significant efforts to address the issue by applying the skills learned and capacities built from EECSP’s Objective 3 activities. Given the significance of the issue, USAID should establish a more effective performance management system to collect, monitor, and analyze data related to women’s issues. Currently, EECSP tracks data focused on women but they are unable to get the relevant data from the responsible institutions. For instance, NDI requested for gender-disaggregated data from INEC on the percentage of registered voters, but INEC did not provide it until Q1 FY19, when INEC released a disaggregated-voters register in Kogi (49.85 percent) and Bayelsa (45.97 percent). These data will now serve as a baseline for measuring percentage change in participation of registered women voters. However, there are opportunities to enhance monitoring mechanisms seamlessly within program operations. For example, desk officers who receive calls for support from women in the community can record information in a case management approach. EECSP currently collects this information, but only for Stop-VAWIP; these collection efforts can be scaled to include more CSOs and have more representative data. 7.6 FINDINGS ON YOUTH CAMPAIGNS The CSO assessment in 2016 by Kristina Wilfore shared concerns and doubt about the ability of YDI and YIAGA Africa to organize a successful campaign, based on her assessment of the advocacy capacity and then-lack of leadership.10 For instance, in the CSO assessment meeting, both Executive Directors were absent. Additionally, according to the report, “neither YDI nor YIAGA Africa demonstrated any sense of the policy-making process, knowledge of the issue with the National Youth Policy or grasp of community influencing and mobilization. 11 It is possible that this assessment is not valid because of the knowledge and experience (quality) of YDI and YIAGA Africa representatives at the assessment meeting, which led the ET to question whether this is true of the organization. Despite these concerns, in the past three years (2016 to 2019) YDI and YIAGA Africa have recorded significant success with various campaigns, such as the #NotTooYoungToRunCampaign which led to a reduction in the minimum age to contest for election to a public office. The age reduction required an amendment to the Nigerian Constitution and is a significant achievement for YDI and YIAGA Africa. . Twenty-two individuals between the ages of 25 and 30 years were able to run and win seats in their respective State Houses of Assembly; this is a direct result of the age reduction legislation that was passed by the National Assembly and assented to by the President on 31 May 2018. 12 Their interventions led to system changes such as coalition, mobilizing, resource management, CSO 10 Wilfore, Kristina “Increasing Women’s Participation in Political and Local Government Processes,” Karakoyun Strategies, May 2016. 11 Ibid, page18. 12 How Youth Fared in the 2019 Elections; a June 2019 YIAGA Africa report. 44 organization, use of informal and traditional institutions, and window for reforms; all of these achievements constitute a case to be studied and a model for galvanizing support for reform that needs to be harvested fully and replicated. 7.6.1 Not Too Young to Run With NDI’s support, YIAGA Africa and YDI launched the campaign NTYTR. This campaign was backed by a bill seeking to reduce the age of political candidates to allow young people to run for public office. NDI supported YDI and YIAGA Africa to craft action plans and strategies to push for the adoption and passage of this bill. YIAGA Africa led this campaign with support from YDI. Key Findings 1. Campaign successfully led to a constitutional amendment on age reduction for elective office. This amendment contributed to the following: 13 o 1,515 youth candidates (18–35 years) representing 23 percent of all candidates contested for seats in the National Assembly. o Ten or 14 percent of Presidential candidates were aged between 25 and 40 years. o Eleven or 15 percent of political parties fielded candidates for the Vice-Presidential position were aged between 35 and 40 years. o Youth candidacy increased from 18 percent in the 2015 House of Representatives election to 27.4 percent in the 2019 House of Representatives elections. There are 22 direct beneficiaries of the Not Too Young to Run Act in the State Houses of Assembly between the ages of 25 and 30. 14 2. By loosely coalescing with youth groups across the nation, YIAGA Africa and YDI built a model demand structure for youth-focused issues. 3. Respondent’s views on the utility of the NTYTR campaign was mixed; some felt it created access for young people, others disagreed. The contest stems from a position that participation of young people in the electoral process has never been limited by their age, but rather by their “economics.” This is referring to the high cost of campaigning in Nigeria and how that excludes youth participation. 4. Social media proved to be a defining tool for YIAGA Africa and YDI’s campaign. Young Nigerians learned about the campaign and supported it, mostly through engagements on social media platforms like Twitter and Facebook. An online petition launched on Twitter garnered almost 1,153 signatures. The success of the NTYTR campaign revealed the absence of a youth-policy agenda in different states and at INEC. In Benue, Ekiti, Osun, Kaduna, and Rivers, INEC is working with local CSOs to develop a youth-policy agenda. The purpose of this agenda is to empower youth and youth groups to continue targeted advocacy for change and implement specific policies as agreed in the agenda. YIAGA Africa is championing the development of an INEC youth policy. Through the Strengthening Youth Voices for Advocacy project supported by NDI, YIAGA Africa worked with INEC to hold youth consultation meetings as part of the development of the INEC youth strategy. YIAGA Africa is working with INEC to finalize the draft youth policy, currently with the INEC leadership for review. 13 Factsheet on Youth and the 2019 Elections in Nigeria; a February 2019 YIAGA Africa publication. 14 How Youth Fared in the 2019 Elections; a June 2019 YIAGA Africa report. 45 According to YIAGA Africa, the NTYTR campaign gave rise to the “Ready to Run” campaign. The Ready to Run campaign is a movement dedicated to inspiring young men and women to run for political office. This campaign’s goal is to prove that young people can demonstrate excellent public leadership with immense capacity to address Africa’s governance challenges. YIAGA Africa is supporting this project from other sources beyond NDI. What worked? Mixed use of innovative tools, especially tools that appeal to the beneficiary group—an example is social media, Twitter, and Facebook in particular. Identifying a change champion worked. The “window for reform”—this refers to the window for change that was possible because of the timeliness of the constitution reform and election cycle. The NTYTR campaign was supported by Senator Abdul Azziz and Honorable Tony Nwulu. Pursuing a constitutional amendment for an issue that culturally has biases (a society that prioritizes age and seniority in leadership). Concerns were that it would be an uphill task; according to one INEC Commissioner, “The NTYTR bill was a huge success because altering the constitution is not child’s play.” It was considered ambitious and several hurdles were identified as bottlenecks to the NTYTR success. For instance, getting the Constitution Review Committee to consider the bill, then adopt it, and subsequently recommend the bill as an amendment to the Nigeria Constitution, seemed insurmountable in the beginning. Despite the challenges, YIAGA Africa and YDI constantly adjusted their strategy and advocacy tactics to adapt to the political realities surrounding the campaign. For instance, when the bill was stone-walled in the National Assembly, YIAGA Africa relied on the alliance with Sahara Reporters, an online news agency,15 to inform constituents and thereby mount pressure on the elected representatives. Using its extensive network, YIAGA Africa successfully mobilized an impactful social protest, mobilizing over 2,000 youths. Also, relying on their social networks, YIAGA Africa adopted tactics like launching a “hall of fame and shame” for supporters and killers of the bill respectively, making public statements of kick-starting a recall process against members who voted against the bill, publicly challenging the position of non-supporters and praising supporters, all of which sustained campaign momentum and yielded results. Mobilizing local networks, using strong community-influencing skills, adopting ingenious ways of applying social pressure, multisector coalescing (especially with media, consistent messaging, etc.), were some of the ways YIAGA Africa iterated its advocacy strategies and tactics. This proved successful, because on May 31, 2018, President Muhammadu Buhari signed the NTYTR bill into law. 7.6.2 Vote Not Fight: Election No Be War NDI organized a two-day retreat in August 2014 on non-violence voter education for the 2Baba Foundation and 12 youth groups from 12 states in Nigeria with representation from all the six geo￾political zones. A key output of this activity was the VNF strategy developed and owned by local youth groups. In September 2014, NDI, in partnership with the 2Face Foundation, YDI, and 12 state-level youth organizations, launched a youth non-violence voter-education campaign entitled, “Vote Not Fight: Election No Be War.” Following the launch, YDI and the 2Baba Foundation organized several VNF campaigns to mobilize youths to vote peacefully in the lead-up to all the off-cycle elections from 2015 and the 2015 and 2019 national elections. 15 http://saharareporters.com/2017/07/27/plot-kill-not-too-young-run-bill-thickens-house-reps 46 Key Findings 1. Increased participation of marginalized groups in elections and political process. Anecdotal evidence from respondents on the VNF campaign efforts shows that the active mobilization and participation of marginalized groups contributed significantly to an increase in voice and space. Youth groups felt seen and valued, which increased their confidence and interest in elections and political process. 2. Impactful messaging on youth voter education and non-violence in elections. The messaging tools and tactics used by YDI and 2Baba Foundation were effective. At the AAR review meeting facilitated by NDI in May 2019, 2Baba Foundation, YDI, the New Generation Girls and Women Development (NIGAWD), KDI, Rivers’ MAJI, and OPA were in attendance and attested to a largely peaceful election in three out of the four states where the campaign was conducted and there was widespread public awareness on VNF campaign issues and messages. 3. A peace pledge signed by youths, political leaders, political parties, and candidates. The VNF pledge cards were signed by three presidential candidates—Oby Ezekwesili, Allied Congress Party of Nigeria; Fela Durotoye, Alliance for New Nigeria; and Kingsley Moghalu, Young Progressives Party. Other stakeholders such as the Nigerian Police Force, the PDP governorship candidate in Kaduna State and Emir of Zazzau, pledged peaceful elections for themselves and their constituents. 4. The convening and convincing power of a celebrity. VNF became a movement. The messaging, delivery, and channel connected with the target group. As one of Nigeria’s most celebrated and respected musicians, 2face Idibia, who also has a passion for youth development and Nigeria’s political growth, served as an excellent choice as the face of the campaign. 2face is one of Nigeria’s most successful and respected musicians with youth-convening power and used music to help young Nigerians find reasons to engage in the political process rather than sitting back and complaining. 5. The loose coalition between YDI, the 2Baba Foundation, and other youth groups worked. All the groups demonstratively owned the campaign and committed to its success, possibly because these groups are youth groups targeted by the campaign. Hence, the clear relationship and shared purpose may have led to the immediate uptake and collegiality. 7.6.3 Youth Policy Agenda Building on the VNF efforts, OPA, KDI, and NIGAWD developed youth policy agendas for Ekiti, Kaduna and Osun States. The NTYTR campaign also birthed youth-policy agenda advocacy in Benue State. 7.7 CONCLUSIONS Impediments to achieving campaign outcomes include structural barriers like traditional and religious norms and CSO capacity. Some groups said that NDI forced partnerships between groups, which caused a rift, thereby slowing down the campaign progress. Outcome 11: Increased youth candidacy in 2019 elections through successful advocacy campaign leading to passage of the NTYTR constitutional amendment Evidence indicated that VNF and NTYTR campaigns mutually reinforced each other due to: the political climate during the last election; the promise of NTYTR; and the resistance to NTYTR by some politicians and political elite with the view that Nigerian youths are incapable of being entrusted with the huge responsibility that comes with holding political office. This indirectly contributed to 47 widespread acceptability and ownership of the VNF campaign by that same group. There was a direct relationship between the age reduction or awareness of the NTYTR campaign to the increase in youth candidacy in 2019. Twenty-two individuals between the ages of 25 and 30 years were able to run and win seats in their respective State Houses of Assembly, a direct result of the age-reduction legislation passed by the National Assembly and assented to by the President on May 31, 2018. According to the INEC website (Figure 7), 243 candidates, or 5.2 percent of all candidates for the 2019 House of Representatives election, were between the ages of 25 and 29. The nationwide proportion of candidates of the same age group in the 2019 State House and Assembly election was 11.0 percent, or 1,608 candidates. Just over 12 percent (12.3 percent) (representing nine candidates between the ages of 35 and 39 years) contested for the 2019 Presidential election, while 13.71 percent (representing 10 candidates between the ages of 35 and 39) contested for the Vice-Presidential election. FIGURE 7: 2019 BENEFICIARIES OF NTYTR BILL 48 7.8 FINDINGS ON OTHER CAMPAIGNS 7.8.1 PWD Campaigns Access Nigeria (formerly referred to as #LeaveNoOneBehind) Access Nigeria achieved significant success. This campaign which was started by CCD and continued by Inclusive Friends Association (IFA) aimed at “improving the participation of persons with disabilities in Nigeria’s Electoral and Political Processes.” Despite mixed reviews from respondents on the performance of the Access Nigeria campaign, it successfully removed some key barriers that limit participation of PWDs in the electoral process. Key Findings • The campaign contributed to increased awareness on the barriers limiting active participation of PWD in elections. • Progress in reducing the barriers to PWDs’ participation in elections and the political process. Some of the progress made includes: o Polling accessibility review and reform for PWDs ▪ Quantitative evidence on PWD election day barriers. In the Edo 2016 audit, NDI supported the deployment of 31 roving teams of PWD observers and their assistants in 254 sampled polling units in all of Edo’s 18 LGAs and 192 wards. Evidence showed that access for PWDs and their voting challenges was not a major concern for INEC. For instance, 96 percent of PWDs did not have election day instructions and procedures explained to them in sign language—90 percent for the hearing impaired. o Documentation of baseline data of four PWDs voting in any elections in Nigeria during the Anambra 2017 governorship election. o INEC amended the election day form EC.40H to disaggregate the data of PWD by type of disability as a result of IFA’s advocacy. o The inclusion of special access for PWDs to polling units as part of the 2017 Constitution and electoral reform recommendations to the National Assembly and Constitutional and Electoral Reform Committee, in line with IFA’s recommendation. 49 o The sustained advocacy by NDI’s advocacy partner, IFA, and other disability-rights organizations, led to the assent of the Discrimination Against Persons with Disabilities (Prohibition) Act 2019 on January 23, 2019, by Nigeria’s President, Muhammadu Buhari. o INEC and IFA piloted election day written instructions for deaf voters (Form EC 30D PWD) during the Ekiti 2018 governorship election. o INEC following advocacy by IFA, piloted Braille ballot guides for the first time in Nigeria’s elections in the Osun 2018 governorship elections. o IFA’s advocacy also led to INEC’s deployment for the first time ever of the Tactile ballot guide in the 2019 presidential elections at 46% deployment rate based on YIAGA Africa’s PVT findings. Despite the easing of election-day voting obstacles for PWDs by INEC, their implementation was poor and thus, challenges remain and impact on direct beneficiaries was minimal. What worked? Partner selection. IFA appears to enjoy strong credibility with decision-makers in Plateau State, where they are based, illustrated by the association’s key role in establishing the Plateau State Disability Office. CCD, as a member of the Situation Room, has an access point to decision￾makers at the national level, has built a prominent reputation, and holds evidence-based dialogues with INEC. What did not work? “Coalition drag.” Progress on this campaign was slowed down by internal rancor between the two lead partners, IFA and CCD. This led to unnecessary delays in reaching consensus, missed opportunities, and overall low morale among the team. Following the exit of CCD, IFA’s action scaled up significantly. Access Nigeria, led by CCD and IFA, agreed on short- and long-term objectives. 16 The short-term objectives were to “facilitate polling accessibility reforms in Nigeria and improving PWD participation in elections through evidence-based advocacy, civic education, and get-out-the-vote campaigns.” The five goals agreed to under this objective were: 1. To facilitate the use of sign language interpreters during all civic and voter education messaging on television; 2. To assess and improve the distribution and reach of civic and voter education messages to citizens with different forms of disabilities; 3. To facilitate improvements in ballot box accessibility for PWDs and improve the placement of ballot boxes to enhance voting by PWDs; 4. To improve and facilitate the knowledge and incentives of election officials to assist PWDs to vote on election day; and 5. To facilitate the use of Braille IECs for civic and voter education at all levels. All five have been achieved to varying degrees. This means that progress was made sometimes in the form of policy changes, but in most cases, implementation was weak. Successes with short-term goals have set up a mechanism for the Access Nigeria campaign to achieve its four long term objectives. 17 16 See Wilfore’s 2016 report (pp.13-14). 17 Ibid. 50 7.8.2 IDP Campaigns NDI’s interest in the integration of IDPs in the electoral process started in 2014. Using several CSO forums, NDI raised the issue of possible IDP disenfranchisement and subsequently recruited the Center for Community Health and Development (CHAD) to organize a get-out-the-vote campaign targeting IDPs residing in 19 camps in Adamawa (five), Borno (nine), and Yobe (five), and surrounding areas. In Benue State, Community Links and Human Empowerment Initiative (CLHEI), a local CSO, partnered with NDI to work with IDPs in nine IDP camps and several host communities, for a total of 30,000 IDPs. CLHEI also worked with three LGAs where they had three camps each, in Makurdi, Guma, and Logo. Key Findings • Political will is important. This was evident in the success of the Benue State IDP campaign, as opposed to Adamawa State, where the INEC Resident Commissioner did not have the political support. • Despite INEC’s commitment, IDPs in Benue were not all able to vote because of the swift changes in the voting framework. The framework had made provisions for IDPs from other states to vote in the camp, but logistical issues came up and prevented it. • Despite the policy commitment by INEC, poor implementation negatively affects impact. INEC committed to setting up voting points in all IDP camps in Benue State. However, less than 48 hours prior to the presidential election in 2019, interviewees stated that INEC changed course. CLHEI continued to engage INEC and polling points at IDP camps which were created. This initial setback on polling units and changes in the voting framework affected turnout and votes. CLHEI and CHAD raised awareness of election-related barriers faced by IDPs and secured commitments from relevant stakeholders to address them. The meetings targeted RECs, emergency management agencies, the NOA, IPAC, political party representatives, CSOs, IDP camp leaders, and the media. As mentioned earlier, in Adamawa State, despite an interest to support IDP voting expressed by the REC, it proved not to be a priority for the state government. The Benue State REC had the buy-in of the state, which made it easier to introduce and fund the IDP voting changes. 7.8.3 Other Campaigns Disinformation and Hate Speech Campaigns against hate speech and disinformation commenced in the last year of the EECSP activity. In partnership with CDD-West Africa, NDI is supporting partner CSOs like OPA in Kaduna, KDI in Osun, and MAJI in Rivers to use existing platforms such as the VNF campaign to call on political parties and other election stakeholders to eschew electoral violence, hate speech, and/or inflammatory statements during the elections. NDI supported CITAD research to update and expand its current hate-speech lexicon, particularly with regard to gender-based hate speech and hate-speech terminology in local languages. NDI partnered with CDD-West Africa, who was already working on countering disinformation and misinformation in Nigeria, to scale up countering interventions for fake news and disinformation in electoral processes in Nigeria. CDD-West Africa worked with Graphika to setup an automated social media monitoring of disinformation, and thereafter analyze the data and develop stories or reports on trends observed ahead of the presidential election in 2019. Key Findings • These campaigns have made significant progress in the short time frame. 51 • Selecting the right partner is critical in any advocacy work; for instance, aligning with CDD￾West Africa to enhance its existing strategy and delivery for disinformation is helping secure quicker results. • Leveraging the infrastructure of the VNF campaign, the youth-policy agenda campaigns have moved speedily. NDI’s partner selection method has proven to be effective and responsible for the quality of the campaign delivery. • A common challenge expressed by all Objective 3 partners was the short time frame for campaign projects. Social and attitudinal change takes time; to sustain an impact or practice change requires even more time. The short project time frame was a key challenge, despite some success. Women, youth, IDP, PWD, and other campaigns have created a movement. A shift in the political sphere occurred as a result of these campaigns, leading to a raised consciousness and empowerment of previously marginalized groups. Some groups, youth for instance, benefitted more and responded by showing an increased level of awareness and interest in election and elections administration, and politics. In concluding the findings section, key factors responsible for the advocacy campaign successes include: • Loose coalescing and resource leveraging among NDI partners; • NDI’s support to the groups (partner selection criteria, CSO profile and trainings); • Multi-stakeholder participation and inclusion of traditional and informal institutions; • Timeliness of the campaigns. Timeliness here refers to the period in the political space; NTYTR, PWD, and IDP campaigns in particular; • Identifying and engaging change champions at all levels; examples are legislators for NTYTR, 2face for VNF, religious and traditional leaders for VAPP and VAWIE. 7.9 CONCLUSIONS Access Nigeria was a very important campaign. PWDs in Nigeria are historically excluded in stakeholders’ conversations across issues. Building capacity within the CSO space in Nigeria to engage in a rights-centered conversation on inclusivity for PWDs in election administration and political processes is critical and should be sustained. In addition to building the demand-side capacity, the capacity and willingness of the supply side to participate in the conversation with the demanders remains a challenge. One of the respondents confirms this by saying that INEC and other suppliers view PWD issues as a charity issue and engage on that basis, rather than on the basis of fundamental human rights. Findings show an increase in the participation of marginalized groups in elections and political process. It is also clear that NDI’s training and support helped local CSOs deepen their capacity to conduct and sustain policy advocacy. For instance, local partners CSOs like YIAGA Africa, IFA, Face Initiative in Bayelsa, KDI in Osun, CHLEI in Benue, among others, have successfully applied the acquired skills to other advocacy efforts beyond NDI-supported campaigns. These groups have also independently fostered partnerships with state and national agencies to sustain state-level efforts to combat violence against women in elections and politics. WACOL, Enugu, and LHI, Sokoto continue to work closely with their governments to respectively implement the VAPP law and sustain inclusion of women in governance and politics. 52 Demonstrating growth in CSOs’ capacities and their independent fostering of relationships with government and other stakeholders, some unintended outcomes have emerged: • One unintended outcome of the YIAGA Africa and NDI partnership is YIAGA Africa's Ready to Run campaign, an offshoot of the NTYTR campaign. • Two IFA members were elected into States Houses of Assembly. These legislators have now become the face of IFA. • Through IFA intervention, there is now a Disability Rights Commission in Ondo State. • IFA now has a Disability Resource Center, offering guidance and leading discussions on disability issues in Nigeria. 7.10 RECOMMENDATIONS 1. Develop election and political sensitization interventions for local communities targeting youth and women. This intervention should be combined with a livelihood intervention, especially for rural women. Despite the successes of women’s campaigns, there is still a concern about the effect of poverty, poor education, and other underlying causes for lack of participation and the lack of demand from women. Tackling these constraints will address the fundamental problems and will be sustainable. 2. Support programming and campaigns advocating for democratizing political parties’ processes to ensure equity and fairness, especially for youth and women. Despite the NTYTR success, there are concerns that the internal party structure is weak and needs to be improved in order to reduce entry barriers for young politicians. Future campaigns may be tailored to advocate for improved party processes to create space for young people and women to be involved in the daily affairs of political parties. 3. Develop long-term campaigns. Partners raised concerns about the short length of the campaign and expressed concerns about the sustainability of the impact. Attitudinal change happens over a deliberate and concerted effort at reorienting and reeducating. These campaigns require more time for a mind and culture shift to happen. 4. Campaign support strategies should include tangible plans for the CSOs to deepen or scale impact post-donor support. Most of the partners that received Objective 2 and 3 support especially mentioned the abrupt end of the support. While this is expected and obviously communicated to the partners from inception, developing a transition and gradual withdrawal plan with the partner at the start of the partnership should be part of future partner engagement plans. 5. The ET recommends that a case study of the VNF and NTYTR campaign (and the youth-demand infrastructure built by YIAGA Africa and YDI) be conducted to document the proven strategies for community organizing and influencing apparent in both campaigns. In essence, NTYTR in particular highlights a possible punctuated equilibrium case. Punctuated equilibrium in public policy refers to a model of change. This theory posits that most social systems exist in an extended period of stasis, which may be punctuated by sudden shifts leading to radical change. In the NTYTR case, a disfavored group with new ideas successfully broke through an established system of policy-making. This case study can serve as a reference for future elections and community mobilizing campaigns. The lessons, skills, and networks developed from this campaign can be kept “active” and possibly incubated to serve as an advocacy model for ongoing and future elections and political campaigns. 53 6. LHI’s campaign model should be copied and replicated across the country. To address resistance to the campaign stemming from social and religious barriers to women’s participation in politics, LHI found, relied upon, and aligned with local traditional and religious structures to garner support, trust, and influence. Donor communities overlook traditional institutions in Nigeria for obvious reasons. In the Nigerian context, its informal institutions and rules direct its political landscape. It is recommended that a case study is commissioned to investigate the unwritten rules of influence of informal institutions and how formal institutions like the donor communities can successfully partner with these informal institutions. 7. An advocacy strategy should be developed to build and scale on the momentum of the 2019 PWD wins for sustained dialogue on PWD and election issues. 8. EVALUATION QUESTION NO. 4 Evaluation question 4: To what extent can the developed institutional management capacity of partners be sustained beyond the life of the activity? o How self-sustainable are the efforts undertaken by the CSOs without donor assistance? 8.1 SUSTAINABILITY OF INSTITUTIONAL MANAGEMENT CAPACITY OF PARTNERS BEYOND THE LIFE OF EECSP 1. Organizational strengthening facilitates sustainability. Fundraising training and other institutional capacity strengthening given to EECSP partners have put them in good stead to maintain and sustain good practices. The benefits of institutional strengthening to Face Initiative, YIAGA Africa, IFA, YDI, LHI, and WACOL include having a diversity policy, adoption of accounting software (QuickBooks) for their financial reporting, a cost-sharing framework, a revision of the Financial Accounting Manual, and policies for whistleblowers, travel, and procurement. 2. YIAGA Africa, having received these institutional strengthening skills, is now engaged in transfer of knowledge by providing financial management training to other organizations such as YDI, Kukah Centre, and National Institute for Policy and Strategic Studies based in Kuru, Jos. KPMG’s intervention involved assessing the internal controls that were documented by the sub-grantees and making recommendations to strengthen policy controls in place with a focus on financial control. KPMG reviewed the expenses and staff salaries and compared the partners’ leading practice per their procurement process with the regulatory requirement for statutory payments such as pension. From KPMG’s reviews, they have upgraded partners’ accounting systems to keep records, backups of records, and IT security for Enterprise resource planning. 3. Six18 out of the eight of the sub-grantees that received KPMG support under Objective 2 support stated that: (1) policies had been reviewed in line with recommendations and trainings; (2) sub￾grantees have been training others both in-house and at other CSOs; (3) documentation of the training materials has been done in a way that enables new employees to access and learn by reading; (4) they recruited and trained volunteers as backup to take care of staff that may leave after training; (5) they trained all staff in a manner that will ensure no knowledge gap in the system, i.e., program staff can undertake administrative work and vice versa; (6) skills acquired have already been utilized, especially communications and finance staff who equally showed improvement in procedure; (7) new accounting software is already in use by various accounting departments; and 18 IFA, LHI, YDI, YIAGA Africa, Face Initiative, and WACOL 54 (8) advocacy skills acquired are already being deployed in campaign management. Two sub￾grantees (CCD and GAT) were dropped for non-compliance with NDI’s financial regulations. The ET made efforts to meet GAT, but the Executive Director couldn’t be reached to provide insight on the support they received from NDI. On the other hand, CCD gave the ET an audience but did not acknowledge that NDI’s support helped their organization in any way. 4. Priorities are moving from leveraging existing institutions to get things done to making efforts to get government ownership of the program and therefore secure budgeting by relevant government institutions. For instance, LANW is seeking a partnership with Ministry of Humanitarian Affairs, Disaster Management and Social Development to secure grants for its activities and foster program continuity beyond the life of the EECSP activity. NDI consultants also suggested proposal writing and annual fundraising to sub-grantees to increase their fund base. YDI, YIAGA Africa, LHI, and IFA have received grants through these initiatives. It was also noted that the enhanced capacity of the sub-grantees had made them attractive to other donors. These sub￾grantees also noted that they have been able to attract funding on account of their visibility; only recently has NDI trained sub-grantees on fundraising and proposal writing. 8.2 SELF-SUSTAINABILITY WITHOUT DONOR SUPPORT 5. The cost of deploying PVT without donor support levels provided to date is currently unsustainable within Nigeria's CSO space. Leveraging resources via other groups to reduce costs and increase impact should be considered alongside trade-offs for how these arrangements may affect the design and execution of the PVT and perception of bias. 6. CSOs engaged in advocacy campaigns benefitted from EECSP, but all of them have been operating campaigns before the activity and will continue to do so after EECSP. CSOs will be able to sustain their campaigns without donor support, but only within their means. Larger CSOs may be able to conduct broader campaigns while smaller CSOs, such as community-based women’s groups, may be more focused on targeted campaigns (e.g., community hotline services). 7. All the Objective 2 partners have developed sufficient technical and managerial capacity to continue activities, but still require some level of donor support to pay for campaign and program expenses such as hiring of audit firms, venues, logistics, and other miscellaneous expenses. 8.3 RECOMMENDATION - NEW CIVIL SOCIETY ELECTORAL ACTIVITY In a complex ecosystem like Nigeria, sources of social capital are changing. Through myriad civil society activities and relationships, some actors, such as faith and religious cultures as well as social media communities and networks, are starting to play an enhanced role. 19 It is in light of the shifting roles and forms of CSOs that a new CSO activity should be designed. 8.3.1 Expand Partnerships to Include Informal Institutions and CBOs for Deeper and more Sustained Impact. Previously mentioned under Evaluation Question 3, new activities should consider roles for traditional, informal, and religious institutions in the project design and implementation. This recommendation was made by some of our respondents during the interviews. For impact and sustainability, traditional 19 WEF, Future Role Civil Society Report 2013.pdf 55 institutions, who have significant influence on policy-makers, need to become part of the design; this ensures home-grown tactics and local ownership. USAID can advance action in this area by piloting a low-risk intervention whereby a system of due diligence, applicable to informal institutions, is applied. 8.3.2 Strengthen Governance and Sustain Impact. This recommendation offers a two-pronged solution. It forces government agencies to adopt practices that support open government and fight corruption. Secondly, it demonstrates uniformity in donor standards and expectations to CSOs and local partners. These subsequently contribute to the quality and impact of programs overall. For instance, there should be a charter of demands from donor agencies, like USAID, for government institutions being supported by donor funds. These should include things like having a Freedom of Information request portal, compliance with Public Procurement Act, Annual Audit Report, Whistleblower Policy. These standards need to be applied jointly with the donor community in general. 8.3.3 Expand Local Capacity The model used for developing local human capacity for the campaigns and the PVT were successful. Despite these successes, anecdotal evidence showed that some respondents had weak ownership of the knowledge. For instance, there were repeated references to the external/international experts who trained them and constant reference to NDI’s work. While attribution is important for INGOs, it weakens the confidence of local partners, creating a fragile sense of ownership which limits impact in the long run. To scale on a new program, the ET recommends a deliberate and significant utility of local experts to build local capacity where that option is the best technical available option. Not only does this reduce program cost, it enhances local trust and ownership, thereby faster adoption, assimilation, and replication. Another value of using local experts is that being grounded in a shared culture, language, and context makes a real difference. 8.3.4 Develop an Election and Political Sensitization Activity for Local Communities Targeting Youth and Women This intervention should be combined with a livelihood intervention, especially for rural women. Despite the successes of the women campaigns, there is still the challenge of the effect of poverty on the participation of marginalized groups in the political process. Tackling these constraints will address fundamental problems and will be sustainable. 8.3.5 Donors Need To Engage Better And More Strategically Evidence shows that stakeholders view donors as running programs at arm’s length. For future activities to be more effective, USAID can encourage other donors to engage more directly in order to have a deeper awareness of the dynamics and the complexities of political, cultural, and socio￾economic issues that affect election outcomes. Understanding these factors will help to design activities that will be more responsive to the needs of the Nigerian electorate. Secondly, such donors should be more engaged with local CSOs and demand more accountability, given the significant funding and resources they have provided. Donors should understand the channels their funding flows through and be aware of how partners, sub-grantees, and beneficiaries are meeting the activity’s goals and objectives. 56 8.3.6 CSOs Need To Interface with INEC. As the entity primarily responsible for the validity and legitimacy of Nigeria’s elections, the role of INEC in the political process cannot be understated. Thus, CSOs should engage INEC to ensure that the commission establishes and implements processes and systems to achieve electoral empowerment for all individuals. For example, while INEC can claim some accomplishments in supporting the abilities of PWDs to vote, the AAR for the Access Nigeria campaign following the 2019 general election showed that INEC failed to meet 23 of the 36 objectives and only achieved five objectives (five other objectives were partially met while three were undetermined). INEC has quarterly roundtables to update CSOs on progress regarding PWD issues, but the CSO community should hold INEC accountable in its responsibilities to ensure participation of PWDs in the electoral process. ********* -/- ********* 57 ANNEXES ANNEX A: EVALUATION STATEMENT OF WORK ELECTORAL EMPOWERMENT OF CIVIL SOCIETY PROGRAM (EECSP) STATEMENT OF WORK (SOW): FINAL PERFORMANCE EVALUATION I. ACTIVITY IDENTIFICATION DATA Development Objective Activity Title Strengthened Good Governance Electoral Empowerment of Civil Society Program (EECSP) Award Number Award Dates AID-620-A-14-00003 April 1, 2014 – March 31, 2020 Type of Agreement Total Estimated Cost Cooperative Agreement $30,600,000 Agreement Officer’s Representative (AOR) Implementing Partner (IP) Beatrice Reaud National Democratic Institute (NDI) I. ACTIVITY BACKGROUND: Program Goals and Objectives Awarded April 1, 2014, the Electoral Empowerment of Civil Society Program (EECSP) is a consortium of NDI, as the lead partner, with World Learning (WL), Search for Common Ground (SFCG) and the Center for Democratic Development-Ghana (CDD-Ghana) that is currently working primarily nationwide, with targeted interventions in selected states. The EECSP activity’s overall goal is to promote sustainable long-term civil society engagement in electoral and political processes. EECSP has supported civil society organizations that have produced tangible improvements in the capacity of these institutions to lead successful efforts in increasing participation in the Nigerian electoral process. These efforts include enacted legislation that decreases the minimum age required for Nigerians to run for office, the Independent National Electoral Commission’s (INEC) use of tactile ballot guides to increase the participation of persons with disabilities, and monitoring efforts in seven off-cycle gubernatorial and the 2019 general elections. Despite severe challenges with logistics that led to the one-week postponement of the 2019 general elections and several supplementary elections needed to address irregularities and localized violence, the PVT and subsequent domestic observer reporting concluded that INEC’s management of the elections was credible. The original total estimated amount (TEA) for this award was $24.0 million. However, EECSP was extended for one year from April 1, 2019 to March 31, 2020 and this TEA increased by $6.6 million to $30.6 million, of which a total of $11.7 million will be contributed by UKAID’s Department for International Development and extended the end date of the period of performance from March 31, 2019 to March 31, 2020. Strategic Approach and Objectives Voice, Space, and Accountability (VSA), A Theory of Change: The VSA theory of change rests on the intrinsic right of citizens to be heard and the power of organized citizens to drive democratic change. Voice refers to the quality and quantity of citizens’ expression of their interest; 58 space refers to the avenues available to citizens to exercise their voice; and accountability refers to the mechanisms by which citizens can make electoral actors responsive to their concerns. Often, citizens’ voice in Nigeria is loud but ineffective, chiefly because in some cases citizen groups are not sufficiently organized and their messages are not well targeted and evidence-based. The space for marginalized groups, such as women, youth, and persons with disabilities, is limited, and their voices are often absent. The upsurge of violence in various parts of Nigeria also continues to threaten opportunities for engagement. Lack of accountability, inclusiveness and unresponsiveness of elected officials to citizens’ concerns constitute some of the main democratic deficits in Nigeria. The program proposed complementary civic participation efforts to assist CSOs to employ evidence￾based methodologies—such as the PVT, focus groups, and surveys—to collect accurate and timely information on the electoral process and develop targeted messages advocating for accountability of political institutions and actors. The program would identify the primary barriers to meaningful civic participation and assist marginalized groups to develop strategies for participation that take these specific needs into consideration. Improving the quality and inclusiveness of citizen voices would set the stage for greater government accountability so that over time, new norms and more responsive institutions could emerge and contribute measurably to electoral integrity, peaceful elections, public confidence and inclusion of marginalized groups. In order to increase citizen participation and democratization, the EECSP Consortium adopted the following three objectives: 1. Improve impartial observation, accurate reporting on, and recording of election vote totals in 2015 and beyond; 2. Improve management within Nigerian partner CSOs; and, 3. Increase participation by marginalized groups in Nigerian political processes. NDI is the lead implementing partner and brings more than 30 years of experience from around the world working with civil society to monitor elections. Today more than four million citizen monitors in over 90 countries engage in this effort. NDI contributes a wealth of global experience and comparative best practices in democracy and governance work and has worked with local partners to share ideas, knowledge, experiences, and expertise in 130 countries and territories. NDI has supported Nigerian partner organizations to conduct 17 PVTs since 2011. For Objective 1, to foster South-South learning and regional collaboration and sustainability, NDI partners with the Ghana Center for Democratic Development (CDD-Ghana), with which the Institute has had a working relationship since 1998. CDD-Ghana, an independent, non-governmental, and non-profit research and advocacy institute dedicated to the promotion of democracy and good governance, has led the successful organization of PVT activities in Ghana since 2008 and is seen as a regional leader in election monitoring. NDI partnered with World Learning, and later KPMG, for Objective 2. NDI partners with Search for Common Ground (SFCG) under Objective 3 to develop conflict-sensitive voter education and reduce the incidence of electoral violence. SFCG has been engaged in Nigeria since 2004, modeling innovative media products that foster ethnic and religious tolerance and promote gender equality. 59 NDI has also worked with a number of local organizations to implement its various lines of effort, which include: Under Objective 1: ● National and off-cycle gubernatorial PVTs (TMG and YIAGA Africa) Under Objective 2: ● Organizational capacity building (through World Learning, Development Alternatives and Resource Center, KPMG) Under Objective 3: ● Access Nigeria campaign (Inclusive Friends Association) ● Misinformation and hate speech (CDD-West Africa), ● Vote Not Fight campaign (Youngstars Development Initiative) ● Countering dangerous speech (Center for Information Technology and Development (CITAD) ● Youth Advocacy (Kimpact Development Initiative, Media Awareness and Justice Initiative, and One Project Afrika) ● Stop Violence Against Women in Politics (Gender and Development Action and Legal Awareness for Nigerian Women) ● IDP voter education (Community Links and Human Empowerment Initiative and Forward in Action Education Poverty and Malnutrition). Existing Data The evaluation team will have access to the program description, performance monitoring plans (PMPs), annual work plans, quarterly and annual reports, final reports and any other relevant materials documenting the management, implementation process and results for EECSP permitted by the Office of Acquisition and Assistance. The Evaluation Team will also work with USAID/Nigeria and EECSP to determine what other relevant items related to products funded by assistance (i.e. PVT statements) may be included in determining the impact and outcome of EECSP. II. EVALUATION PURPOSE Evaluation Purpose The contractor shall conduct an evaluation in the final year of this six-year activity and provide a report of the findings of EECSP implemented by EECSP Consortium. While this evaluation is intended to gauge if EECSP has achieved its objectives, what are the specific outcomes, and what challenges EECSP faced that affected its ability to achieve its objectives, the ultimate purpose of the evaluation is to use these findings to inform future programming. USAID intends to incorporate the evaluation findings into the design of a new civil society elections activity. USAID would like DevTech to use the outcome harvesting approach in the evaluation across the three objectives of EECSP listed above. Initial planning suggests that there may be up to 21 outcomes worth exploring, with over 40 stakeholder organizations/key informants that might be interviewed across about 14 states and the Federal Capital Territory (see Tab 1). USAID would like to review these with DevTech and then scope the evaluation appropriately. Specifically, this external evaluation will: ● Develop key evaluation questions and a methodology that will answer these questions that will focus on the following areas: o How effective the outcomes identified (and others that may have arisen from interviews) were in addressing three objectives 60 o What factors (both attributable to EECSP and outside of EECSP) led to those outcomes; o What were key impediments to achieving the outcomes; o How sustainable will the institutional management capacity of partners (Objective 2) be beyond the life of EECSP; o How self-sustainable are the efforts undertaken by the CSOs without donor assistance. ● Evaluate progress made by partners over the life of the project in all three EECSP objectives (April 1, 2014 to date); and, Revised Evaluation Questions ● Evaluation question 1: To what extent have the election observation mechanism (PVT and PREO) adopted by the activity influenced impartial observation, accurate reporting and recording of vote totals (April 1, 2014 to date)? o What factors led to the election observation outcomes and what were the key impediments to achieving the outcomes? o What is the perception of stakeholders, particularly civil society organizations (CSO) regarding the utility of the PVT? What can be done differently? ● Evaluation question 2: To what extent have NDI partners management skills (objective 2) influenced their capacity to implement the PVT and campaigns towards increased marginalized groups participation in Nigerian political processes (April 1, 2014 to date)? o What factors led to the key outcomes achieved in improving management within partner CSOs and what were the key impediments to achieving the outcomes? o Are partners (particularly the PVT subgrantee) implementing the new skills or recommendations from NDI’s capacity assessment? ● Evaluation question 3: To what extent have marginalized groups (women, youth, persons with disabilities and internally displaced persons) campaigns affected their participation in Nigerian political processes (April 1, 2014 to date)? o Specifically, which of the marginalized groups campaigns have been successful and which were not? o What factors led to the campaign outcomes and what were the key barriers to achieving successful campaigns? ● Evaluation question 4: To what extent can the developed institutional management capacity of partners be sustained beyond the life of the activity? o How self-sustainable are the efforts undertaken by the CSOs without donor assistance? Audience and Intended Users The primary users of the evaluation findings are the USAID Peace and Democratic Governance Office (PDG), other Mission colleagues, and other U. S. Government (USG) officials, as deemed appropriate by USAID/Nigeria. Other audiences could include non-USG donor organizations, the Government of Nigeria (GON), and CSOs, all of whom can use the findings to improve democracy and governance and human rights programming in Nigeria. The report will also serve as a reference material for program design, implementation, monitoring, evaluation and management of future PDG activities. IV. EVALUATION METHOD AND METHODOLOGY Evaluation Design This evaluation could use a mix of qualitative and quantitative approaches on the basis of discussions between DevTech and USAID/Nigeria. It is expected that the team will need to gather data in Abuja, 61 Benue, Adamawa, Kogi, Bayelsa, Rivers, Kaduna and other states/sites to be determined by USAID/Nigeria. Data Collection Methods The key approaches that will be used to collect and analyze data for the evaluation, as well as the questions, will be proposed by DevTech and then discussed and possibly modified through an iterative process with USAID/Nigeria. This evaluation will use key informants, focus groups, and outcome harvesting to map how outcomes were achieved to determine what interventions might be replicable. To carry out the work, MEL will form a core team comprised of a Team Leader (TL)/Senior Evaluation Specialist (SES) and a Mid-level Evaluation Specialist and two Junior-Level Evaluation Specialists; MEL Specialist will provide quality control and logistics for data collection in the 14 states and Abuja: ● North Central: Kogi, Benue, FCT; ● Northeast: Adamawa; ● Northwest: Kaduna, Sokoto; ● Southeast: Anambra, Enugu; ● South-South: Akwa Ibom, Bayelsa, Rivers; ● Southwest: Lagos, Ekiti, Ondo, Osun. Workplan: The evaluation will be conducted in four phases over a period of five months. Phase One: Preparation for the evaluation will start immediately after notification from the COR that MEL will implement this task. This phase will include identification and recruitment of the evaluation team, attending contractual matters, and initial logistics. Duration of this phase is up to two weeks. Phase Two: Preparatory work will start as soon as the evaluation team is contracted. The MEL DCOP, COP, MEL Specialists and the evaluation team (TL/SES, mid-level evaluation specialist and local evaluators) will have a kick-off phone or video conference with USAID. The TL/SES and the evaluation team members, as they are recruited, will begin the document review immediately after the kick-off and will begin refining the evaluation workplan (EWP). The evaluation team will submit the draft EWP to the mission within three weeks after the kick-off call. The final EWP will be submitted to the Mission within a week after receiving comments on the draft. The EWP will include draft data collection instruments such as key informant interview protocols, structured survey questions, and focus group discussion guides and protocols. If necessary, the MEL Activity will apply for an Internal Review Board (IRB) review of protocols to demonstrate the evaluation is carried out ethically. Duration of this phase is up to four weeks. Phase Three: Field work will start immediately after the EWP is approved. MEL plans to mobilize the evaluation team to begin field work within a week of EWP approval. Parallel with mobilization, the evaluation team will continue refining data collection tools and protocols. The TL/SES will travel to Nigeria, and the local Mid-Level Evaluation Specialist and two local evaluators will travel to Abuja for an in-brief presentation at the Mission. While in Abuja, the MEL Activity will provide a data collection training on KIIs and FDGs with the evaluation team and local evaluator and enumerator consultants. The team will jointly conduct the interviews in Abuja for two weeks to better understand the activity and the perspectives of the implementing partners, subgrantees and other Abuja-based stakeholders. Following these initial activities, the team will split into two groups, each including team leader or mid￾level evaluator and junior evaluator to travel to the eleven states (phone interviews will be conducted for AkwaI-Ibom and Sokoto CSO partners, while the Enugu partner will be invited to Anambra State). According to plan, the team will spend two weeks in Abuja conducting KIIs and FDGs among key 62 government, media, CSO and USAID stakeholders. Each group will then spend about three weeks conducting structured surveys (preferably electronic, paper based if appropriate) among CSOs, FGDs among CSOs members and CSO beneficiaries and KIIs among key stakeholders (GON, local government, USAID, DFID CSOs, media, political parties, security). The team will re-assemble in Abuja to implement remaining KIIs, discuss preliminary finding, and present these findings to USAID stakeholders. The field work phase is not expected to exceed six weeks, though local team members may continue collecting additional data from sources that were previously unavailable after the departure of the TL. Duration of this phase is up to six weeks. Phase Four: Analysis and Report Writing starts immediately after the team returns to Abuja. The team continues analysis begun during field work, drafts report, two-pager, develops debrief PowerPoint and submits report to MEL before TL/SES departs. MEL submits the debrief PowerPoint to USAID within two weeks after field work and transmits draft report and two-pager to Mission within three weeks after the end of field work. The team submits the final report within three weeks of receiving comments from the Mission. The expected duration of this phase is six weeks, including the time needed for the mission to collect comments on the draft from relevant stakeholders, if desired. Phase Five: Two members of the evaluation team (one mid-level evaluation specialist and one local Evaluation Specialist will attend the dissemination meeting with USAID after the final report is approved. Both will develop the final report PPT and share with USAID three days before the dissemination. The expected duration of this phase is two weeks, including travel time for the evaluation team. Data Collection Methods Systematic Desk Review MEL will conduct document reviews to gain a greater understanding of EECSP deliverables and literature related to electoral processes that have been implemented 2014-present. The review will be used to inform the design of the data collection tools and to provide information about the target population (electorates). Document reviews will also provide extensive information on specific challenges and opportunities in Nigeria’s electoral process that will inform future programming. The initial document and literature review will be conducted during the first week of the evaluation by the TL/SES, Mid-Level Evaluation Specialist and two Local Evaluation Specialists. It will be based on EECSP program documents since 2014, CSO policies, GON policies and laws on electoral processes, and evaluations or assessments dedicated to Nigeria’s electoral process. The review will also include data sets, such as the PVT results, peer review articles, and grey literature, as appropriate. The document review will also include documents suggested by key informants. Key Informant Interviews (KII) Key informant interviews (KII), the principal data collection method for this evaluation, will be used to deepen our understanding of the challenges within the Nigerian electoral process, the trends and expectations of different USAID, government and civil society stakeholders, as well as beneficiaries from targeted groups, i.e. Nigerian marginalized groups. The KIIs will help to understand their views and contribution to EECSP activities, and their assessment of the challenges, successes and unintended consequences or benefits of program implementation. There are five major groups of key informants that will be interviewed for this evaluation: (1) USAID PDG staff; (2) EECSP consortium program staff (NDI, CDD-Ghana, World Learning, Search for Common Ground); (3) CSOs (TMG, YIAGA Africa, Inclusive Friends Association, CDD-West Africa, Youngstars Development Initiative, Center for Information Technology and Development, Kimpact 63 Development Initiative, Media Awareness and Justice Initiative, One Project Afrika, Gender and Development Action and Legal Awareness for Nigeran Women, Community Links and Human Empowerment Initiative, and Forward in Action Education Poverty and Malnutrition, among others); (4) Media and (5) GON key stakeholders. The evaluation team will be looking for appropriate key informants at the international, national and state level, which may include program implementers (Ghana Center for Democratic Development, NDI, World Learning, Search for Common Ground), Independent National Electoral Commission (INEC) officials, CSOs, or representatives of USAID, and other donor projects. The initial list of key informants will be developed based on information provided by USAID. The assessors will also use “snowball” techniques to identify additional relevant key informants during initial interviews. MEL specialists anticipate that about 300 KIIs will be conducted in Abuja, and about four KIIs in each of the 14 states -– about 74 KIIs in all. Interviews will be guided by separate open-ended interview schedules for each group of informants. This will include targeted inquiries to collect precise information to answer specific evaluation questions, as well as more open-ended questions aimed at better understanding the how and why of EECSP activities, and the impact of the initiatives. DevTech will obtain all necessary approvals and adjust to changes as requested by the COR and other key stakeholders. Interviews will be designed to be completed within one hour. KIIs will commence as soon as the evaluation methodology is approved by the COR. Key informants will be contacted in advance to schedule interviews and will be provided a copy of the KII schedule and questionnaire, if required. If any of the planned KIIs are not possible in-person due to accessibility, security or availability issues, the team will make a follow-up attempt to interview the key informant in-person or over the phone. To assure accuracy and efficiency, the interviews will be conducted by two person teams working in parallel. Each team will consist of the Team Lead/SES or a Mid-Level M&E expert, and one Nigerian junior team member. KII information will be analyzed as soon as possible after the interviews are completed. Analysts will identify the most typical responses to the questions and will tabulate them. A separate list of “outlier” responses will be created for further investigation by the evaluation team. Follow-up interviews or emails may be conducted if clarification is needed. All data collection and reporting will abide by the ethical principles of informed consent, respect, sensitivity, do no harm, non-discrimination, and confidentiality. Interviewers will read a brief statement about the purpose of the study and voluntary participation in the interview and will answer respondents’ questions before proceeding. In addition, if needed IRB approval will be obtained to protect interviewee sensitive information. Focus Group Discussions (FGD) The evaluation team will conduct FGDs among CSOs and CSO beneficiaries, as well as EECSP consortium program staff. FGDs will ascertain the most important issues facing marginalized groups participating or not participating in the Nigerian political process, as well as CSO organizational capacity and management. These FDGs will provide insight into challenges to voter participation for specific groups, as well as how well trained electoral CSOs are in Nigeran election processes. The team will conduct FGDs among ten CSOs funded by EECSP that focus on increasing electoral participation for marginalized groups, as well as conduct FGDs with CSO beneficiaries. CSO beneficiaries will be divided into focus groups based on demographic group (women, youth, IDP, disabled, etc.) to ensure each group feels comfortable expressing their viewpoints. The team will also conduct FGDs with NDI, World Learning, Development Alternatives and Resource Center, KPMG staff and their training beneficiaries to understand the training techniques, resources provided and if knowledge was obtained by staff. In addition to the FDGs, the evaluation team will request CSO policies/protocols to support any claims of organizational changes. The team will try to conduct focus 64 groups at the CSO headquarters’ LGA, in the states in all of the 11 states. Ideally, there will be 10 to 15 participants in each FGD, balanced between male and female participants. If necessary, however, depending on cultural norms, FGDs will be segregated by gender or age. Each FGD will last from one and a half to two hours. The FGDs will be carried out by one core evaluation team member, supported by a MEL specialist and a local evaluator. The core team (with the team lead) will visit six states, the second team will visit five states. The results of FGDs will be tabulated and analyzed for most common themes and issues. Surveys Structured electronic surveys will be sent out to each CSO’s office to complete in addition to conducting KIIs and FDGs. The structured surveys will provide the evaluation team with insight on what each CSO does for EECSP, and the challenges, successes and potential for life-after-program sustainability opportunities. Surveys will be sent via email to a designated point person, as specified by ECCSP program staff, at the CSO. If the CSO cannot complete the electronic version, a paper version will be provided to the CSO when one the evaluation teams are at the CSO office, either in Abuja or in one of the 14 states. CSO survey data will strengthen the validity of data collected on observations, KIIs and FDGs. Estimated Level of Effort (LOE) Based on the proposed budget and implementation schedule, DevTech estimates the following LOE for team members: LOE (days) Phase 1 Phase 2 Phase 3 Phase 4 Phase 5 TOTAL Team Leader (TL)/Senior Evaluation Specialist (1) 0 15 36 21 0 72 Mid-Level Evaluation Specialist (1) 0 15 36 18 0 69 Local Evaluation Experts (2) 0 20 68 26 10 124 Home Office Program Manager 7 1 2 2 1 13 TOTAL 7 51 142 67 11 278 Travel KIIs and FDGs will be conducted in two teams, Team A will travel to the South South (Bayelsa and Rivers), South East (Anambra) and North Central (Benue and Kogi) to conduct KIIs and FDGs and Team B will travel to the North West (Kaduna), North East (Adamawa) and South West (Ekiti, Lagos, Ondo, and Osun). The team will be made up of one Team Lead or Mid-Level Evaluation Specialist and Junior Evaluation Specialist. The MEL Activity MEL Specialists will join each of the teams. Deliverables The following deliverables will be submitted to USAID/Nigeria. The timeline for submission of deliverables will be finalized and agreed upon during the team planning meeting: 65 1. Evaluation work plan and timeline 2. Literature Review of Existing Data 3. Questionnaires/surveys/guidelines for conducting key informant interviews/focus group discussions: These documents will be prepared during the team planning meeting and submitted to USAID/Nigeria for review and approval prior to the initiation of key informant interviews and site visits. 4. List of proposed sites and list of respondents: USAID/Nigeria will support the process of producing a list of sites to be visited and a list of respondents to be interviewed as key informants. 5. Interview notes and completed surveys: All interview notes and completed survey instruments used for the evaluation will be submitted to USAID/Nigeria. The data used for analysis will also be submitted to USAID/Nigeria. 6. Debriefing(s): The Team Leader will regularly debrief USAID/Nigeria on the progress made during fieldwork. At the end of fieldwork and prior to departing Nigeria, the team will debrief USAID/Nigeria on its findings, conclusions and recommendations. 7. Draft Evaluation Report: The team will provide USAID/Nigeria with a draft report that includes all the components of the final evaluation report prior to their departure from Nigeria. USAID/Nigeria will provide written comments on the draft report to the team within ten working days of receiving the draft report. 8. Final Evaluation Report: The final report will address the comments provided by USAID/Nigeria and other stakeholders on the draft report. The Evaluation Team Leader will revise the draft report and deliver a final revised version to USAID/Nigeria within three calendar weeks of receiving USAID feedback. The final report in both hard and electronic format will be submitted to USAID/Nigeria and approval given before submission to the Development Experience Clearinghouse (DEC). 9. A summary of "key steps in the evaluation, key findings, recommendations and lessons learned, next steps" and a PPT of these. 10. Two - pager summary of the evaluation report. Reporting Guidelines USAID’s evaluation policy requires that all evaluation SOWs include USAID’s Criteria to Ensure the Quality of the Evaluation Report (see USAID Evaluation Policy, Appendix I). The policy also indicates that the report will outline in detail, any additional expectations USAID has regarding a report’s structure, format, and length. The report, not including appropriate and relevant annexes, should be between 25 and 30 pages with an executive summary. The format for the evaluation report is as follows (number of pages is illustrative): ● Executive Summary (2 pp.) ● Table of Contents (1 pp.) ● Introduction (1 pp.) ● Background (2-3 pp.) ● Methodology (1 pp.) 66 ● Findings/Conclusions (17- 20 pp.) ● Issues and Challenges (1-2 pp.) ● Recommendations/Future Directions (5-10 pp.) ● References ● Annexes The draft report and final report will be submitted electronically. All reports will be in English. The report must: ● Distinguish clearly between findings, conclusions (based strictly on findings) and recommendations (based clearly on the evaluation findings and conclusions); ● Comply with USAID’s Evaluation Policy ● Ensure submission to the DEC after finalization. Dissemination Plan It is expected that USAID/Nigeria will plan a dissemination session with other technical units within the Mission, implementing partners, donor community as well as other stakeholders. VI. TEAM COMPOSITION Team Qualifications USAID/Nigeria will engage the services of a team of two local and one (or two) international consultants to work on the evaluation team. The team will require: An International Senior Evaluation Specialist/Team Leader (IQC Labor Category – Program Development, Senior) with experience in evaluating peace, democratic governance and human rights activities in developing countries. The team leader will finalize the evaluation design, coordinate activities, arrange meetings, consolidate individual input from team members, and coordinate the process of assembling the final findings and recommendations. S/he will also lead the preparation and presentation of the key evaluation findings and recommendations to USAID/Nigeria. At least eight (8) years of experience in evaluation management is required. Experience in conducting evaluations or assessments of peace and democratic governance as well as human rights programming in developing countries is required. Ability to produce a high-quality evaluation report in English is essential. A Local Evaluation Specialist – Mid (IQC Labor Category – Social Scientist, Mid) with at least six (6) years of experience in democracy and governance programs and some experience managing or implementing peace and democratic governance as well as human rights programming in developing countries is required. Experience preferred in assessing capacity-building efforts in nascent democracies; the role of civil society in strengthening citizen participation; and, involvement in holding governments to account. African/regional experience is desired. Ability to conduct interviews and discussions and write well in English is essential. For the two local evaluators, a good knowledge of gender issues as well as social inclusiveness is required. A fair knowledge of the operations of civil society. and election management bodies (INEC and SIECs). Team member experience should include graduate level economics, social science, law, and/or public finance training. In-depth knowledge of issues relating to Nigerian legislative functions and process improvement programs, improving governmental institutional capacity, and fostering civil society oversight of government institutions in developing and/or transitional economies and democracies is required. Some experience in conducting evaluations or assessments is expected. Ability to conduct interviews and discussions and write well in English is essential. 67 Overall, the team will need experience in USAID practices and expectations in program evaluation; program design and analysis; quantitative and qualitative data collection and analysis; survey design and analysis; program issues, innovations and challenges in building civil society capacity (including financial and personnel management, including value for money); gender-sensitivity; and USAID practices and requirements in program performance measurement. Conflicts of Interest All evaluation team members will provide signed statements attesting to a lack of conflicts of interest or describing an existing conflict of interest relative to the activity being evaluated. USAID/Nigeria will provide the conflict of interest forms. VII. SCHEDULING AND LOGISTICS The proposed evaluation will be funded and implemented by the Mission through The Learning Program/DevTech. DevTech will be responsible for all offshore and in-country administrative and logistical support, including identification and fielding appropriate consultants. DevTech will arrange and schedule meetings, international and local travels, hotel bookings, working/office spaces, computers, printing and photocopying. DevTech will make all logistics arrangements, including the vehicles for travels around Abuja and other cities and will not expect any logistics support from the Mission. DevTech will also make their own arrangements about space for team meetings and equipment support for producing the report. TIMELINE The evaluation team will present a draft timeline with level of effort for the consultants and products to USAID/Nigeria. While it is expected that the final evaluation should be completed by no later than September 2019, in order to inform the concept note design and subsequent program description process, it is expected that the timeline that will be jointly discussed and agreed to between DevTech and USAID/Nigeria will include regular debriefs that will be used to inform the design process prior to the final report submission by October 2019. USAID/Nigeria will provide the following support: ● Supervise the monitoring and evaluation (M&E) unit and ensure that the M&E advisor complies with the job requirements; ● Ensure that confidentiality of information is maintained and that files are well kept; ● Ensure that copies of relevant documents are available to the evaluation team; ● Create an enabling environment for using and learning from activity data and for promoting data￾and information-sharing among activity stakeholders and to interested external audiences; ● Oversee and support the above and troubleshoot as needed or requested. 68 ANNEX B: OUTCOME MAPPING 69 ANNEX C: DATA COLLECTION INSTRUMENTS C.1. KII QUESTIONNAIRE KII Guide and Questions I. Participant info Name: Organization: Role: Sex: II. Introduction (brief intro to why the KII) III. Interviewer’s guide: 1. Introduce 2. Listen to answers and ask follow-up questions 3. Feel free to add questions or change order, if necessary 4. Note keywords 5. Record quotes “word-for word” 6. For your personal observations and opinions, please record separately Probe prompts, if needed: a. Tell me more b. Why do you feel that way? c. Why was that important to you? d. How did that affect you? e. Can you explain what you mean by that? f. What was your contribution? g. What do you think was really making it work? h. How has it changed you? Thank you for your time and honest responses. Please feel free to email or call me if you would like to share more insights. 70 Evaluation Question 1 1. To what extent have the election observation mechanism (PVT and PREO) adopted by the activity influenced impartial observation, accurate reporting and recording of vote totals (April 1, 2014 to date)? 1a. What factors led to the election observation outcomes and what were the key impediments to achieving the outcomes? Questions Outcome(s) Claimed Stakeholder(s)/Respondents 1. How do you think the PVT and PREO election observation mechanisms affected election outcome in Nigeria Outcome 1: PVTs served to increase credibility of election results PVTs/PREO - National: 2015 & 2019 - Off cycle NDI staff ● YIAGA Africa Working Group ● TMG ● Past INEC Leadership ● INEC Leadership INEC Focus Group ● Special Assistant to the Chair, INEC ● INEC National Commissioner ● INEC National Commissioner ● INEC National Commissioner ● INEC National Commissioner ● INEC National Commissioner ● Yar’adua Centre, ● Situation Room ● Ondo and Osun INEC ● Anambra INEC ● Ekiti INEC ● Osun INEC ● Niger State INEC ● YIAGA Africa in Ondo, Anambra, Ekiti and Osun States. Media Focus Group, Abuja Contact of Reporters on WTV engagement ● Premium Times - ● Legit.ng - ● News Agency of Nigeria - ● Leadership ● Blueprint - ● Channels TV Lagos ● The Daily Trust, Editors ● FCRN ● Channels TV ● The Punch ● The Authority DFID, EU, British Council, and Other Donors 71 Outcome 3: PVT Recommendations used to inform recommendations for electoral law reform Outcome 3 • NDI staff • YIAGA Africa • INEC • IPAC Outcome 4: Changes to INEC procedures due to PREO and PVT findings Outcome 4 - NDI - INEC staff - INEC National Comm. - IFES 2. In what ways, do you think the PVT system contributed to: a) impartial observation; b) accurate reporting; and, c) recording of vote totals? Outcome 1: PVTs served to increase credibility of election results Outcome 3 - INEC Commissioners - DFID, EU, British Council, and Other Donors 3. In what ways, do you think the PREO system contributed to: a) impartial observation; b) accurate reporting; and, c) recording of vote totals? Outcome 3: PVT recommendations used to inform recommendations for electoral law reform Outcome 3 • NDI • YIAGA Africa • INEC • IPAC • DFID, EU, British Council, and Other Donors 4. i. What factors, if any, do you think positively contributed to your capacity to deploy PVT? ii. Which of these factors can be directly attributed to the EECSP intervention? Outcome 2: Growing evidence of local CSO capacity to deploy PVTs Outcome 2 • NDI • TMG • YIAGA Africa • CDD Nigeria • CDD -Ghana • YDI • YIAGA Africa in Ondo, Anambra, Ekiti and Osun states • DFID, EU, British Council, and Other Donors 5. i. What other factors, if any, do you think negatively affected your capacity or ability to deploy PVTs. ii. What can be done differently 1b. What is the perception of stakeholders, particularly civil society organizations (CSO) regarding the utility of the PVT? What can be done differently? Questions Outcome(s) Claimed Stakeholder(s)/Respondents 6. How relevant were the PVT training to your organization • TMG • YIAGA Africa • YIAGA Africa in Ondo, Anambra, Ekiti and Osun states 7. From your experience, how beneficial was the PVT system to the election process? Outcome 5: PREO and PVT by TMG provided initial data that led to subsequent Stop-VAWIP campaigns by CSOs. Outcome 5 • Former TMG • YIAGA Africa • NDI • DFID, EU, British Council, and Other Donors 72 8. What do you consider as the limitations of the system? How can these limitations be addressed? • Former TMG • YIAGA Africa • NDI • YIAGA Africa • YIAGA Africa in Ondo, Anambra, Ekiti and Osun states • DFID, EU, British Council, and Other Donors 9. Based on your experience, do you have recommendations, to improve the PVT system and support improved impartial observation, accurate reporting and recording of vote totals? • Former TMG Staff • YIAGA Africa • NDI • DFID, EU, British Council, and Other Donors Evaluation Question 2 To what extent have NDI partners management skills (objective 2) influenced their capacity to implement the PVT and campaigns towards increased marginalized groups participation in Nigerian political processes (April 1, 2014 to date)? 2.a What factors led to the key outcomes achieved in improving management within partner CSOs and what were the key impediments to achieving the outcomes? Questions Outcome(s) Claimed Stakeholder(s)/Respondents 10. Please describe your organization’s decision making process. Describe briefly, how priorities are set and communicated? Outcome 8: CSO partners took steps to improve and diversify governance and advisory boards and executive management Outcome 8 • NDI • YIAGA Africa, LHI, Face Initiative, GAT, WACOL, IFA, YDI • DFID, EU, British Council, and Other Donors 11. How do you ensure representation and balance? Outcome 8: CSO partners took steps to improve and diversify governance and advisory boards and executive management Outcome 8 • NDI Staff • YIAGA Africa • LHI, GAT, WACOL, IFA, YDI • DFID, EU, British Council, and Other Donors 12. Which of your management and accountability system(s) are directly as a result of the EECSP support? Outcome 7: EECSP consortium, through World Learning and NDI, established financial and procurement management systems within local CSOs. Outcome 7 • NDI Staff • WL Staff Washington, DC • IFA, YIAGA Africa LHI, GAT, WACOl, YDI. • IFA. • DFID, EU, British Council, and Other Donors 13. CSO: How are you sustaining the improved changes to your financial, procurement and HR management systems? KPMG 1: what were the identified lapses within the CSOs prior to KPMG’s intervention and how were Outcome 9: Sequential KPMG assessments identified improved financial, procurement and HR management systems within local CSO partners. Outcome 9 • NDI • KPMG • LHI • YIAGA Africa • FACE Initiative 73 they addressed? KPMG 2: What factors led to these improvements? • GAT, WACOL, YIAGA Africa, YDI • DFID, EU, British Council, and Other Donors 14. What is your organization’s fundraising capacity? Is your improved fundraising capacity directly attributable to the EECSP support? Outcome 10: Improved CSO partner capacities to raise funds Outcome 10 • NDI • Life Helpers Initiative • FACE Initiative • GAT, WACOL, YIAGA Africa, YDI • DFID, EU, British Council, and Other Donors a. Are partners implementing the new skills or recommendations from NDI’s capacity assessment? Questions Outcome(s) Claimed Stakeholder(s)/Respondents 15. How have you responded to NDI’s capacity assessment feedback on your governance structure? Outcome 8: CSO partners took steps to improve and diversify governance and advisory boards and executive management • NDI • LHI • YIAGA Africa • FACE Initiative • GAT, WACOL, YIAGA Africa, YDI • DFID, EU, British Council, and Other Donors 16. What are challenges, if any, do you face with adopting new skills and enforcing change within your organization? 17. What factor, outside your organization’s control, continue to affect the performance of your organization? Evaluation Question 3 To what extent have marginalized groups (women, youth, persons with disabilities and internally displaced persons) campaigns affected their participation in Nigerian political processes (April 1, 2014 to date)? a. Specifically, which of the marginalized group campaigns have been successful and which were not? Questions Outcome(s) Claimed Stakeholder(s)/Respondents 18. Which advocacy campaigns did your group advocate for? • YIAGA Africa • LHI • FACE Initiative • GAT, WACOL, YIAGA Africa, YDI • Other objective 3 partners • DFID, EU, British Council, and Other Donors 19. i. How successful were your campaigns? ii. VAPP Kaduna: How is the Outcome 6: Women successfully recruited and deployed as 30% of total PVT observers. Outcome 6 • NDI • YIAGA Africa 74 implementation of the VAPP law? iii. VAPP Enugu: what efforts are ongoing to secure the assent of the VAPP bill? • LHI • FACE Initiative • GAT, WACOL, YIAGA Africa, YDI • Other objective 3 partners • DFID, EU, British Council, and Other Donors Outcome 11: Deepened capacity of local CSOs to conduct and sustain policy advocacy Outcome 11 PWD Programs • NDI • Inclusive Friends Association (IFA). • IFA. • Youngstars Development Initiative (YDI) • YDI. • KIMPACT Development Initiative (KDI) • Media Awareness and Justice Initiative (MAJI). • One Project Afrika (OPA) Women’s Programs • NDI Staff • NDI • NDI • Legal Awareness for Nigerian Women (LANW) • Gender Awareness Trust (GAT) • Community Links and Human Empowerment Initiative (CLHEI) • Women’s Aid Collective • DFID, EU, British Council, and Other Donors Outcome 12: Increased youth candidacy in 2019 elections through successful advocacy campaign leading to passage of the NTYTR constitutional amendment Outcome 12 • NDI • YIAGA Africa • Connected Development. • Ondo State NTYTR Coordinator. • Kogi State NITTTR Coordinator. • Osun State NTYTR coordinator. • Anambra State NTYTR Coordinator. • Benue State NITTTR coordinator. • NASS sponsor for the NTYTR bill in the House of Reps 75 • DFID, EU, British Council, and Other Donors Outcome 13: Increased participation of marginalized groups in elections and political process (Youth - VNF, PWD, Women and IDP) Outcome 13 Youth • NDI Staff • YDI. • Kimpact Development Initiative (KDI). • 2Baba Foundation • PWD • Inclusive Friends Association (IFA). • INEC Chairman. • INEC National Commissioner. • INEC National Commissioner • Joint National Association of Persons with Disabilities (JONAPWD). • IDP • NDI Staff • Community Links and Human Empowerment Initiative (CLHEI) • INEC, Resident Electoral Commissioner • Centre for Community Health and Development International (CHAD) • INEC Resident Electoral Commissioner • DFID, EU, British Council, and Other Donors Outcome 14: CSO advocacy campaigns led to VAPP law enactment (Kaduna) and legislative passage (Enugu). Outcome 14 • NDI Staff • Legal Awareness for Nigerian Women (LANW) • Gender Awareness Trust (GAT) • Women’s Aid Collective • A State House of Assembly member • DFID, EU, British Council, and Other Donors 20. If successful, what factors do you think contributed to your success? Outcome 11: Deepened capacity of local CSOs to conduct and sustain policy advocacy Women, Youth, PWD and IDP stakeholders listed above 76 Outcome 13: Increased participation of marginalized groups in elections and political process (Youth - VNF, PDW, Women and IDP) 21. If unsuccessful, what factors contributed? Outcome 11: Deepened capacity of local CSOs to conduct and sustain policy advocacy Women, Youth, PWD and IDP stakeholders listed above Outcome 13: Increased participation of marginalized groups in elections and political process (Youth - VNF, PDW, Women and IDP) 22. What factors influenced your choice of policy advocacy campaigns and strategies? Women, Youth, PWD and IDP stakeholders listed above 23. In what ways did you ensure that the interest of your marginalized groups was represented at all times? Women, Youth, PWD and IDP stakeholders listed above 24. What systems did you put in place to provide feedback and gather input (ensure participation and representation)? Women, Youth, PWD and IDP stakeholders listed above 25. Do you have any ongoing partnerships with government? Share the objectives and goal of the partnership. Outcome 15: Fostered CSO￾government partnerships to independently sustain state-level efforts to combat violence against women in elections and politics • DFID, EU, British Council, and Other Donors 26. i. If yes, what factors have fostered that partnership with government? ii. If no, what factors mitigated against it? If any? Outcome 15: Fostered CSO￾government partnerships to independently sustain state-level efforts to combat violence against women in elections and politics b. What factors led to the campaign outcomes and what were the key barriers to achieving successful campaigns? Questions Outcome(s) Claimed Stakeholder(s)/Respondents 27. i. Why was the conference on government transition organized? ii. What led to the development of the draft law and advocacy campaign to regulate executive transitions in Nigeria? iii. What is the status of this bill today? Outcome 16: Conference on government transitions (Nov 2015) led to a draft law and advocacy campaign to regulate executive transitions in Nigeria, which would improve national governance. 77 Note on Outcomes: ET reviewed the activity outcomes and reduced them from 23 to 16 for the following reasons: Outcome deleted Note 1. Successful PVTs Considered redundant, especially as following outcome would allow data collection to produce result. 2. NOA identified for underperformance in its voter education responsibilities leading to its increased performance. On the grounds that it does not explicitly help to understand what is intended and NOA did not come up significantly as a role player under the objective. It was noted that NOA may be interviewed if there is sufficient time 3. Improved and diversified communications practices by CSO partners Considered ambiguous. Also, because corresponding outcomes under this objective probed deeper into improved organizational practices like HR, procurement, governance, etc. all of which produce info on communication as well. 4. Increased youth participation in elections through peacemaking efforts, youth voter registration and PVC collection (VNF campaign) Increased and diversified PWD participation in elections through the Access Nigeria: Disability Votes Matter Campaign. All of these outcomes were merged into one since they all focused on the subject of increased participation through the various campaigns: Increased participation of marginalized groups in elections and political process (Youth - VNF, PWD, Women and IDP). More questions were framed to collect specific group data. 28. What advocacy tools did you apply? All EECSP advocacy partners 29. Which tools or strategies do you consider to be most useful? What were/are the limitations to the advocacy tools deployed? All EECSP advocacy partners 30. Describe the challenges faced by your group in your campaigns? All EECSP advocacy partners 31. What strategies did you use to mitigate or address these challenges? All EECSP advocacy partners Evaluation Question 4 To what extent can the developed institutional management capacity of partners be sustained beyond the life of the activity? a. How self-sustainable are the efforts undertaken by the CSOs without donor assistance? Questions Outcome(s) Claimed Stakeholder(s)/Respondents 32. How do you intend to sustain your organization’s current institutional capacity? All EECSP partners 33. Describe how you leverage other resources to execute your strategies or activities? 78 Increased IDP voter participation in 2019 in at least one of two states of focus. Increased women’s participation in elections and politics (Stop-VAWIP campaign; VAPP Act campaign; and Sokoto women’s campaign) C2: FGD QUESTIONNAIRE EECSP Evaluation Focus Group Guiding Questions Evaluation question 1: To what extent have the election observation mechanism (PVT and PREO) adopted by the activity influenced impartial observation, accurate reporting and recording of vote totals (April 1, 2014 to date)? a. What factors led to the election observation outcomes and what were the key impediments to achieving the outcomes? Question(s) Sample Group Outcome Claimed/Campaign 1 Let’s talk about the usefulness of PVT and PREO to impartial observation, accurate reporting and recording of vote totals. Were these two mechanisms effective? 1. Abuja - YIAGA Africa - NDI Outcome 1: PVTs served to increase credibility of election results PVTs/PREO - National: 2015 & 2019 Off cycle Outcome 3: PVT recommendations used to inform recommendations for electoral law reform Outcome 4: Changes to INEC procedures due to PREO and PVT findings Outcome 5: PREO and PVT by TMG provided initial data that led to subsequent Stop-VAWIP campaigns by CSOs. 2 Describe your training model for the PVT and PREO with CSOs? Any lessons? YIAGA Africa and NDI b. What is the perception of stakeholders, particularly civil society organizations (CSO) regarding the utility of the PVT? What can be done differently? 3 What do you know about the PVT methodology? YIAGA Africa All groups 4 Has the PVT been useful to your group? If Yes, in what ways? If No, How can PVTs be more relevant in Nigeria? IPAC NDI and YIAGA Africa 79 Evaluation question 2: To what extent have NDI partners management skills (objective 2) influenced their capacity to implement the PVT and campaigns towards increased marginalized groups participation in Nigerian political processes (April 1, 2014 to date)? a. What factors led to the key outcomes achieved in improving management within partner CSOs and what were the key impediments to achieving the outcomes? Questions Group Outcome Claimed/Campaign 5 Describe how your organization’s management capacity has improved? EECSP grantees 6 What support and training did you receive through the EECSP activity? E.g. new skills, improved financial management systems, organizational capacity, procurement, HR, etc. EECSP grantees - YIAGA Africa - YDI - WACOL - IFA - LHI - CCD - GAT - Face initiative 7 How has your group applied the training and support received through EECSP? E.g. new skills, improved financial management systems, organizational capacity, procurement, HR, etc. EECSP grantees b. Are partners (particularly the PVT sub-grantee) implementing the new skills or recommendations from NDI’s capacity assessment? 8 In your opinion, do you think through the support received, your group’s participation in the political process increased? If yes, in what ways? If not, explain? EECSP grantees 9 What measures are in place to ensure that your group sustains any new and improved practices and systems. EECSP grantees 10 What do you think would challenge your group’s ability to sustain change E.g. new skills, improved financial management systems, organizational capacity, procurement, HR, etc. EECSP grantees 80 Evaluation question 3: To what extent have marginalized groups (women, youth, persons with disabilities and internally displaced persons) campaigns affected their participation in Nigerian political processes (April 1, 2014 to date)? Questions Group Outcome Claimed/Campaign 11 What campaign(s) did your group initiate or support under the EECSP activity? Was it successful? What factors contributed to its success? All groups Outcome 11: Deepened capacity of local CSOs to conduct and sustain policy advocacy Outcome 6: Women successfully recruited and deployed as 30% of total PVT observers. 12 If no, what were the challenges encountered and what were the lessons? All groups 13 What are the other barriers to marginalized groups participation in the political process in Nigeria? EECSP grantees and NDI IPAC Evaluation question 4: To what extent can the developed institutional capacity of partners be sustained beyond the life of the activity? Questions Groups Outcome Claimed/Campaign 14 Have your group’s advocacy capabilities improved? If yes, in what ways? If no, explain? EECSP grantees 15 How do you plan to sustain improvements to your advocacy capabilities, if any? Do you have any ongoing advocacy campaigns? Share your experience please? EECSP grantees 16 Has your group applied any lessons or knowledge from EECSP advocacy campaign to new advocacy activities? EECSP grantees Thank you for your time and honest responses. Is there anyone else you recommend we speak with? Please feel free to email or call me if you would like to share more insights. 81 C.3: SURVEY QUESTIONAIRE FOR CSO PARTNERS/TRAINING BENEFICIARIES Part A 1 Which of the following categories represents your group? Tick all that apply  Women  Youth  PWD  IDP  Other 2 Which of the following groups do you belong? __________ 3 What advocacy efforts or initiative did your group lead or support through the EECSP project? Tick all that apply  Stop-VAWIP  Access Nigeria: Disability Votes Matter Campaign  VAPP Law Enactment  Not too young to run  Youth policy agenda  Get out the Vote  Vote Not Fight  Hate Speech  Disinformation  PVT  PREO  Others (please specify) Part B: PREO and PVT (only for activity beneficiaries engaged in election observation) Questions Yes No 1 Did you receive PREO training? 2 Did you receive PVT training? 3 What was your role in the PVT process? Open-Ended Questions: 4 Do you think the PVT and PREO increased the credibility of the elections? If so, can you provide 2-3 examples of how it contributed to elections credibility? 5 Do you think a local CSO can deploy an impartial PVT for future elections? Give reasons for your choice answer. 6 Do you think a local CSO has the technical capacity to conduct a PVT without donor assistance? Why or why not? Part C: Institutional Capacity Questions 1 In what areas, did you get support and training? Tick as appropriate  Financial management  HR  Procurement  Governance (board)  Decision making  Advocacy  Communication  Election Observation (PVT & PREO)  Other (list) 82 2 Did your organization adopt new practices as a result of this support? If yes, please state the new practices adopted, and if no, please state why Yes No 3 How do you plan to sustain the new processes/practices/change? 4 Are there any constraints to sustaining change? Yes No Explain: 5 Kindly propose, if any, 2 or 3 innovative ideas that can enhance CSO participation in Nigeria’s electoral and political process? 83 ANNEX D: SOURCES OF INFORMATION/LIST OF RESPONDENTS LIST OF KII RESPONDENTS Note: Table has been deleted from public version of report to protect privacy of respondents. LIST OF FGD RESPONDENTS Note: Table has been deleted from public version of report to protect privacy of respondents. 84 ANNEX E: DISCLOSURE OF ANY CONFLICT OF INTEREST Note: COI Forms have been redacted in public version of report to protect privacy of team members. 85 86 87 88 ANNEX F: SUMMARY INFORMATION ON THE EVALUATION TEAM Reuben R. Hermoso, Ph.D. is experienced in the design and implementation of quantitative monitoring and evaluation (M&E) methods for projects under USAID, the World Bank, the United States Department of State, the International Monetary Fund and other international donor agencies and organizations. He has developed various M&E components such as framework construction, indicator development, survey design and administration, sampling, data collection and analysis, and evaluation methods design. He has technical qualifications in conducting more rigorous experimental and quasi‐experimental impact evaluation methods such as random assignment, double difference, regression discontinuity, and matching. He has done M&E work for projects in microfinance and rural credit, entrepreneurship and small business development, food and agriculture, health, youth and education, peace, democracy, human rights, and other sectors. Mr. Hermoso is a US citizen and has Secret Level clearance. Adaozo Umenwaliri brings over 15 years of international development experience working on economic and social reform policies and governance projects. Her areas of expertise include policy analysis and strategy development as well as project planning, implementation, and evaluation. As Senior Portfolio Manager of Private Sector Development for the DFID-funded ENABLE1 and 2 Projects, Ms. Umenwaliri managed a portfolio budget of £11million (phase 1) and £17million (phase 2), developing and managing policy dialogues and advocacy campaigns. In this role, she was responsible for evidence-based advocacy activities leading to 8 economic policy reforms. As National Assembly Senior Program Officer for the National Democratic Institute for International Affairs (NDI), she developed strategies for project and knowledge management of the team’s activities and outcomes in relation to DFID’s logical framework and USAID’s performance monitoring plan. Ms. Umenwaliri currently serves as Associate Director for the University of North Carolina African Studies Center, where she plans, develops, and operates the center’s strategic plan in policy, education, business, promotion of African arts, film and music, and communication. She also works as a Consultant Strategist for Results for Development (R4D), where she supports the development of a health systems strategy intervention for the UNC Nigerian project. Ms. Umenwaliri holds a MA in International Development Policy from Duke University in Durham, North Carolina. Babajide Ojo brings over 20 years of experience as a development consultant, author and public affairs analyst. His areas of expertise lie in development and governance issues, specifically in the areas of research, advocacy, and training. As Program Manager for the International Foundation for Electoral Systems (IFES), he implemented project activities, liaised with IFES partners, provided statistical data for the United States Agency for International Development (USAID) Performance Management Plan, and prepared draft reports of project activities. Mr. Ojo has also served as a consultant for the Department for International Development (DFID) Baseline Assessment on Participation, Representation and Influence of Women in Political Parties. While working as a consultant for the UNDP/DGD Post-Election Activities, he helped conduct an independent technical assessment of the October 2014 local government elections and trained Non-Governmental Organizations (NGO)/ Community Based Organizations (CBO) on how to demand for good governance from the newly elected officials. In this role, he led capacity building workshops for Delta State Independent Electoral Commission (DSIEC), officials on election administration and training of newly elected council officials on good governance. He has also been involved in capacity building of Katsina State Independent Electoral Commission and Kaduna State Electoral Commission Mr. Ojo has been involved in capacity building of critical election stakeholders such as political party executives, senior management staff of the Independent National Electoral Commission, Nigerian journalists, especially political correspondents as well as Civil Society Organizations. He is a consultant to 89 National Institute for Policy and Studies (NIPSS), European Centre for Electoral Support (ECES), Westminster foundation for Democracy (WFD) among others. He currently serves as Executive Director of OJA Development Consult Limited where he provides training, research, and advocacy consultancy services for clients on democracy and good governance services. He holds a MS in Political Science from the University of Ibadan in Ibadan, Nigeria. Mr. Benson Adetona has over 25 years of experience as a social scientist and research practitioner. As Lead Consultant/Trainer to top local government functionaries for the 2006 census, Mr. Adetona liaised with local government directors and state directors of the National Population Commission and drafted the commission’s end report. He served as Co-Researcher for ADB-funded National Planning Commission’s Evaluation of Impact of Poverty Reduction Program. In this role, he gathered and analyzed data from various sites and concluded by drafting and submitting the state report to the commission. As Co-Researcher of the MEMS/USAID Nigeria Evaluation of Legislative Intervention, Mr. Adetona developed field instruments, gathered data, performed data analysis, and wrote reports at various evaluation stages. He currently serves as Coordinator and Cofounder of The New Covenant Baptist Academy, where he oversees the academy’s operations and participates in research and development consultancy. He holds a MS in Population and Manpower Planning from the University of Jos in Jos, Nigeria.