USAID/Zimbabwe CSSP Final Evaluation FINAL REPORT FINAL EVALUATION OF THE ZIMBABWE CIVIL SOCIETY STRENGTHENING PROGRAM (CSSP) This publication was produced at the request of the United States Agency for International Development (USAID). It was prepared independently by Kourtney Pompi on behalf of Social Impact, Inc. USAID/Zimbabwe CSSP Final Evaluation FINAL REPORT FINAL EVALUATION OF THE ZIMBABWE CIVIL SOCIETY STRENGTHENING PROGRAM (CSSP) USAID/Zimbabwe Task Order Number: AID-613-M-16-00002, Contract Number: GS-10F0294V Submitted December 23, 2016 Contact: Social Impact, Inc. 2300 Clarendon Blvd., Suite 1000 Arlington, VA 22201 COVER PHOTO CREDIT PACT World DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. i TABLE OF CONTENTS I. INTRODUCTION................................................................................................................. 1 Context........................................................................................................................................................................... 1 Project Background...................................................................................................................................................... 2 II. EVALUATION PURPOSE AND QUESTIONS ................................................................. 4 Evaluation Purpose ....................................................................................................................................................... 4 Evaluation Questions .................................................................................................................................................. 4 III. EVALUATION DESIGN, METHODOLOGY, AND LIMITATIONS ............................... 5 Overview ........................................................................................................................................................................ 5 Evaluation Design And Methodology....................................................................................................................... 5 Phase One: Planning and Desk Review............................................................................................................... 5 Phase Two: Qualitative and Quantitative Data Collection............................................................................ 6 Phase Three: Data Analysis................................................................................................................................... 7 Limitations and Biases.................................................................................................................................................. 7 IV. FINDINGS AND CONCLUSIONS ..................................................................................... 9 EQ 1: Appropriateness and Effectiveness of Program Design........................................................................... 9 EQ 2: Achievement of Intended Results............................................................................................................... 13 EQ 3: Monitoring and Evaluation............................................................................................................................ 20 V. RECOMMENDATIONS...................................................................................................... 24 Recommendations for USAID/Zimbabwe ............................................................................................................ 24 Recommendations for Implementing Partner...................................................................................................... 24 ANNEXES Annex A: Evaluation Statement of Work .......................................................................................... 26 Annex B: Evaluation Evidence Matrix................................................................................................ 31 Annex C: Data Collection Protocols ................................................................................................ 38 Annex D: Bibliography of Documents Reviewed ............................................................................... 52 Annex B: Key Informants................................................................................................................. 59 Annex F: Mini-Survey Data............................................................................................................... 61 Annex G: Disclosure of Any Conflicts of Interest.............................................................................. 67 USAID/Zimbabwe CSSP Final Evaluation FIGURES Figure 1. Sites Visited by the Evaluation Team......................................................................................................... vii Figure 2. CSSP's Development Hypothesis................................................................................................................. 2 Figure 3. Key Informant Statistics.................................................................................................................................. 5 Figure 4. FGD Participant Statistics............................................................................................................................... 6 Figure 5. CSSP Theory of Change ................................................................................................................................. 9 Figure 6. Pact’s Intermediate Results for CSSP ........................................................................................................ 10 Figure 7. USAID Gender Equality and Female Empowerment Policy (2012).................................................... 14 Figure 8. Case Study on Women’s Participation in CSSP...................................................................................... 15 Figure 9. Grantees’ perceptions of inclusion of women in CSSP-funded activities ........................................ 18 Figure 10. Grantees’ perceptions of CSSP support and effect on sustainability............................................... 19 Figure 11. Use of M&E data by CSSP grantees......................................................................................................... 21 Figure 12. Specific Uses of M&E data by CSSP grantees........................................................................................ 21 Figure 13. Sample of Expected Results, Outcome Indicators, and Output Indicators.................................... 23 i ACRONYMS CBO Community-Based Organization CDCS Country Development Cooperation Strategy CEDAW Convention on the Elimination of All Forms of Discrimination Against Women CSO Civil Society Organization CSSP Civil Society Strengthening Program DO Development Objective DRG Democracy, Human Rights, and Governance EQ Evaluation Question ET Evaluation Team FGD Focus Group Discussion GNU Government of National Unity GPA Global Political Agreement GUC Grants Under Contract IR Intermediate Result IS Institutional Support ISP Institutional Support Plan KII Key Informant Interview M&E Monitoring and Evaluation MDC Movement for Democratic Change OCA Organizational Capacity Assessment ONA Organizational Network Analysis OPI Organizational Performance Index PAD Project Appraisal Document PEA Political Economy Analysis RFA Request for Application SPSS Statistical Package for Social Sciences USAID United States Agency for International Development ZANU PF Zimbabwe African National Union USAID/Zimbabwe CSSP Final Evaluation i EXECUTIVE SUMMARY I. INTRODUCTION The political, social, and economic environment in Zimbabwe has been tenuous, at best, for more than 35 years. Zimbabwe’s political environment is often described as restrictive, controlling, and violent. Since independence in 1980, election cycles have been marred with disputes over the results, as well as political violence and voter intimidation. The highly contested 2008 elections continued the trend of rampant political violence, with reports of hundreds of political activists being murdered. Economic collapse, including a widespread cholera epidemic, deepened the crisis. Seeking legitimacy after the internationally condemned election, the ruling Zimbabwe African National Union – Patriotic Front (ZANU PF) party conceded and reached a political settlement with the opposition Movement for Democratic Change (MDC), signing the inter-party Global Political Agreement (GPA), which established a government of national unity (GNU). Since June 2012, the USAID/Zimbabwe Civil Society Strengthening Program (CSSP) has sought to strengthen the ability of civil society organizations (CSOs) and community-based organizations (CBOs) to articulate citizens’ demands and advocate for democratic governance. Implemented by Pact, CSSP is funded at a total estimated cost of USD $26,797,019. The project provides institutional and technical support via sub-grants, rapid response grants, and institutional capacity building to CSOs and CBOs, as well as analytical support to United States Agency for International Development (USAID)’s democracy, human rights, and governance (DRG) portfolio. The CSSP project originally focused its activities on voter registration and mobilization in the run up to the 2013 elections. Zimbabwe’s July 2013 elections, however, saw the re-establishment of ZANU PF into power and the subsequent passing of a new constitution in August that same year. The government remains slow to implement the constitution and realign the new laws. ZANU PF has limited citizen access to resources, selectively regulated the economy, and made political appointments based on party patronage.1 Since 2013, ZANU PF led the country into further economic despair. In addition, the reform agenda necessary to implement the new constitution has been slow. Meanwhile, citizens’ expectations for the government to defend human rights and advance development have increased, thereby increasing pressure on the government to deliver. The opening of political space following the 2013 elections, including the passage of a new constitution and bill of rights, allowed Pact’s CSSP project to begin to link democratic practices and processes to issues that were affecting the everyday lives of Zimbabwean citizens, including water, sanitation, food, shelter, health, and education. With a re-focus on raising awareness about the content of the new constitution and bill of rights, as well as promoting social accountability activities, the CSSP project sought to capitalize on this slight opening by supporting local CSOs and CBOs trough rapid response and institutional support (IS) grants that targeted women, youth, and other community members to begin to hold their government officials to account. II. EVALUATION PURPOSE AND QUESTIONS USAID requested a final performance evaluation of CSSP to assess programmatic achievements, factors that impeded or augmented success, and positive or negative unintended consequences. The primary audience of the evaluation is USAID/Zimbabwe’s DRG Office, which seeks to document lessons learned 1 Pact (2016). Rethinking Democracy, Rights, and Governance Programming in Zimbabwe: An Integrated Approach. USAID/Zimbabwe CSSP Final Evaluation ii that could inform the Mission’s design of future civil society and local capacity development programs. Specifically, the DRG Office plans to immediately apply evaluation findings, conclusions, and recommendations to its design of a follow-on activity, which is currently underway. Conducted by Social Impact, Inc. (SI), the evaluation focused on the entire grant period, including activities supported before the 2013 election, as well as activities supported in the post-election period. This evaluation assessed the appropriateness and effectiveness of the CSSP program design, achievement of intended results, and effectiveness of the project’s monitoring and evaluation (M&E) system. The evaluation also examined how gender was incorporated into program design and implementation. Per USAID’s request, the Evaluation Team (ET) focused on the following districts: Harare, Mutare, Masvingo, Bulawayo and Bindura (see Figure 1). Evaluation Questions (EQs) The EQs and sub-questions below address key issues of program design, implementation effectiveness, impact, sustainability, and lessons learned/best practices that are relevant for the Mission’s design of future programming. In answering these questions, the ET assessed the performance of both USAID and Pact. 1. Appropriateness and effectiveness of program design a) To what extent has the CSSP model with its mix of components (local capacity development and both long- and short-term (rapid response) sub-grants to local organizations on the one hand and analytical support to USAID on the other), and the way it was implemented actually aided or detracted from achieving project outcomes? b) What could be modified to improve effectiveness? 2. Achievement of intended results a) To what extent did the CSSP achieve its intended results? What was successful and not successful? Why? b) How effective was CSSP at increasing youth and women’s participation in target communities? c) To what extent did the CSSP intervention contribute to increasing sustainable institutional and technical capacity of target local CSOs? In what areas do they still need support? d) What recommendations to improve methods or approaches can be made? 3. Monitoring & Evaluation a) How effective was the program's monitoring and evaluation system in providing quality data for decision making and reporting? b) What challenges and successes were realized? III. EVALUATION DESIGN, METHODOLOGY, AND LIMITATIONS The ET used a mixed methods approach that included document review, key informant interviews (KIIs), focus group discussions (FGDs) with project participants, and a mini-survey with grantees in order to provide both quantitative and qualitative data for analysis. The ET used a case study approach to assess the role of CSSP in increasing participation of women and youth in DRG activities. See Figure 8. Case Study on Women’s Participation for findings from the team’s in-depth interview with a woman-led CSO to better understand its experiences, approach, and challenges. The ET reviewed more than 170 project documents, conducted 29 KIIs and 14 FGDs, and surveyed 38 recipients of CSSP institutional support and rapid response grants. In total, the ET met with 218 key informants. Fieldwork occurred during a 2.5 week period and included travel to Mutare, Masvingo, Bulawayo, Bindura, and Domboshava. See Figure 1 for the map of sites the ET visited. USAID/Zimbabwe CSSP Final Evaluation iii Limitations The ET faced some limitations related to bias, data sources, and availability: • Selection Bias: For this evaluation, the list of respondent groups was provided by USAID. The ET received a complete list of CSSP grantees from Pact during the first week of data collection. To help mitigate selection bias, the ET took a random sample of all grantees. However, USAID informed the ET that two groups randomly selected could not be included in the evaluation due to sensitivity and confidentiality concerns. • Contact with CSSP Grantees: Some of the contact details provided by Pact for the grantees were incorrect, which negatively impacted the team’s attempts to schedule KIIs and FGDs with CSSP beneficiaries. In some cases, grantees never responded to requests for interviews. Some leaders of CSSP partner organizations were out of the country, and they did not provide an alternative contact for the ET to interview. • Mini-Survey Administration: Some prospective respondents had reservations about data being collected over the phone, citing security concerns. Alternatively, when the mini-survey was administered via e-mail, some responses were incomplete. Additionally, there was some poor network connection in some areas and this made administering the mini-survey difficult. • Access to M&E Data: The ET did not have unrestricted access to the DevResults M&E system that Pact created to maintain all project monitoring data. Pact provided some M&E data to the evaluation team, but it was limited. The ET is unsure why access was not granted; perhaps it was due to security and safety concerns about data on partner activities beyond CSSP, which is housed in the system. This limited the team’s ability to verify achievements against intended results. Despite these limitations, the ET is confident in the quality of the data, analysis, and findings. The ET addressed the threat of bias by using multiple sources of data to triangulate data for each evaluation question. By combining information found in documents and interviews with multiple sources, any one piece of biased data did not skew the analysis. Feedback was consistent across all respondent groups; there was no feedback or responses that constituted outliers from the other information collected. IV. FINDINGS AND CONCLUSIONS Based on the evidence collected, the ET concluded that CSSP had positive outcomes and met its stated objectives. CSSP partners overwhelmingly stated that Pact’s support helped them become stronger, more effective organizations and better able to accomplish their goals. The ET notes, however, that the incongruity between USAID’s DO and the CSSP results framework (as well as corresponding indicators) makes it difficult to draw direct linkages between CSSP interventions (capacity building and financial support) and DRG outcomes that contribute to the DO. EQ 1: APPROPRIATENESS AND EFFECTIVENESS OF PROGRAM DESIGN Sub-Question 1a) - To what extent has the CSSP model with its mix of components (local capacity development and both long- and short-term (rapid response) sub-grants to local organizations on the one hand and analytical support to USAID on the other), and the way it was implemented actually aided or detracted from achieving project outcomes? FINDINGS 1. The ET identified a missing link between USAID’s DO and Pact’s Intermediate Results. 2. Grantees provided nearly unanimous feedback that Pact’s mixed model approach (financial and capacity building support) helped them achieve their desired outcomes. 3. USAID reported that Pact effectively utilized analytical products to continually refine the project (IR 3). USAID/Zimbabwe CSSP Final Evaluation iv CONCLUSIONS 1. Pact’s approach was effective in reaching the CSSP project goal of contributing to USAID’s DO of protecting and promoting democratic space. Sub-Question 1b) - What could be modified to improve effectiveness? FINDINGS 1. Both Pact and CSSP grantees indicated that the length of institutional support grants was insufficient. 2. Several contextual factors affected project outcomes. 3. Grantees reported that solely focusing on DRG-related activities hindered their ability to incorporate livelihoods and other community issues into their work. 4. CSSP grantees desired a more local presence from Pact. CONCLUSIONS 1. Modifications to the Institutional Support grant cycle could increase organizations’ effectiveness in achieving their desired outcomes. 2. CSSP could have further advanced USAID’s DO if CSSP grantee activities incorporated livelihoods and other community issues. EQ 2: ACHIEVEMENT OF INTENDED RESULTS Sub-Question 2a) - To what extent did the CSSP achieve its intended results? What was successful and not successful? Why? FINDINGS 1. CSSP issued more than 180 grants to CSOs and CBOs across 40 districts. 2. CSSP grantees reported that CSSP-supported activities helped them 1) occupy existing democratic space or 2) expand democratic space in their communities. 3. USAID reported some use of Pact’s analytical products. CONCLUSIONS 1. Pact achieved its intended results. Sub-Question 2b) - How effective was CSSP at increasing youth and women’s participation in target communities? FINDINGS 1. CSSP took a mainstreaming approach to women’s participation, including for organizations that focused on women’s issues. 2. CSSP grantees equate “gender inclusion” with “women’s inclusion.” 3. Despite a number of challenges, CSSP grantees reported a modest ability to increase youth participation. 4. CSSP staff worked with local partners to identify solutions aimed at increasing participation of women and youth. USAID/Zimbabwe CSSP Final Evaluation v CONCLUSIONS 1. Civic participation of women and youth expanded under CSSP. 2. Barriers to youth and women’s participation could be addressed with additional project support. Sub-Question 2c) - To what extent did the CSSP intervention contribute to increasing sustainable institutional and technical capacity of target local CSOs? In what areas do they still need support? FINDINGS 1. CSSP grantees reported being more professional, sustainable organizations as a result of Pact’s capacity building support. 2. The ET lacks sufficient evidence to assess the extent to which Pact increased the technical capacity of CSSP grantees. 3. Institutional Support grantees reported that they continue to utilize new skills and knowledge gained through their CSSP grants, even once financial support from Pact ended. 4. CSSP grantees reported a “knowledge gap” and a desire for a higher level of technical support. CONCLUSIONS 1. CSSP grantees are more sustainable because of the support received from Pact. EQ 3: MONITORING & EVALUATION Sub-Question 3a) - How effective was the program's monitoring and evaluation system in providing quality data for decision making and reporting? FINDINGS 1. Grantees expressed overwhelming appreciation for and use of Pact’s M&E training and guidance on recording meaningful monitoring data. 2. The ET was unable to verify CSSP M&E data, including sample grantee data, because the ET was not given access to the DevResults system. CONCLUSIONS 1. Pact’s M&E training and procedures contributed to improved reporting, especially in regards to grantee outcomes. Sub-Question 3b) - What challenges and successes were realized? FINDINGS 1. CSSP M&E data measured apples and oranges: outcomes were focused on organizational development, while indicators measured outputs related to DRG activities. CONCLUSIONS 1. The incongruity of CSSP M&E data makes it difficult to measure Pact’s overall effectiveness in contributing to USAID’s DO and DRG outcomes. USAID/Zimbabwe CSSP Final Evaluation vi V. RECOMMENDATIONS The ET offers the following recommendations based on its findings and conclusions: RECOMMENDATIONS FOR USAID/ZIMBABWE 1. Continue to support DRG-related activities in Zimbabwe. This is especially pertinent in the run up to the 2018 elections, when many CSSP grantees fear that democratic space will close if there is no support for them to maintain the gains they have made in the last several years. 2. Develop support mechanisms that link DRG activities to other issues affecting Zimbabweans’ everyday lives—including livelihoods and economic empowerment. Citizens do not separate these concepts; neither should USAID support. 3. Continue to support implementing partners to conduct frequent analytical research to directly inform program implementation. 4. Develop flexible funding mechanisms that support CSOs/CBOs through long-term grants (2-3 years), as well as rapid response funding that is shorter and more flexible in nature. Additionally, identify mechanisms to continue to support grantees after a grant cycle ends. 5. Design projects that are cross sectoral that include both issues of community concern, such as livelihoods, food security, education or health, with DRG related skill sets, such as advocacy, social accountability and community mobilization. As these issues are not separate for citizens, new project designs should look for ways to meet these linked needs. 6. Support projects that address existing barriers to participation of youth in civic life. Youth need to be engaged in activities that address their economic realities and motivate them to become active citizens. 7. Support projects that promote activities that specifically target the inclusion of young women. Women need to continue receiving mentoring and coaching so that they may be confident in assuming leadership roles and expressing their opinions. 8. As the political and social context allows, support more in-depth analysis of gender roles as they relate to DRG programming. For example, discussion should include the importance of sensitizing men to the benefits of including women in decision-making processes. 9. Consider utilizing Pact as a sector-wide capacity builder for M&E with local partners. 10. Determine whether DevResults meets USAID’s needs, and explore whether it is useful to maintain the system for existing and future programming. 11. For all projects, ensure that intended results, expected outcomes, and indicators are measuring the same thing and “speak to each other” to ensure that project effectiveness and impact can be measured at regular intervals and at the conclusion of the project. RECOMMENDATIONS FOR IMPLEMENTING PARTNER 12. Develop a systematic, metric-based feedback mechanism to document why local partners did or did not move through RFA rounds of support. Such a mechanism would allow applicants to understand areas they need to improve upon. A system similar to USAID’s own evaluation and scoring criteria could be tailored and used for this purpose. 13. Target and tailor capacity building support to CSOs/CBOs based on individual organizational needs, including more mentoring and in-person coaching. USAID/Zimbabwe CSSP Final Evaluation vii GEOGRAPHIC COVERAGE Figure 1. Sites Visited by the Evaluation Team 1 I. INTRODUCTION CONTEXT The political, social, and economic environment in Zimbabwe has been tenuous, at best, for more than 35 years. Zimbabwe’s political environment is often described as restrictive, controlling, and violent. Since independence in 1980, election cycles have been marred with disputes over the results, as well as political violence and voter intimidation. The highly contested 2008 elections continued the trend of rampant political violence, with reports of hundreds of political activists being murdered. Economic collapse, including a widespread cholera epidemic, deepened the crisis. Seeking legitimacy after the internationally condemned election, the ruling Zimbabwe African National Union – Patriotic Front (ZANU PF) party conceded and reached a political settlement with the opposition Movement for Democratic Change (MDC), signing the inter-party political agreement: the Global Political Agreement (GPA), which established a government of national unity (GNU). Zimbabwe’s July 2013 elections saw the re-establishment of ZANU PF into power and the subsequent passing of a new constitution in August that same year. However, the government has been slow to implement the constitution and realign the new laws. Since 2013, ZANU PF led the country into further economic despair. It limited citizen access to resources, selectively regulated the economy, and made political appointments based on party patronage.2 Freedom House’s annual reports in 2014 and 2015 rank Zimbabwe as “not free” on civil liberties, political rights, and freedom of the press.3 Furthermore, the provision of socioeconomic rights to water, sanitation, food, shelter, health, and education remains limited due to Zimbabwe’s low economic performance, low employment in formal sectors, high rate of youth unemployment, and rampant corruption.4 Civil society remains in a fragile, unfocused, and fragmented state within this political context, and organizations often compete with each other for scarce resources. Meanwhile, citizens’ expectations for the government to defend human rights and advance development have increased, thereby increasing pressure on the government to deliver. Zimbabwe has made commitments toward the promotion of gender equality and women’s empowerment. The government has ratified and signed a number of international and regional conventions and declarations on gender, including the 1979 Convention on the Elimination of All Forms of Discrimination Against Women (CEDAW), the 1995 Beijing Declaration and Platform for Action, the 1997 SADC Declaration on Gender and Development, the 2000 Millennium Declaration and Development Goals, and the 2008 SADC Protocol on Gender and Development. Domestically, the 2013 constitution includes strong provisions on gender equality and established an independent Gender Commission. A number of legislative reforms and policy frameworks have been introduced as part of the national drive toward women’s empowerment and gender equality, including a National Gender Policy in 2013. Most of these gains remain on paper, however, with little in terms of implementation or dedicated resources to make them a reality. The women of Zimbabwe therefore have a steep hill to climb. A traditionally patriarchal society, gender-based violence, and discrimination have marginalized women. Traditionally, women and youth are limited in their participation in political activities and decision-making processes due to their marginalization, lack of knowledge about the constitution, and limited economic empowerment.5 2 Pact (2016). Rethinking Democracy, Rights, and Governance Programming in Zimbabwe: An Integrated Approach. 3 Freedom House. Freedom in the World 2014; and Freedom House. Freedom in the World 2015. 4 Zimbabwe Democracy Institute (2015). ZIMBABWE COUNTRY REPORT, Governance, Politics and the Shifting Political Economy in Zimbabwe. Harare, Zimbabwe. 5 International Republican Institute (2015). Survey on Local Governance & Constitutionalism: Zimbabwe. USAID/Zimbabwe CSSP Final Evaluation 2 PROJECT BACKGROUND Since June 2012, the USAID/Zimbabwe Civil Society Strengthening Program (CSSP) has sought to strengthen the ability of civil society organizations (CSOs)/community-based organizations (CBOs) to articulate citizens’ demands and advocate for democratic governance. Implemented by Pact, CSSP is funded at a total estimated cost of USD $26,797,019. The project provides institutional and technical support via sub-grants, rapid response grants, and institutional capacity building to CSOs/CBOs, as well as analytical support to United States Agency for International Development (USAID)’s democracy, human rights, and governance (DRG) portfolio. Pact’s CSSP project has four components to which all project activities contribute. CSSP’s stated program components are as follows: 1. Institutional and Technical Strengthening of Selected Local CSOs 2. Rapid Response Grants to Traditional and Non-Traditional Civil Society Actors 3. Institutional Capacity Building of Small or CBOs 4. Analytical Support to USAID for the DRG Portfolio Although the project description submitted by Pact does not explicitly state a development hypothesis, the technical proposal’s executive summary6 gives some indication of how Pact formulated the CSSP theory of change: “Pact’s integrated approach to capacity development, grant administration and monitoring and evaluation (M&E) will empower civil society to usher in a new era of accountability, good governance, democracy, gender equality and respect for human rights in Zimbabwe.” Pact articulated a refined development hypothesis in the FY 2013 implementation plan: “[…] representative, effective, and cohesive civil society will enable a fairer electoral process and the transition to more accountable and democratic system of governance.”7 The Evaluation Team (ET) understands the CSSP development hypothesis as follows: The ET notes that USAID/Zimbabwe’s results framework changed in September 2013 during CSSP project implementation. The original DO to which CSSP contributed was: “transition to a democratic system of governance enhanced.” Following the 2013 elections, USAID revised the DO to which CSSP contributed to: “protect and promote democratic space.” The ET notes that the 2013 elections catalyzed CSSP to adapt its approach to the political context; the project shifted its emphasis from election-related activities such as voter mobilization and registration to civic education (focusing on the constitution and bill of rights) and social accountability issues. The ET did not find any evidence to suggest that this shift in the DO and prioritized activities had any negative effect on CSSP; in fact, this shift took advantage of openings to occupy democratic space, at a minimum—and seized the opportunity to expand that space, in some cases. 6 Pact (2012). Civil Society Strengthening Project in Zimbabwe Technical Proposal. 7 Pact (2013). Civil Society Strengthening Program (CSSP) Annual Implementation Plan FY 2013 USAID's DO to 'Protect and Promote Democratic Space' will be achieved THEN CSOs and CBOs strengthen their organizational and technical capacity IF Figure 2. CSSP's Development Hypothesis USAID/Zimbabwe CSSP Final Evaluation 3 While more in-depth analysis is provided in the findings and conclusions section below, it is important to note that the CSSP results framework and M&E plan were incongruous. While all project outcomes were focused on organizational improvement for partner CSOs, related indicators focused on DRG-related outputs (e.g., number of voter clubs engaging youth, number of recommendations taken up by local authorities). The programmatic shift presented an opportunity for USAID and Pact to revisit the CSSP results framework and M&E plan to ensure that—via monitoring achievements against appropriate indicators—Pact’s interventions were contributing to their desired outcomes. However, because of the sustained incongruity between project outcomes and indicators, it is difficult to determine to what extent USAID’s DRG DO was achieved through CSSP. 4 II. EVALUATION PURPOSE AND QUESTIONS EVALUATION PURPOSE Conducted by Social Impact, Inc. (SI), this final performance evaluation focused on pre- and post-election activities of the CSSP project. USAID requested an evaluation to assess CSSP’s programmatic achievements, factors that impeded or augmented success, and positive or negative unintended consequences. The primary audience of the evaluation is USAID/Zimbabwe’s DRG Office, which seeks to document lessons learned that could inform the Mission’s design of future civil society and local capacity development programs. Specifically, the DRG Office plans to immediately apply evaluation findings, conclusions, and recommendations to its design of a follow-on activity, which is currently underway. The purpose of this evaluation is to: 1. Assess performance, including programmatic achievements, factors impeding or augmenting success (including the project design), and positive or negative unintended consequences; and 2. Document lessons learned from CSSP to inform potential future projects. The evaluation focused on the appropriateness and effectiveness of CSSP’s program design, achievement of intended results, and effectiveness of the project’s M&E system. The evaluation also examined how gender was incorporated into program design and implementation. EVALUATION QUESTIONS (EQS) The EQs and sub-questions below address key issues of program design, implementation effectiveness, impact, sustainability, and lessons learned/best practices that are relevant for the design of future programming. In answering these questions, the ET assessed the performance of both USAID and Pact. 1. Appropriateness and effectiveness of program design a) To what extent has the CSSP model with its mix of components (local capacity development and both long- and short-term (rapid response) sub-grants to local organizations on the one hand and analytical support to USAID on the other), and the way it was implemented actually aided or detracted from achieving project outcomes? b) What could be modified to improve effectiveness? 2. Achievement of intended results a) To what extent did the CSSP achieve its intended results? What was successful and not successful? Why? b) How effective was CSSP at increasing youth and women’s participation in target communities? c) To what extent did the CSSP intervention contribute to increasing sustainable institutional and technical capacity of target local CSOs? In what areas do they still need support? d) What recommendations to improve methods or approaches can be made? 3. Monitoring & Evaluation a) How effective was the program's monitoring and evaluation system in providing quality data for decision making and reporting? b) What challenges and successes were realized? 5 III. EVALUATION DESIGN, METHODOLOGY, AND LIMITATIONS OVERVIEW SI conducted this final performance evaluation using a four-person team: Team Leader Kourtney Pompi; DRG Specialist Jack Zaba; M&E Specialist Cleophas Tavaya; and Research and Logistics Specialist Stephen Ndoma. Team members contributed expertise in local political knowledge, implementation of DRG projects, M&E, survey research, data analysis, and information management. The ET used a mixed methods approach that included document review, key informant interviews (KIIs), focus group discussions (FGDs) with project participants, and a mini-survey with grantees in order to provide both quantitative and qualitative data for analysis. The ET reviewed more than 170 project documents, conducted 29 KIIs and 14 FGDs, and surveyed 38 recipients of CSSP institutional support and rapid response grants. In total, the ET met with 218 key informants. The ET used a case study approach to assess the role of CSSP in increasing participation of women and youth in DRG activities. See Figure 8. Case Study on Women’s Participation for findings from the team’s in-depth interview with a woman-led CSO in Bindura to better understand its experiences, approach, and challenges. The Evaluation Evidence Matrix in Annex B details the data sources and key informant categories that informed the evaluation team’s answers to each of the evaluation questions. The Matrix references Program Indicators, which include a compilation of indicators listed in the CSSP quarterly reports provided to the evaluation team. EVALUATION DESIGN AND METHODOLOGY Phase One: Planning and Desk Review The ET conducted an in-depth desk review of project documents supplied by both USAID and Pact. The review included the following documents (see Annex D for an exhaustive list of documents reviewed by the evaluation team): • Pact’s CSSP Proposal • Annual Implementation Plans • Annual and Quarterly Reports and Indicator Data • Analytical products including PactBriefs, research papers, environmental assessments, and lessons learned documents • Annual RFAs for CSSP grant cycles • Compendium of CSSP Programming Lessons 2012-2016 • 2012 DRG Assessment • USAID Transition Country Development Cooperation Strategy • A random sample of grantee project documents (proposals and reports) The ET notes that, while Pact was willing to supply requested project documents, the speed and volume was problematic. Project documents such as quarterly and annual reports were readily provided. However, grantee documents were slower in being made available (partially due to the volume and sensitivity of documents, which made e-mailing them difficult during the ET’s field travel). Given these challenges, the ET relied on primary sources for analysis, and referenced project documents for clarification or triangulation as necessary. Figure 3. Key Informant Statistics 109 Women (33 youth) 109 Men (28 youth) 218 Key Informants USAID/Zimbabwe CSSP Final Evaluation 6 Phase Two: Qualitative and Quantitative Data Collection Phase Two took place in November 2016 when the Team Leader arrived in Zimbabwe. The ET held an in-brief with USAID/Zimbabwe to clarify expectations and discuss future utilization of evaluation results to ensure that the evaluation was responsive to the Mission’s needs. Following the in-brief, the ET finalized the work plan and all data collection instruments. Fieldwork occurred during a 2.5 week period and included travel to Mutare, Masvingo, Bulawayo, Bindura, and Domboshava. See Figure 1 for the map of sites the ET visited. Key Informant Interviews (KIIs) The team conducted KIIs to triangulate the data collected in the desk review and gain further insights into perceptions about the project’s effectiveness. The ET conducted 29 KIIs with 44 individuals (27 male, 17 female) representing the following individuals and organizations (see Annex E for an exhaustive list): • Current and former USAID/Zimbabwe staff • Current Pact staff • CSO and CBO Sub-grantees • Reference Group Members KII protocols were tailored to the key informant’s role and “causal distance” from CSSP activities, as well as the extent of the key informant’s involvement in CSSP activities and the time available for interviewing. The questions not only addressed knowledge and general perceptions, but also probed for specific examples of attitude and behavior change. KII protocols are included in Annex C. Focus Group Discussions (FGDs) The ET conducted FGDs with direct CSSP beneficiaries, as well as with indirect beneficiaries. A total of 14 FGDs were held with 137 direct and indirect beneficiaries (59 male, 78 female), including 57 youth.8 The ET held 3 FGDs per site—separately with women, men, and youth—in Mutare, Masvingo, Bulawayo, Bindura, and 2 FGDs in Domboshava/Harare (see Figure 4). Each FGD protocol differed slightly depending on the group’s role and “causal distance” from CSSP activities, as well as the extent of the group’s involvement in CSSP activities and the time available for the FGD. As with the evaluation team’s KIIs, questions assessed knowledge, general perceptions, and specific examples of attitude and behavior change. FGD protocols are included in Annex C. 8 The ET intended to hold a total of 18 FGDs. The ET could not schedule 3 planned FGDs in Harare as a CSSP￾funded CSO had difficulty obtaining approval/clearance from local authorities. In addition, a CSSP-funded Domboshava/Harare CSO was only able to gather enough participants for 2 FGDs (as the ET set a FGD threshold of at least 5 participants per FGD). 8 adults 5 youth 8 adults 5 youth 13 adults* 5 youth 7 adults* 7 youth 9 adults 4 youth 8 adults 12 youth 1 local leader* 6 adults 15 youth 4 local leaders* 8 elders* 4 youth 4 elders* 5 youth Mutare Masvingo Bulawayo Bindura Harare/ Domboshava Figure 4. FGD Participant Statistics * indirect beneficiary USAID/Zimbabwe CSSP Final Evaluation 7 Mini-Survey Finally, the ET conducted a mini-survey with CSSP grant recipients. Drawing from a sample of 60 CSSP grantees (30 rapid response fund recipients, and 30 institutional support grant recipients), the team achieved a sample size of 38 respondents (63 percent male, 27 percent female). Of the respondents, 61 percent were recipients of institutional support while 39 percent received funding under the rapid response mechanism. Most of the mini-survey questions were close-ended to provide the ET with quantifiable data; the protocol is included as Annex C. The Research and Logistics Specialist administered the mini-survey via phone, e-mail, and in person, depending on the preference of the respondent. For complete data analysis from the mini-survey, see Annex F. Phase Three: Data Analysis The ET generated both “horizontal” analysis (across main themes to create the big picture) and “vertical” analysis (in-depth understanding of the most important issues) using the following approach: • The ET began data analysis by conducting an in-depth discussion in order to identify and conceptualize common trends and themes that emerged from the qualitative data collected. • Once all qualitative data were consolidated according to the evaluation questions, the ET identified the most prevalent themes and concepts mentioned by respondents in order to yield in-depth analysis pertinent to the evaluation questions. • Data collected via the mini-survey was captured using the quantitative Statistical Package for Social Sciences (SPSS). Data was cleaned and frequencies were run to identify the topline data on key questions. • The ET’s Research Assistant provided inputs to how the quantitative data collected from the mini￾survey corroborated or diverged from the identified themes. As detailed in this report, the ET had difficulty in analyzing CSSP M&E data because outcomes and indicators were incongruous. Project outcomes focused solely on the changes within each partner grantee’s organizational development, while all output indicators focused solely on DRG activities. Project data that the ET was privy to made it difficult to draw connections between outcomes and output indicators. Perhaps, had the ET been granted full access to DevResults, more in-depth analysis could have occurred. Specific analysis on DevResults is included below under the findings and conclusions related to Evaluation Question 3. LIMITATIONS There are few, but notable, limitations to the methods used and data collected during this evaluation. Below are details on limitations related to bias, availability of respondents, and gender inclusion, as well as the evaluation team’s mitigation strategies to address each limitation. Bias As is expected in any social research project, there are biases and other limitations that must be addressed through methodological or analytical methods: • Recall bias occurs when informants share inaccurate or incomplete recollections about experiences from the past. This can happen for a variety of reasons including when informants are participants in more than one intervention, e.g., FGD participants responding to team questions with answers related to a different capacity building program. The ET notes that it does not think any data was compromised by recall bias. • Response bias is the risk that key informants may have been motivated to provide the ET with responses that would be considered socially desirable or influential in obtaining donor support. USAID/Zimbabwe CSSP Final Evaluation 8 The team fully expected that key points of contact, trainees, and partner organizations may understand that this evaluation will shape future project opportunities and funding. • Selection bias is an inherent risk when implementers help to facilitate contact with project beneficiaries because they may select the most active, responsive, or engaged candidates— meaning that the ET may only hear from key informants who report positive experiences. For this evaluation, the list of respondent groups was provided by USAID. The ET received a complete list of CSSP grantees from Pact during the first week of data collection. To help mitigate selection bias, the ET took a random sample of all grantees. However, USAID informed the ET that two of the groups randomly selected could not be included in the evaluation due to sensitivity and confidentiality concerns. The ET addressed the threat of bias by using multiple sources of data to triangulate data for each evaluation question. By combining information found in documents and interviews with multiple sources, any one piece of biased data did not skew the analysis. Data Sources and Availability The ET faced a number of limitations related to data sources and availability: a) Contact with CSSP Grantees: Some of the contact details provided by Pact for the grantees were incorrect, which negatively impacted the team’s attempts to schedule KIIs and FGDs with CSSP beneficiaries. In some cases, grantees never responded to requests for interviews. Some leaders of CSSP partner organizations were out of the country, and they did not provide an alternative contact for the ET to interview. b) Mini-Survey Administration: Some prospective respondents had reservations about data being collected over the phone, citing security concerns. Alternatively, when the mini-survey was administered via e-mail, some responses were incomplete. Additionally, there was some poor network connection in some areas and this made administering the mini-survey difficult. c) Requirements for Authority Clearance: A CSSP grantee based in Harare, which the team selected for participation in FGDs, had to seek both police clearance and permission from the local authorities (District Administrator). The clearance took too long to acquire, and the team was not able to include the organization in its data collection. d) Access to M&E Data: The ET did not have unrestricted access to the DevResults M&E system that Pact created to maintain all project monitoring data. Pact provided some M&E data to the evaluation team, but it was limited. The ET is unsure why access was not granted; perhaps it was due to security and safety concerns about data on partner activities beyond CSSP, which is housed in the system. This limited the team’s ability to verify achievements against intended results. Despite these limitations, the ET is confident in the quality of the data, analysis, and findings. Feedback was consistent across all respondent groups; there was no feedback or responses that constituted outliers from the other information collected. 9 IV. FINDINGS AND CONCLUSIONS Based on the evidence collected, the ET concluded that CSSP had positive outcomes and met its stated objectives. CSSP partners overwhelmingly stated that Pact’s support helped them become stronger, more effective organizations and better able to accomplish their goals. The ET notes, however, that the incongruity between USAID’s DO and the CSSP results framework (as well as corresponding indicators) makes it difficult to draw direct linkages between CSSP interventions (capacity building and financial support) and DRG outcomes that contribute to the DO. The missing link between USAID’s DO and Pact’s intermediate results (IRs) did not, however, seem to hinder the ability of CSOs and CBOs to affect the democratic space in Zimbabwe. The ET’s understanding of CSSP is summed up best through Figure 5, a graphical representation of the CSSP theory of change (which differs slightly from Figure 2 in the Introduction). Figure 5 below conceptually maps onto the original program design, shifts made by USAID in its priority DO, and post￾election activity changes. However, it does not draw a direct line between Pact’s interventions (primarily the capacity building support) and the democratic changes USAID was looking for. This does not mean that democratic change did not occur. Unfortunately, the incongruity between organizational development outcomes and DRG–focused indicators makes it impossible to identify exactly which of Pact’s interventions led to the ultimate DRG outcomes grantees reported to the ET. In speaking to former USAID staff, this incongruity did not exist in the original design or solicitation for the project. These issues may have manifested after the contract was awarded; although, the ET also identified these inconsistencies in Pact’s original proposal. This report unpacks this issue further in Evaluation Questions 1 and 3 below. EQ 1: APPROPRIATENESS AND EFFECTIVENESS OF PROGRAM DESIGN Sub-Question 1a) - To what extent has the CSSP model with its mix of components (local capacity development and both long- and short-term (rapid response) sub-grants to local organizations on the one hand and analytical support to USAID on the other), and the way it was implemented actually aided or detracted from achieving project outcomes? FINDINGS Finding 1: The ET identified a missing link between USAID’s DO and Pact’s Intermediate Results. While USAID primarily looked to push the democratic needle forward, Pact concentrated on building the capacity of DRG-oriented CSOs and CBOs. By pursuing achievement of its approved IRs—which focused on capacity building, Pact was only indirectly contributing to the DO (i.e., USAID’s DO was met because grantees were already focused on DRG issues). Additionally, this missing link at the results framework level meant that Pact’s monitoring and reporting—to the extent that it aligned with its approved IRs— Democratic Development Analytics Capacity Building Financial Support Figure 5. CSSP Theory of Change USAID/Zimbabwe CSSP Final Evaluation 10 could not address achievements or challenges associated with the overall USAID DO. USAID acknowledged this issue during the group KII. Finding 2: Grantees provided nearly unanimous feedback that Pact’s mixed model approach (financial and capacity building support) helped them achieve their desired outcomes. Grantees described to the ET how both the financial support and the capacity building were effective. This is described more fully under Evaluation Question 2. The missing link between USAID’s DO and Pact’s IRs did not seem to hinder the ability of CSOs and CBOs to affect the democratic space in Zimbabwe. CSSP grantees reported they felt their CSSP-supported activities helped them occupy existing democratic spaces and/or expand democratic spaces in their communities (IR 2). The majority of respondents confirmed having contributed to a culture of democratic governance. Nearly every grantee the ET met indicated that through CSSP support, communities witnessed: (1) increased participation of citizens in demanding accountability from local leaders, something that never existed before, (2) improved responsiveness and transparency by local leaders, and (3) improved direct engagement between citizens and “solution holders” at various tiers of governance. Some respondents highlighted that they managed to achieve a shift in democratic practices in target communities partly due to their high performance levels as organizations enabled by CSSP support. Mini-survey data supports findings from the KIIs and document review in which grantees described the CSSP as relevant. An overwhelming majority (95 percent) of CSSP grantee respondents reported that CSSP activities were very relevant, while only 3 percent reported that CSSP services or activities were irrelevant in advancing their organizational goals. Finding 3: USAID reported that Pact effectively utilized analytical products to continually refine the project (IR 3). The ET received positive feedback from USAID that Pact utilized analytical tools and products to continuously refine project scope, desired outcomes, and intended results. This was evident in the revised CSSP RFAs for the institutional support grants. During a group KII, Pact staff reported using the environmental scans, political economy analysis, and other analytical tools to inform program decisions and to inform how Pact staff should support CSSP grantees over the life of the project. CSSP’s program scope, as revised across CSSP grant pool RFA cycles, directly links to the political changes and contextual analysis found in Pact’s issue briefs and research reports. Additionally, several CSSP grantees noted that they conducted, with the support of Pact, environmental scans and contextual analysis, which they subsequently used to inform the activities they proposed for CSSP funding. According to Pact, environmental scans were also incorporated into its analytical reports for USAID. There was little hard evidence, however, to indicate exactly how Pact, grantees, or USAID used the analytical resources produced under CSSP. All evidence gathered was anecdotal from KIIs. CONCLUSIONS Conclusion 1: Pact’s approach was effective in reaching the CSSP project goal of contributing to USAID’s DO of protecting and promoting democratic space. Despite the incompatibility between USAID’s DO and CSSP’s results framework, the ET concluded that the project did “protect and promote” the DRG space within Zimbabwe. CSSP-funded activities contributed to occupying existing democratic space and, in some cases, may have expanded the space for additional DRG activities. With CSSP support, CBOs and CSOs opened pathways for engagement between solution holders and citizens on matters of governance affecting their communities. In a country like Zimbabwe, where the political environment is marred with violence and restrictions to fundamental freedoms, the ability for national and subnational groups to move the democratic needle forward (or at a minimum, hold it steady) is significant. Grantees concurred that Figure 6. Pact’s Intermediate Results for CSSP • IR1: Grants to CSOs are targeted and administered effectively • IR2: Targeted CSOs and CSO networks are more effective and sustainable • IR3: Improved information available and analysis for the USAID program USAID/Zimbabwe CSSP Final Evaluation 11 such direct engagement was rare prior to CSSP, as communities were largely polarized and hostile against citizen demands for transparency, accountability, and inclusion in governance decision-making processes. Sub-Question 1b) - What could be modified to improve effectiveness? FINDINGS Finding 1: Both Pact and CSSP grantees indicated that the length of institutional support grants was insufficient. Institutional support grants that the ET reviewed were 8-12 months in length. Many grantees felt that even the longer institutional support grants were not long enough to 1) implement and/or complete all proposed activities and 2) complete institutional support plans (ISPs). In fact, more than 30 percent of the grantees the ET met with reported not completing their ISPs before their CSSP grant ended.9 From Pact’s perspective, many staff expressed a frustration with the Grants Under Contract (GUC) mechanism. They cited a lack of flexibility in the RFA process that resulted in 1) dropping some good partners from year to year due to RFA revisions and competition and 2) not being able to extend grants. Several key informants also mentioned that the grant cycle was problematic (June-May vs fiscal year or annual calendar) and often did not map onto local development budget or legislative calendars. The latter is relevant because local development resource spending was difficult to monitor when an organization’s CSSP grant was not long enough to cover the entire local budget cycle. Finding 2: Several contextual factors affected project outcomes. The ET asked CSSP grantees about the challenges they faced in conducting their activities, in order to learn about any positive or negative aspects of the Pact approach. Grantees and their beneficiaries mentioned a range of political, economic, and social factors that affect their work. Politically, grantees are affected by the receptiveness of the ruling party to DRG-focused activities. In a country where “democracy” is a controversial topic, many partners were challenged by local authorities when conducting social accountability activities. Grantees commonly reported that the introduction of CSSP-supported activities in most communities faced resistance from local solution holders who misconstrued DRG programming as designed to supplant them from power. As such, local leaders made the operating environment more restricted to CSOs and CBOs. Grantees reported using a variety of tactics including concerted engagement with local leaders to demystify the objectives of CSSP; most grantees reported improved relations with such stakeholders. The ET was encouraged to hear from CSSP grantees who conducted participatory budgeting activities, advocated for policy changes that were ultimately taken up by parliamentarians, and increased the number of women on village councils (specifically mentioned in at least two KIIs/FGDs). Several CSSP grantees reported that they would circumvent or “work around” barriers presented by the operating environment. While it is reassuring that CSSP partners could safely identify mechanisms to address community development challenges, the ET finds this strategy somewhat concerning in regards to DRG processes. For example, one CSO reported helping its community build and resource a clinic, something that was in its local development plan and budget. When the local authorities were unwilling to take up the issue, the CSO rallied the community members to gather their own money and building supplies to construct the clinic on their own. While this is certainly a community success to be noted, it does not demonstrate that the CSSP partner knew or employed DRG processes (such that might challenge or change power dynamics so that the local authorities felt compelled to address this need and deliver services to meet citizen demands). 9 Many of these grantees reported that they continued to implement their ISPs even after their CSSP grant ended, utilizing other resources as necessary to complete the ISP. USAID/Zimbabwe CSSP Final Evaluation 12 Finding 3: Grantees reported that solely focusing on DRG-related activities hindered their ability to incorporate livelihoods and other community issues into their work. CSSP grantees, specifically those working on social accountability issues and/or at the local level, repeatedly stated that they had a difficult time focusing beneficiaries and stakeholders on DRG issues and processes. CSSP grantees mentioned that their activities often compete with urgent economic needs, with the most extreme statement from a key informant as follows: “It’s hard to get participants when they are more worried about where their next meal is coming from.” Given the economic crisis in the country, many CSSP grantees stated that they could have had a greater impact in their communities if they could have incorporated livelihoods or other economic empowerment issues into their DRG activities. All CSSP grantees the ET spoke with said that they, and their beneficiaries, did not separate DRG processes and practices (such as awareness raising or advocacy) from contextual issues, such as livelihoods/employment, education, or healthcare. Therefore, many CSSP grantees reported that CSSP’s explicit focus on DRG activities hampered their ability to reach their desired outcomes. The ET notes that both Pact and CSSP grantees lamented that they were not more flexible with grant scope and activity focus, including the possibility of expanding into other geographical locations if the political context allowed. While grantees acknowledged the role played by CSSP support in pushing the democratic needle forward, some felt that more could be done in broadening the reach of the project. Finding 4: CSSP grantees desired a more local presence from Pact. Approximately 20 percent of the CSSP grantees the ET met with described their desire for Pact to have a more “present” role at the local level. When describing their relationship with Pact, every grantee was extremely complimentary. However, several noted that the interaction with Pact was more meaningful for them when Pact was “in the field” working directly with them. They felt that when Pact was present for their activities, Pact understood their challenges better. One partner specifically lamented the closing of Pact’s Bulawayo field office, noting that the support the partner received from the Bulawayo team was more targeted and felt more personal, as compared to the support provided by Pact’s Harare office (or grantees being required to come to Harare for all trainings). CONCLUSIONS Conclusion 1: Modifications to the Institutional Support grant cycle could increase organizations’ effectiveness in achieving their desired outcomes. Based on findings from KIIs, the ET concluded that the Institutional Support grants would have been more effective had they been longer than 8-12 months. In fact, many Institutional Support grants ended with unfinished activities and incomplete institutional support plans, the cornerstone of Pact’s integrated approach for capacity building. The restrictive nature of the GUC mechanism also hindered Pact’s ability to be more flexible with grantees—including extending grants. Conclusion 2: CSSP could have further advanced USAID’s DO if CSSP grantee activities incorporated livelihoods and other community issues. When asked what the project could have done more/less of and for recommendations about future programming, grantees overwhelmingly commented on the need to link DRG activities to other issues such as livelihoods, economic empowerment, or food security. Grantees need to better understand how to use DRG tools and approaches to help citizens address a wide range of issues that matter to them. That would not only address the immediate interests of citizens and communities, but also advance a culture of accountability and engagement. At least 30 percent of KII respondents specifically stated that their organizations had difficulty in recruiting participants for scheduling activities because beneficiaries would prioritize income-generating or food security opportunities over participating in a community dialogue (even if the dialogue could have had an impact on livelihoods down the road). While the ET did not analyze the content of Pact’s specific training modules, their trainings on social accountability and community mobilization may be a start in addressing some of the community concerns. Perhaps the trainings could be more deliberate in working with CSOs and USAID/Zimbabwe CSSP Final Evaluation 13 training participants to better understand and identify the inherent link between DRG activities and these critical issues. By working across sectors, for example, those primarily interested in entrepreneurship could work with DRG-focused groups to identify entry points and barriers to bringing new products or ideas into the marketplace by engaging government entities or advocating for necessary reforms to existing regulations or laws. While this doesn’t address the poverty many are living in, it could be a start in helping beneficiary groups to see the link themselves. A good example of this linking of DRG and entrepreneurship can be found in the grantee that is highlighted in the gender case study below. EQ 2: ACHIEVEMENT OF INTENDED RESULTS Sub-Question 2a) - To what extent did the CSSP achieve its intended results? What was successful and not successful? Why? FINDINGS Finding 1: CSSP issued more than 180 grants to CSOs and CBOs across 40 districts. These grants focused on activities aimed at empowering civil society to usher in a new era of accountability, good governance, democracy, gender equality, and respect for human rights in Zimbabwe. This directly addresses CSSP IR1: “Grants to CSOs are targeted and administered effectively.” The ET received no negative feedback on the grants administration or management process. Finding 2: CSSP grantees reported that CSSP-supported activities helped them 1) occupy existing democratic space or 2) expand democratic space in their communities. This directly addresses CSSP IR 2: “Targeted CSOs and CSO networks are more effective and sustainable.” At the same time, the ET was hampered in gathering further information on project and grantee achievements as the ET did not have unrestricted access to the DevResults system. While the Pact team was very helpful in giving the ET an overview of the system and the type of analytical reports that could be produced from the raw data, the ET never had access to the raw data to draw conclusions about the effectiveness of grantee activities. Finding 3: USAID reported some use of Pact’s analytical products. Anecdotally, USAID reported to the ET that the Mission used the information and analysis provided by Pact to adjust DRG programs and activities, including CSSP. Specifically, USAID relayed that its Country Development Cooperation Strategy (CDCS) for 2015-2020 cites the civil society gender analysis completed by Pact and that its Project Appraisal Document (PAD) concept paper cites the media assessment completed by Pact. The Mission also noted that both the CDCS and PAD concept paper reference the Political Economic Analysis (PEA) completed by Pact. More detailed information could have helped the ET further analyze the achievement of CSSP IR 3: “Improved information available and analysis for the USAID program.” CONCLUSIONS Conclusion 1: Pact achieved its intended results. The project administered more than 180 grants through the rapid response and institutional support mechanisms (IR 1). CSSP-funded activities contributed to occupying existing democratic space and, in some cases, may have expanded the space for additional DRG￾supported activities (IR 2). Every KII informant and nearly all FGD participants could articulate how their work, funded through CSSP, contributed to democratic practices and processes. The ET concluded that the activities of CSSP grantees indeed affected democratic development in Zimbabwe. Pact also worked with partners to utilize M&E best practices (outcome-based programming) and M&E data to refine and improve project activity performance (IR3). Sub-Question 2b) - How effective was CSSP at increasing youth and women’s participation in USAID/Zimbabwe CSSP Final Evaluation 14 target communities? FINDINGS Finding 1: CSSP took a mainstreaming approach to women’s participation, including for organizations that focused on women’s issues. Mainstreaming included internal organizational measures to achieve inclusion as well as grant activities/approaches to promote participation. All CSSP grantees, when asked, said that Pact worked with them to develop a gender policy, as well as think about how they could have better female representation on staff, with trainers/speakers, and with participants. Pact also worked with grantees to ensure that M&E data included a breakdown of women’s representation in grantee activities. Most grantees (84%) concurred that CSSP support enabled them to increase participation of women in democracy and governance broadly, and at community platforms where related issues were discussed. Most CSSP grantees in Bulawayo and Matebeleland South provinces, for example, indicated that participation of women now exceeds that of men, who used to be dominant at most democracy and governance platforms. A CSSP grantee in Bulawayo took pride in sharing that at one of the organization’s flagship demonstrations— which resulted in council rescinding its intention to use pre-paid water meters— women were not only the majority in number, but were also dominant in providing strategic direction and giving speeches during the citizen-driven initiative. Finding 2: CSSP grantees equate “gender inclusion” with “women’s inclusion.” All interviewed CSSP grantees equated “gender” with “women,” which goes against USAID’s definition of gender (Figure 8). When queried on the topic by the ET, grantees stated that CSSP did not include issues that affect men in discussions about gender inclusion. The ET asked CSSP grantees whether male participants were ever included in discussion on women’s participation. Less than 10 percent reported that they were able to involve men, with most citing cultural and social barriers as the reason. The ET specifically asked KII and FGD participants whether CSSP provided training or coaching focused on sensitizing men to women’s participation. Only one CSSP grantee could recall such a discussion but said that the strategy was not culturally palatable, given the barriers to women’s empowerment in Zimbabwean society. This reality may explain Pact’s focus on women’s inclusion versus a gender integration approach. • Gender is the socially defined set of roles, rights, responsibilities, entitlements, and obligations of females and males in societies. • Gender equality concerns women and men, and it involves working with men and boys, women and girls to bring about changes in attitudes, behaviors, roles and responsibilities at home, in the workplace, and in the community. Genuine equality means more than parity in numbers or laws on the books; it means expanding freedoms and improving overall quality of life so that equality is achieved without sacrificing gains for males or females. • Female empowerment is achieved when women and girls acquire the power to act freely, exercise their rights, and fulfill their potential as full and equal members of society. While empowerment often comes from within, and individuals empower themselves, cultures, societies, and institutions create conditions that facilitate or undermine the possibilities for empowerment. • Gender integration involves identifying, and then addressing, gender inequalities during strategy and project design, implementation, and monitoring and evaluation. Since the roles and power relations between men and women affect how an activity is implemented, it is essential that project managers address these issues on an ongoing basis. Figure 7. USAID Gender Equality and Female Empowerment Policy (2012) USAID/Zimbabwe CSSP Final Evaluation 15 This case study examines Pact’s mainstreaming approach to increasing women’s participation in CSSP activities. The case study also seeks to describe the work of one CSSP grantee to better understand its approach to increasing women’s participation in decision-making, leadership, and inclusive development for its community. Using evidence gathered from a KII with grantee organization staff and FGDs with that organization’s direct and indirect beneficiaries, this case study considers best practices, challenges, and recommendations for how to replicate this work with other CSOs and CBOs in Zimbabwe. Gender Mainstreaming Analysis Pact’s technical application proposed a gender mainstreaming approach to increase women’s participation in CSSP activities. Specifically, Pact stated that it would “ensure equal access of men and women to capacity development activities under Component 1. To strengthen gender sensitivity among our partners, Pact will provide gender training, mentoring and coaching of both men and women.”10 The technical application goes on to state that “Pact will promote gender equality by working to change the knowledge, attitudes and behaviors of civil society leaders – both men and women – through gender-sensitive training, mentoring and peer learning.”11 This gender mainstreaming approach aligns with USAID’s Gender Equality and Female Empowerment Policy (2012). In practice, evidence suggests that Pact indeed took a mainstreaming approach to women’s inclusion. All CSSP grantees that the ET met with described how Pact provided dedicated training and coaching on women’s inclusion—both to support internal organizational changes that would be more equitable for women, and to promote women’s participation in CSSP grantee activities. Specifically, grantees described how Pact helped them each develop gender policies to guide a gender-sensitive approach to all aspects of the organization’s work (including human resource management, financial management, and programming). CSSP grantees the ET met with described how Pact training and support helped them re-think why women should be included equally in their work. One grantee discussed how Pact’s gender training helped the organization to increase women on staff and ensure that trainers and speakers were more representative and included women. Similarly, every CSSP grantee the ET met with described how Pact encouraged women- (and youth-) focused activities. Working to increase women’s participation in activities was a natural fit for grantees that were already working on gender issues, were women-led, or focused on engaging women as a primary beneficiary group. Grantees that focused on youth, human rights, or good governance reported that Pact’s training helped them re-think their approach to activity design in order to overcome barriers to women’s participation. For example, one group described how it planned activities to coincide with the times women were available to attend events, as well as focus community discussions on issues of importance to women (such as healthcare, education, or food security). Overcoming Challenges and Barriers to Women’s Participation Political: Many grantees found that if they increased women’s knowledge of their rights as outlined in the new constitution—and worked with local authorities to explain why women should be invited to community dialogue events—they could make progress in including women as speakers, those asking questions, and audience members at these events. At least three CSSP grantees reported using this tactic as a successful method of increasing the presence of women in community activities. Additionally, the same CSSP grantees described that once women involved in the organizations’ constitutional awareness activities understood their rights and what their role should be, those women were more eager to make the time to participate in the CSOs’ activities. 10 Pact (2012). Civil Society Strengthening Project in Zimbabwe Technical Proposal. 11 Ibid. Figure 8. Case Study on Women’s Participation in CSSP USAID/Zimbabwe CSSP Final Evaluation 16 Economic: Several grantees and their beneficiaries described that Pact provided guidance on how to incorporate activities or topics that specifically addressed the economic issues of most concern to female community members. (In this context, women are often concerned about food security, meal preparation, and care of children and homesteads.) Two CSSP grantees specifically mentioned that by focusing on social accountability issues, they increased the number of female attendees at their community events.12 Others described that the main tactic they used to increase women’s participation was to schedule events at times that women were most likely to be free of other household duties. At least 30 percent of CSSP grantees that the ET met with mentioned difficulties in getting women to take part in activities that do not include an economic empowerment component. Thus, a strict interpretation of DRG programming themes may hamper civil society’s ability to encourage a greater number of women to participate in such activities. This barrier might have been addressed had the CSSP program design allowed or encouraged Pact to link its DRG-focused activities with other livelihoods, economic empowerment, or food security programs. Social and Cultural: Patriarchal norms and traditions relegate many women, especially once they have had children, to the home. Whether it is providing childcare, ensuring meals are cooked, or assisting elderly family members, women are often seen as having less to contribute to decision-making in communities. One youth FGD participant stated that once a girl has a child, even if she is in her late teens or early twenties, she is no longer considered a youth but is considered a woman; activities that focus on youth leave her out, and activities focused on women are generally about topics or issues that are not yet of concern to her. This dilemma creates a gap in how young women are included in programming, as many activities that CSOs implement do not target young women as participants or beneficiaries. Culturally, men do not have the same respect for women’s inputs as they do for those of their fellow men. Several CSSP grantees worked to raise awareness of issues important to women, which incidentally are also of concern to other community members. Several CSOs described how this approach was helpful in discussing issues and empowering women to query local decision-makers on these issues, thereby raising their profile within the community. CSSP Grantee Success Story The ET met with a CSSP grantee organization in Bindura that successfully addressed the challenges and barriers to women’s inclusion described above. The organization took a holistic approach to its work on political engagement, entrepreneurship, self-care, and stewardship specifically for young women. The organization contended that if women understood their rights and roles in the community, understood what local decision-makers could and could not provide—and if there were specific entry points for women and decision-makers to discuss community priorities—then issues that were important to women and their families could be better addressed. The grantee’s activities focused on two main areas of support: 1) training and education, and 2) community events that engaged local development councils and community members. The training and education component sought to inform women of their rights under the new constitution and educate women on the roles and responsibilities of local decision-makers. Following the training component, participants were encouraged to participate in social accountability activities funded under the CSSP grant, including attending community dialogue events, attending village development council sessions, and engaging fellow community members on issues of concern. The ET notes that the grantee allowed women to bring their children to training events and sought to accommodate the needs of young mothers in order to encourage more women to be active in their communities. 12 The ET was unable to verify the grantees’ indicator data, as quarterly reports included incomplete data and the ET did not have access to the raw program data in the DevResults system. USAID/Zimbabwe CSSP Final Evaluation 17 The grantee’s integrated programming approach achieved notable results: • Women who graduated from the organization’s training program were eager for more skills building and learning opportunities. One graduate continued to volunteer and take part in the organization’s activities. • One graduate now sits on the local village development council. • One training participant told the ET that the training she received from this group—combined with her involvement in community events—made her want to run for political office. The energy, confidence, and enthusiasm for community activism that the grantee’s work is instilling in these female participants is commendable. USAID should look to use this organization as a model and mentor to other organizations seeking to increase women’s participation in their communities. Finding 3: Despite a number of challenges, CSSP grantees reported a modest ability to increase youth participation. CSSP grantees and beneficiaries reported to the ET that including, and increasing, the number of youth participants in activities is difficult. The economic crisis has made the youth population “fluid,” moving around the country and across borders to find employment and vocational training opportunities. As well, data collected through KIIs and FGDs in many communities indicated that youth voices are not valued due to cultural factors. For example, one male FGD participant stated “I don’t care about engaging local council because as a young person I don’t own a house,” and several FGD participants nodded their heads in agreement. Despite these challenges, CSSP grantees reported that they made a deliberate attempt to ensure that there was fair representation of youth so that all views were considered. Community mobilization and engagement initiatives, including youth dialogue events, provided a platform for the deliberate involvement of this once marginalized group. CSSP grantees reported that youth are now part of community engagement discussions and their views are now considered. One participant reflected that youth were not allowed to speak in the presence of elders, but this has now changed. While this particular respondent didn’t provide additional specifics into how this had changed for him/her, presumably this was because the youth were given an opportunity to speak more, raising their profile and giving them a bit more respect from others. Finding 4: CSSP staff worked with local partners to identify solutions aimed at increasing participation of women and youth. Both Pact and CSSP grantees described to the ET how they identified barriers to increasing the participation of women and youth, as well as implemented solutions to overcome those barriers. CSSP staff reported that they worked with local partners to do so. For example, Pact worked with partners to form voters’ clubs that targeted youth and women. These clubs were formed in order to raise voter awareness among youth who were reportedly very passive. CSSP grantees reported working with women and youth who were engaged in activities that increased their knowledge of the constitution, the new bill of rights, and civic activism. Two CSSP grantees noted that they each had one female training participant appointed to community leadership structures (local village development councils). CSSP grantees designed activities that would attract youth to ensure their active participation, such as sports and theatre arts. CSSP grantees also held activities in spaces where youth were very active, for example, clean-up campaigns at shopping centers. Several CSSP grantees expressed frustration about youth participation, as they invest in training the youth only to have them move away due to economic issues. Overall, as Figure 9 describes, more than half of mini-survey respondents rated inclusion of women and youth in CSSP activities as either “good” or “very good.” USAID/Zimbabwe CSSP Final Evaluation 18 CONCLUSIONS Conclusion 1: Civic participation of women and youth expanded under CSSP. The ET concluded that there was an increase in the participation of women and youth due to a more conducive operating environment and support provided through the CSSP project. CSOs managed to facilitate dialogue between youth and “duty bearers” at the village level. Based on grantee monitoring data, women and youth were identified as the highest participating groups in demonstrations, discussions, and meetings aimed at taking “service bearers” to account for use of community resources.13 Conclusion 2: Barriers to youth and women’s participation could be addressed with additional project support. Some CSSP grantees reported that, given the high level of unemployment in Zimbabwe, economic challenges may have dissuaded some youth and women from making the time to participate in DRG-focused activities. However, Pact and its local partners successfully implemented solutions to increase participation by these groups. For example, activities such as sports and theatre arts promote youth participation. The ET met with a CSSP grantee organization in Bindura that successfully addressed the challenges and barriers to women’s participation, including enabling new mothers to bring children to training events and continuing to engage training program “graduates” in local community events. The energy, confidence, and enthusiasm for community activism that the grantee instilled in these female participants is commendable. USAID could look to use this organization as a model and mentor to other organizations seeking to increase women’s participation in their communities. 13 The ET did not review raw data stored in DevResults, which may track levels of participation by gender and age. 84% 78% 13% 16% 3% 5% Inclusion of Women Inclusion of Youth Very Good - Good Satisfactory Poor - Very Poor Figure 9. Grantees’ perceptions of inclusion of women in CSSP-funded activities Mini-survey Questions 4 and 5 - How do you rate your organization’s inclusion of women/youth in the following activities: community engagement, social accountability, advocacy, voter registration/mobilization, community mobilization, dialogue with local authorities, awareness raising, media/information sharing, women, youth, and justice/human rights? USAID/Zimbabwe CSSP Final Evaluation 19 Sub-Question 2c) - To what extent did the CSSP intervention contribute to increasing sustainable institutional and technical capacity of target local CSOs? In what areas do they still need support? FINDINGS Finding 1: CSSP grantees reported being more professional, sustainable organizations as a result of Pact’s capacity building support. All CSSP grantees independently reported that Pact helped them to build internal systems that increased their efficiency in reaching their organizational goals. Specifically, grantees mentioned program management, financial management, human resources, and M&E as the areas of support from Pact that had the most impact. One grantee reported that the capacity building support provided by Pact through the ISP made them eligible for direct USAID grants, while others reported that the capacity building support Pact provided helped them be more professional and positioned them to compete for funding from other donors. Finding 2: The ET lacks sufficient evidence to assess the extent to which Pact increased the technical capacity of CSSP grantees. Across the board, when asked to describe the capacity building support Pact provided, every single grantee prioritized support on organizational development topics (financial management, M&E, human resource development) over technical topics related to DRG (advocacy, social accountability, community mobilization). Based on anecdotal evidence and analysis from the mini-survey (see Figure 10 below), CSSP grantees indicated that CSSP did contribute to their sustainability. Based on the organizational capacity assessments, organizational network analysis, and organizational performance index tools it used to measure organizational progress toward sustainability and technical capacity, Pact reported to the ET that organizations were successful in these areas. As previously noted, the ET did not have access to the DevResults system and did not have access to all raw M&E data. As the ET could not verify the data in DevResults, the ET cannot independently verify this finding. Figure 10. Grantees’ perceptions of CSSP support and effect on sustainability Mini-survey Question 13 - To what extent do you agree with this statement: The CSSP support has made a huge difference in terms of helping with organizational sustainability. Finding 3: Institutional Support grantees reported that they continue to utilize new skills and knowledge gained through their CSSP grants, even once financial support from Pact ended. This fact speaks to the success of Pact’s approach; despite no longer receiving funding from the CSSP grant pool, grantees continue to implement their institutional support plans—solely because they see the merit and value in what that increased professionalization and organizational development means for the work that they are doing. 8% 11% 81% Agree/Strongly agree Neither agree nor disagree Disagree/Strongly disagree USAID/Zimbabwe CSSP Final Evaluation 20 Finding 4: CSSP grantees reported a “knowledge gap” and a desire for a higher level of technical support. Several grantees that the ET met with described what they called a “knowledge gap”; they had hit the ceiling of what they could learn under the CSSP project and were ready for the next level of experience. For example, one group described that it needed additional technical support to better understand policy areas and legislation, assess decision-makers’ positions, and advocate for related DRG issues. The group described that CSSP had increased its capacity to a certain level, but that it needed additional technical knowledge to be able to speak more confidently and knowledgably to decision-makers. Another example comes from a CBO grantee’s beneficiary who felt that there was a ceiling of support from CSSP in terms of activity and training content. This key informant completed all of the available training but was eager for more and felt that the CSSP project could provide more. Offering grantees tailored technical support on DRG topics may have stretched the limits of Pact’s existing capacity, as the ET notes that most of Pact’s capacity building support via CSSP focused heavily on organizational development. CONCLUSIONS Conclusion 1: CSSP grantees are more sustainable because of the support received from Pact. CSSP grantees consider themselves stronger, more professional organizations as a result of the capacity building support they received under CSSP. During the KII with the Pact CSSP team, the ET was informed about how Pact measured and tracked this portion of the project: Pact’s Organizational Performance Index tool. Unfortunately, the ET did not have an opportunity to independently verify the data that constitutes the indexes results, as the ET did not have access to the data that was stored in the DevResults system. The ET did hear directly from grantees, and from USAID, that several CSSP grantees had increased their organizational capacity to the point that they are now direct grantees of USAID. EQ 3: MONITORING & EVALUATION Sub-Question 3a) - How effective was the program's monitoring and evaluation system in providing quality data for decision making and reporting? FINDINGS Finding 1: Grantees expressed overwhelming appreciation for and use of Pact’s M&E training and guidance on recording meaningful monitoring data. Every respondent had nothing but praise for Pact’s M&E support. Specifically, grantees commented that Pact: • Helped grantees focus more on achieving outcomes than strictly on completing proposed activities; • Strengthened grantees’ activities. Examples include combining activities to maximize available resources, adjusting activities to make them more palatable to local authorities, and providing extra support to young mothers or other young women to encourage their participation; and • Improved grantees’ M&E systems (both for CSSP and other donor projects). CSSP grantees reported that they continue to use Pact’s M&E system, tools, and templates even after their CSSP funding ended. The ET notes that every CSSP grantee that the team met with could describe its work in terms of outcomes. This is due, according to grantees, to Pact’s outcome mapping training. The ET was impressed with the capacity the grantees displayed in terms of discussing their work as intended outcomes and not only as activities. In an industry that is plagued by the grant-to-grant activity cycle to which most civic groups succumb, it was refreshing to find so many local organizations that understood the value of working toward outcomes and that adjusted activities and approaches to meet those outcomes. This speaks directly to the effectiveness of Pact’s outcome mapping and M&E training for CSSP grantees. USAID/Zimbabwe CSSP Final Evaluation 21 Furthermore, mini-survey results show that an overwhelming majority of respondents (89 percent) reported using M&E data (Figures 11 and 12). Of the Rapid Response grant participants, 26.7 percent are using M&E data to solicit more funding while 20 percent are leveraging it to request additional capacity building support. Additionally, 26.1 percent of the Institutional Support Program participants reported using M&E data to add/remove activities, and 17.4 percent are using it to request additional capacity building support. These findings further demonstrate the effectiveness of Pact’s M&E trainings. 26.7% 20.0% 20.0% 13.3% 13.3% 6.7% 0.0% 0.0% 4.3% 17.4% 17.4% 26.1% 8.7% 13.0% 8.7% 4.3% Requested additional financial resources Requested additional capacity building support Combined activities Added/Removed activities Not Applicable Changed target beneficiaries Requested additional technical support Don't know Rapid Response Institutional Support Figure 12. Specific Uses of M&E data by CSSP grantees Mini-survey question 12 - How did you use your M&E data to adjust your activities? 89% 11% Yes No Figure 11. Use of M&E data by CSSP grantees Mini-survey question 11 - Did you use M&E data to adjust your activities? USAID/Zimbabwe CSSP Final Evaluation 22 While the ET did not have access to detailed M&E data from Pact, the team did review a sample of grantee documents, including grantee reports. Most of these reports indicated that grantees were reporting on DRG outcomes. The ET M&E Specialist did find some irregularities across reports from grantees where indicators did not match up from quarter to quarter. Finding 2: The ET was unable to verify CSSP M&E data, including sample grantee data, because the ET was not given access to the DevResults system. DevResults is an online database designed to be a central repository for M&E data from all USAID grantee organizations (cross-sector). The database includes indicator, output, and outcome data as well as Organizational Capacity Assessment (OCA), organizational network analysis (ONA), and Organizational Performance Index (OPI) tools (used by Pact to measure grantee capacity). The Pact team delivered a presentation to the ET about the DevResults system, describing to the ET that the system was designed to track each grantee’s performance over time; reports and visual graphs were presented.14 Beyond reviewing reports generated by DevResults, the ET could not independently verify the nature of the raw data within DevResults. The ET learned, after meeting with USAID, that DevResults contains data about all of the DRG Office’s grantees. The DevResults system is restricted for use by Pact and USAID. Therefore, the ET assumes that it could not obtain uninhibited access to review CSSP data, given sensitivities related to data about other DRG Office grantees. USAID reported to the ET that the DevResults system only became fully functional in the final year of CSSP. This delay resulted in the capturing of data within the DevResults system in retrospect, which may have adversely affected decision-making. The ET does note that CSSP grantees were unaware of these challenges with DevResults and did not comment to the ET that they noticed any delay in receiving feedback from Pact on their M&E reports. CONCLUSIONS Conclusion 1: Pact’s M&E training and procedures contributed to improved reporting, especially in regards to grantee outcomes. CSSP grantees overwhelmingly praised Pact’s M&E support. Grantees’ descriptions of how they reported on project outcomes versus activities indicates to the ET that Pact’s capacity building around M&E was extremely effective. All CSSP grantees reported that Pact’s M&E training and reporting feedback mechanisms were effective in helping them meaningfully document project activities and capture related outcomes. Sub-Question 3b) - What challenges and successes were realized? FINDINGS Finding 1: CSSP M&E data measured apples and oranges: outcomes were focused on organizational development, while indicators measured outputs related to DRG activities. As described above, the CSSP IRs, expected outcomes, and indicators were mismatched. Pact used quantitative output indicators to assess its progress toward the overall goal of protecting and promoting democratic space (USAID DO). In order to capture outcomes that were not quantitative in nature, Pact conducted monitoring visits to discuss grantees’ work. Success stories and case studies were also documented to showcase partners’ contributions in the different communities. 14 The ET could not independently verify the data on which the analysis was based. USAID/Zimbabwe CSSP Final Evaluation 23 Figure 13. Sample of Expected Results, Outcome Indicators, and Output Indicators Expected Results15 • Institutional capacity of community based organizations strengthened (Component 1+3) • CSOs are more effective in advocacy for democratic reforms domestically and regionally (Component 1) • Increased participation, particularly among women and youth, in democratic processes in target communities (Component 3) • Communities and democratic reformers are prepared to protect themselves from intimidation and other forms of violence (Component 3) Outcome Indicators • Percent change in averaged advocacy index scores in target CSOs • Percent change in ONA scores for network CSOs • Percent increase in organizational performance index score for supported organizations (Component 1+3) Indicators16 • Number of USG assisted CSO public gatherings shut down or rescheduled due to actions by Government of Zimbabwe authorities • Number of public gathering planned or implemented • Number of communities where citizens gather to engage with local and/or government leaders, with USG assistance, at least 3x/year • Percentage youth dialogue clubs that met with elected officials to present priority issues quarterly • Percentage public forums held feature youth and/or women speakers • Number of targeted districts with access to at least two alternative media sources through USG assistance • Number of civil society organizations receiving USG assistance engaged in advocacy interventions (standard) CONCLUSIONS Conclusion 1: The incongruity of CSSP M&E data makes it difficult to measure Pact’s overall effectiveness in contributing to USAID’s DO and DRG outcomes. Given the discrepancies between outcomes and indicators, the ET could not definitively determine the extent to which the Pact approach contributed to USAID’s DRG goals. This is as much a design issue as an M&E issue, and the ET is unsure why neither USAID nor Pact caught this discrepancy throughout the life of the project. 15 Information for this table was taken directly from Pact’s technical application and quarterly indicator data provided to the ET by USAID and Pact. 16 A sample of the indicator data the ET was provided with, via quarterly/annual reports, is included in this row. This sample is from the FY2014 annual report. 24 V. RECOMMENDATIONS The ET offers the following recommendations based on its findings and conclusions. RECOMMENDATIONS FOR USAID/ZIMBABWE 1. Continue to support DRG-related activities in Zimbabwe. This is especially pertinent in the run up to the 2018 elections, when many CSSP grantees fear that democratic space will close if there is no support for them to maintain the gains they have made in the last several years. 2. Develop support mechanisms that link DRG activities to other issues affecting Zimbabweans’ everyday lives—including livelihoods and economic empowerment. Citizens do not separate these concepts; neither should USAID support. 3. Continue to support implementing partners to conduct frequent analytical research to directly inform program implementation. 4. Develop flexible funding mechanisms that support CSOs/CBOs through long-term grants (2- 3 years), as well as rapid response funding that is shorter and more flexible in nature. Additionally, identify mechanisms to continue to support grantees after a grant cycle ends. 5. Design projects that are cross sectoral that include both issues of community concern, such as livelihoods, food security, education or health, with DRG related skill sets, such as advocacy, social accountability and community mobilization. As these issues are not separate for citizens, new project designs should look for ways to meet these linked needs. 6. Support projects that address existing barriers to participation of youth in civic life. Youth need to be engaged in activities that address their economic realities and motivate them to become active citizens. 7. Support projects that promote activities that specifically target the inclusion of young women. Women need to continue receiving mentoring and coaching so that they may be confident in assuming leadership roles and expressing their opinions. 8. As the political and social context allows, support more in-depth analysis of gender roles as they relate to DRG programming. For example, discussion should include the importance of sensitizing men to the benefits of including women in decision-making processes. 9. Consider utilizing Pact as a sector-wide capacity builder for M&E with local partners. 10. Determine whether DevResults meets USAID’s needs, and explore whether it is useful to maintain the system for existing and future programming. 11. For all projects, ensure that intended results, expected outcomes, and indicators are measuring the same thing and “speak to each other” to ensure that project effectiveness and impact can be measured at regular intervals and at the conclusion of the project. RECOMMENDATIONS FOR IMPLEMENTING PARTNER 12. Develop a systematic, metric-based feedback mechanism to document why local partners did or did not move through RFA rounds of support. Such a mechanism would allow applicants to understand areas they need to improve upon. A system similar to USAID’s own evaluation and scoring criteria could be tailored and used for this purpose. 13. Target and tailor capacity building support to CSOs/CBOs based on individual organizational needs, including more mentoring and in-person coaching. 25 ANNEXES Annex A: Evaluation Statement of Work Annex B: Evaluation Evidence Matrix Annex C: Data Collection Protocols Annex D: Bibliography of Documents Reviewed Annex E: Key Informants Annex F: Mini-Survey Data Annex G: Disclosure of Any Conflicts of Interest USAID/Zimbabwe CSSP Final Evaluation 26 ANNEX A: EVALUATION STATEMENT OF WORK Statement of Work for Performance Evaluation for USAID/Zimbabwe Civil Society Strengthening Project Background Zimbabwe’s political context is often described as faltering democratization and stalled transition. Since becoming an independent republic in 1980, the country has disputed and often violent elections. The highly contested 2008 elections were fraught with severe political violence and voter intimidation. Economic collapse and a breakdown of the public health system, a cholera epidemic, and food shortages compounded this point of political crisis. Seeking legitimacy after the internationally condemned election, the ruling Zimbabwe African National Union (ZANU) (PF) party conceding and the opposition Movement for Democratic Change (MDC) reached political settlement, signing the inter-party political agreement- the Global Political Agreement (GPA), which established a government of national unity (GNU). The period from 2009 to 2013 provided some respite from the political and economic turmoil of prior years with a promising reform agenda. The adoption of the multi-currency system in 2009 (dollarization) brought a degree of stability, particularly for the economy. However, the GNU struggled and largely failed to effectively implement the actions outlined in the GPA, initiate reforms, or stem the erosion of public institutions. The most significant development of the period was the enactment of a new constitution with a progressive bill of rights in May 2013. If fully implemented, this constitution could serve as a new social contract between the government and citizens. The GNU ended in 2013 with the holding of national elections, which saw a dramatic change in Zimbabwean politics returning single-party domination by ZANU (PF) of the executive and parliament with a super majority for the first time since 1999. Since 2003, the government has largely failed to deliver on improving the economy. In addition, the reform agenda necessary to implement the new constitution has been slow. Meanwhile citizens’ expectations for the government to defend human rights and advance development have increased, thereby increasing pressure on the government to deliver. Description of the Program Development Hypothesis USAID/Zimbabwe has a long history of support to local civil society organizations (CS0s) who seek to promote citizen participation in advocacy and oversight of democratic institutions and processes (including elections); human rights and peace-building. In achieving its Development Objective civil society is both USAID's core implementing partner as well as the primary beneficiary of its assistance. On the one hand, USAID/Zimbabwe provides direct support to 8 local civil society organizations via long-term grants. These programs are complemented by the CSSP contract, which provides a combination of technical assistance, organizational development and leadership mentoring to direct grantees and sub-grantees; a Grants Under Contract (GUC) sub-granting mechanism; The CSSP sub-granting mechanisms work with CSO's and CBOs to contribute to the DRG portfolio's results framework including four intermediate results described below. CSSP also provides analytical support to USAID/Zimbabwe to enhance its analysis and management; monitoring of the overall p01tfolio as well as assess the developments, needs and challenges and opportunities of Zimbabwe's civil society writ large. From 2013 the USAID Zimbabwe's DRG strategy and this program sought to provide strategic and tactical civil society support to sustain critical advocacy campaigns that would contribute to positive democratic gains. Before the 2013 election, the focus was on promoting a more democratic and accountable system of governance by supp01iing a more open and neutral electoral process, citizens' participation, and support to parliament. While holding the government to account during this transitional period was important, there remained an equal need to find ways to articulate citizen demands for a credible election including acceptance of its outcomes. This included voter registration and Get-out-the-vote campaigns particularly USAID/Zimbabwe CSSP Final Evaluation 27 among women and young people as well as election observation. The USAID DRG results framework was as follow: Development Objective (DO) Support more accountable and democratic system of governance • IR 1. 1: Key government institutions more accessible for citizen input • IR I. 2: Citizens are mobilized to make their voices heard However, as the political and governance context in Zimbabwe changed after the July 2013 harmonized elections, strategic adjustment and refocusing has taken place. In the post-election period the program continued to focus on increasing community level cohesion; facilitating constructive citizens' engagement with both elected and non-elected leaders at both national and local level, advocacy for the implementation of the 2013 Constitution and other legislative refom1. The real risk of democratic reversal shifted the strategic focus to safeguarding and expanding democratic space where opportunities arose, paying particular attention to the implementation of the new constitution enacted in May 2013 with its expanded bill of rights. In this new context, support to civil society focused on strategic reflection and strengthening connections with the grassroots. Civil society's strategic shift resulted in a redefinition of issues to focus more on socio￾economic rights and issues such as service delivery and access to basic needs such as water, electricity, food and health) to build citizens' capacity and motivate civic participation. This revised results framework, since September 2013, is as follows: Development Objective (DO) Protect and promote democratic space, • IR 1. l: Citizens particularly women and youth, are mobilized to make their voices heard • IR 1. 2: Evidence based advocacy increased • IR l. 3: Parliamentary Committees exercise independence The main objective of the Civil Society Strengthening Program (CSSP) implemented by Pact is to assist USAID/Zimbabwe and its local implementing partners in achieving the overall Development Objective (DO) of the Democracy, Human Rights, and Governance (DRG) portfolio. The CSSP goal is to support achievement of the DRG DO - Protect and Promote democratic space, with the following Intermediate Results (lRs): IR l: Grants of financial resources to CSOs are targeted and administered effectively IR 2: Targeted CSO's and CSO networks are more effective and sustainable IR 3: Improved information availability and analysis for the USAID program. The Civil Society Strengthening Program therefore included four major components: Component I: Institutional and Technical Strengthening of selected local Civil Society Organizations (CSOs) USAID/Zimbabwe CSSP Final Evaluation 28 Component 2: Rapid Response Grants to Traditional and Nontraditional Civil Society Actors Component 3: Institutional Capacity Building of Small or Community Based Organizations Component 4: Analytical Support to USAID for the Democracy, Human Rights and Governance Portfolio CSSP's activities arc cross-cutting in the USAID Zimbabwe Democracy, Rights and Governance (DRG) portfolio, intended to be flexible and respond to emerging needs/opportunities while purposely reaching out to non-traditional CSOs. CSSP provides institutional and technical assistance to strengthening of targeted local CSOs; small short-term/Rapid response (3 months) and long-term (12-18 months) sub-grants to CSOs and community based organizations (CB0s). USAID anticipates that by its end, the program will have given approximately 120 short-term and 60 long-term sub grants to civil society for activities in 45 districts across the county's 10 administrative provinces. Short term grants mainly focus on support to short term but innovative projects designed to fill a gap in programming or to exploit a window of opportunity in the operating environment CSSP in its design recognized that it was not sufficient to simply support the technical programs of USAID's Zimbabwean partners. It therefore focuses on an institutional and technical capacity-development as part of its long term Institutional Support to CBOs/CSOs (who include USAID direct grantees). The graphic presentation below shows the CSSP Results Fran1ework and how it contributes to the overall goal of the DRG Results Framework Project implementation consisted of two phases, namely pre-election and post-election. In the pre-election phase from June 2012 to July 2013 activities designed to contribute to the conduct of the elections and participation of women and young people dominated. The post-election phase started after the elections (August 2013 to end of project) and the end of the government of national unity (GNU) with more focus on protecting and promoting democratic space which was part of the gains of the GNU. In 2015-2016 the project will focus mainly on citizen capacity to lead social change through citizen based actions and sustained dialogue and engagement with solution holders as well as national level advocacy to influence continuing democratic reforms. USAID/Zimbabwe CSSP Final Evaluation 29 Evaluation Purpose The purposes of this performance evaluation are to: • Assess performance, including programmatic achievements; factors impeding or augmenting success including the program design; andpositive or negative unintended consequences. • Document lessons learned from the CSSP to inform potential future programs. Evaluation Questions The evaluation questions and sub-questions below address key issues of program design, implementation effectiveness, impact, sustainability, and lesson or best practices that are relevant for the design of future programming. 1. Appropriateness and effectiveness of program design - • To what extent has the CSSP model with its mix of components (local capacity development and both long and short term (rapid response) sub grants to local organizations on the one hand and analytical support to USAID on the other), and the way it was implemented actually aided or detracted from achieving program outcomes? • What could be modified to improve effectiveness? 2. Achievement of intended results - • To what extent did the CSSP achieve its intended results? What was successful and not successful? Why? • How effective was CSSP at increasing youth and women's participation in target communities? The ET is requested to take a deeper look at this component of desired program outcomes using a case study approach. • To what extent did the CSSP intervention contribute to increasing sustainable institutional and technical capacity of target local CSOs? In what areas do they still need support? • What recommendations to improve methods or approaches can be made? 3. Monitoring & Evaluation - How effective was the program's monitoring and evaluation system in providing quality data for decision making and reporting? Focus should be at all levels from sub grantee, prime, to USAID. What challenges and successes were realized? Evaluation Design and Data Collection & Analysis Methods The contractor shall propose the most appropriate evaluation design and data collection methods based upon the stated purpose and questions of the evaluation. The contractor should complete the evaluation using a combination of quantitative and qualitative analytical methods. Evaluation methods should ensure demographic and geographic representativeness across the total population of beneficiaries. Methods selected should use sex- and age-disaggregated data and incorporate attention to gender relations in citizen participation. Particularly, the contractor will ensure that women and youth beneficiaries are well￾represented in any sample to gauge the effectiveness of the program in achieving greater gender parity and generational disaggregation. The contractor is responsible for determining the most appropriate data analysis methods given the selected evaluation methodology and the goals and questions of the evaluation. While qualitative data from interviews and focus groups may form a significant portion of the evaluation team's evidence, it is expected that, wherever possible, the quantitative data be used to substantiate the qualitative data and support conclusions and recommendations. USAID/Zimbabwe CSSP Final Evaluation 30 Table 1: Evaluation Design Matrix – Evaluation Questions and Associated Data Sources and Data Collection and Analysis Methods Evaluation question Data sources Data collection method Data analysis method Appropriateness and Effectiveness of Program Design Q1a) To what extent has the CSSP model with its mix of components, and the way it was implemented actually aided or detracted from achieving program outcomes? 1. CSSP reports; Monitoring data; Evaluation and assessment reports 2. USAID staff; Pact admin/program staff; CSO/CBO admin/program staff; external donors and implementers 1. Desk Study 2. KIIs (purposive) Quantitative: • Un/bivariate analysis Qualitative: • Content analysis • Pattern analysis Q1b) What could be modified to improve effectiveness? 1. CSSP reports; Evaluation and assessment reports 2. USAID; Pact admin/program staff; CSO/CBO admin/program staff; external donors and implementers 1. Desk Study 2. KIIs (purposive) 3. Mini-Survey (stratified random) Quantitative: • Un/bivariate analysis Qualitative: • Content analysis • Pattern analysis Achievement of Intended Results Q2a) To what extent did the CSSP achieve its intended results? What was successful and not successful? Why? 1. CSSP reports; Monitoring data; Evaluation reports 2. USAID; Pact admin/program staff; CSO/CBO admin/program staff; Traditional and/or faith leaders 3. Community members (youth/women/men) 4. CSO/CBO admin/program staff 1. Desk Study 2. KIIs (purposive) 3. FGDs (purposive) 4. Mini-survey (stratified random) Quantitative: • Un/bivariate analysis Qualitative: • Content analysis • Pattern analysis Q2b) How effective was CSSP at increasing youth and women’s participation in target communities? 1. CSSP reports; Monitoring data; Evaluation reports 2. USAID; Pact admin/program staff; CSO/CBO admin/program staff; Traditional and/or faith leaders 3. Community members (youth/women/men) 4. CSO/CBO admin/program staff; Community members (youth/men/women) Case Study Approach (In- depth focus on 2-3 CSOs): 1. Desk Study 2. KIIs (purposive) 3. FGDs (purposive) 4. Mini-survey (stratified random) Quantitative: • Un/bivariate analysis Qualitative: • Content analysis • Pattern analysis • Narrative analysis Q2c) To what extent did the CSSP intervention contribute to increasing sustainable institutional and technical capacity of targeted local CSOs? In what areas do they still need support? 1. CSSP reports; Monitoring data; Evaluation reports 2. USAID; Pact admin/program staff; CSO/CBO admin/program staff 3. CSO/CBO admin/program staff 1. Desk Study 2. KIIs (purposive) 3. Mini-survey (stratified random) Quantitative: • Un/bivariate analysis Qualitative: • Content analysis • Trend analysis Q2d) What recommendations to improve methods or approaches can be made? 1. CSSP reports; Evaluation reports 2. USAID; Pact admin/program staff; CSO/CBO admin/program staff; Traditional and/or faith leaders 3. Community members (youth/women/men) 1. Desk Study 2. KIIs (purposive) 3. FGDs (purposive) Quantitative: • Un/bivariate analysis Qualitative: • Content analysis • Pattern analysis Monitoring and Evaluation (M&E) Q3) M&E: How effective was the program’s M&E system in providing quality data for decision making and reporting? What challenges and successes were realized? 1. CSSP reports; Monitoring data; Data Quality Assessment report; Evaluation reports 2. USAID; Pact M&E staff; Pact admin staff; CSO/CBO M&E staff; CSO/CBO admin staff 3. DevResults system 1. Desk Study 2. KIIs (purposive) 3. Mini-survey (stratified random) Quantitative: • Un/bivariate analysis Qualitative: • Content analysis • Pattern analysis Existing Data As part of desk study, a broad range of useful background specific documents is available for the project. USAID will provide: • CSSP original proposal, including project description, core indicators and targets • Performance Monitoring Plan for the project, with baseline, targets and end-line performance indicator data • Implementer annual and quarterly work plans and reports; activity reports • CSSP mid-term evaluation report • Approved Transitional Country Development Cooperation Strategy (TCDCS) (FY2013- 2015) • Democracy and Governance Assessment (2011) • Civil Society Assessment Report for Zimbabwe (2012) • Elections After Action Review • Complex Crisis Fund (CCF) Evaluation report • DRG Portfolio Evaluation Report USAID will provide the ET with these documents after contract negotiation but prior to arrival in country. Of note, a few of these documents are Sensitive but Unclassified (SBU) and should be treated as such. Management and Staffing The Contractor SI proposes a dedicated three-person management team composed of the PD, PM, and Project Assistant who will guide this PE both technically and administratively. These staff will use SI’s customized project management tools for each phase of the evaluation, including launch and preparation, data collection, and analysis and reporting. The PD will be responsible for overseeing the subcontractor and the other home office management staff. The subcontractor, Broderick Haight, will provide the TL and be responsible for managing her logistics and performance. However, since SI will remain responsible for the quality of the evaluation, our home office staff will have direct access to the TL and provide her with instruction, guidance, and support as needed. Lastly, the TL will manage the rest of the team by delegating responsibilities, training and guiding them as needed, monitoring the progress of their work, and reviewing and providing feedback on their inputs. If there are any issues with team member performance, the TL will immediately report it to the PD and PM who will address it. Logistics The contractor is responsible for arranging their own travel and lodging in Zimbabwe. The Contractor will also be responsible for obtaining the required visas, insurance, vaccinations and any other preparation required to travel to Zimbabwe. Work Week A six-day workweek will be authorized for this contract for work performed outside of the United States Deliverables DELIVERABLE DUE DATE Inception Report Within three (3) weeks of signing the contract. In-briefings Upon arrival in Zimbabwe and before commencing fieldwork for the evaluation. The ET should plan to arrive in Zimbabwe no later than November 1, 2016. Out-briefings Within two (2) weeks of concluding fieldwork in Zimbabwe. Interim Report Within two (2) weeks of concluding fieldwork in Zimbabwe. Final Report Within two (2) of receiving comments for USAID on the interim report. The final report must be completed no later than December 23, 2016. Inception Report Within three weeks of the signing of the contract, the Contractor will provide an inception report, which will detail the work plan and tasks. The inception report should describe the methodological approach, methods of investigation, and planned data collection and analysis. An evaluation design matrix showing how each evaluation question will be tackled should be provided. The inception report shall note planned fieldwork including interviews and identify resources to be utilized including intended interlocutors, specific key informants as known and key informant groups, women, youth, community-based organizations, traditional leaders, and/or faith leaders. The report should also include a protection plan which describes appropriate security measures that will be taken to protect those being interviewed, ET members, and the data being collected. The inception report should also identify any areas of support required of the USG. The inception report shall be a desktop exercise and shall not involve travel or fieldwork. The inception report shall be submitted electronically, in Word format, as an email attachment. The inception report will be reviewed and cleared by a USAJD/Zimbabwe evaluation team, led by the Contracting Officer's Representative (COR), within one week of submission by the Contractor. Briefings Upon arrival in country and before commencing the evaluation, the team will deliver an in­ briefing to USAID/Zimbabwe to present its methodology and fieldwork plan, the content of which is specified above. An additional in-briefing with the COR and key USAID/Zimbabwe technical staff will be held. The ET will share preliminary findings and conclusions of the evaluation with the USG stakeholders including Mission management, technical team, and other key staff. The out­ briefing presentation shall consist of: • Evaluation purpose, methodology; • Preliminary findings, conclusions and recommendations; and • Contractor comments on conducting the evaluation and recommendations for improving such evaluations in the future. Interim Report Within two weeks of concluding field-based research in Zimbabwe, the Contractor will electronically submit an interim report, in Word format, as an email attachment (maximum 35 pages excluding executive summary, table of contents and annexes). The interim report shall include the following sections: • Detachable executive summary that can be used as an independent briefing document (not to exceed 10 pages) • Table of contents • Introduction • Methodology • Findings - to include: a. General analysis b. Responses to questions c. Any strategic and programmatic recommendations • Conclusions • Recommendations • Full listing of references and reports utilized (as an annex) • Full listing of persons interviewed (as an annex) The interim report shall list references utilized and persons interviewed. Findings, conclusions and recommendations should be organized by the evaluation question to the extent that this is possible. The contractor will also be expected to submit all data sets which are used or generated as part of the evaluation. USAID will provide comments, edits and requests for clarification or follow-up to the Contractor within 2 weeks of submission of the interim report. In its comments, USAID will provide direction for modifying or redacting particular items as may be needed for the final public version of the report and marking requirements for the internal USG report. Final Report The contractor will submit a draft final report in Word format, as an email attachment to USAID/Zimbabwe no later than 2 weeks after USAID/Zimbabwe provides written comments on the interim evaluation report. This report shall be fully formatted and branded and include all annexes, particularly the data collection tools and list of interviewees. The final report should follow the outline for the draft report, unless otherwise directed in the comments from USAID. The draft final report should reflect Contractor consideration of USAID/Zimbabwe input to the interim report and should note any substantive disagreements. Inspection and Acceptance Before an invoice for payment is submitted, the Contracting Officer’s Representative (COR) must perform an inspection and acceptance of received deliverables. ANNEX B: EVALUATION EVIDENCE MATRIX Evaluation question Related Outcome Indicator Data sources Data collection method Data analysis method APPROPRIATENESS AND EFFECTIVENESS OF PROGRAM DESIGN Q1a) To what extent has the CSSP model with its mix of components, and the way it was implemented actually aided or detracted from achieving program outcomes? • Component 1 outcome indicator: Percentage of CSOs which improve in Pact’s Organizational Performance Index; Component 1 outcome indicator: Percentage change in averaged advocacy index scores in target CSOs; • Component 1 outcome indicator: Percentage change in ONA scores for network connections; Component 2 outcome indicator: Percentage increase in community participation in political processes • Component 2 outcome indicator: Enhanced capacity of civil society actors 1. CSSP reports; Monitoring data; Evaluation and assessment reports 2. USAID staff; Pact admin/program staff; CSO/CBO admin/program staff; external donors and implementers 1. Desk Study 2. KIIs (purposive) Quantitative: • Un/bivariate analysis Qualitative: • Content analysis • Pattern analysis Q1b) What could be modified to improve effectiveness? N/A 1. CSSP reports; Evaluation and assessment reports 2. USAID; Pact admin/program staff; CSO/CBO admin/program staff; external donors and implementers 1. Desk Study 2. KIIs (purposive) 3. Mini-Survey (stratified random) Quantitative: • Un/bivariate analysis Qualitative: • Content analysis • Pattern analysis ACHIEVEMENT OF INTENDED RESULTS Q2a) To what extent did the CSSP achieve its intended results? What was successful and not successf? Why? • Component 1 outcome indicator: Percentage of CSOs which improve in Pact’s Organizational Performance Index; Component 1 outcome indicator: Percentage change in averaged advocacy index scores in target CSOs; • Component 1 outcome indicator: Percentage change in ONA scores for network connections; Component 2 outcome indicator: Percentage increase in community participation in political processes • Component 2 outcome indicator: Enhanced capacity of civil society actors 1. CSSP reports; Monitoring data; Evaluation reports 2. USAID; Pact admin/program staff; CSO/CBO admin/program staff; Traditional and/or faith leaders 3. Community members (youth/women/men) 4. CSO/CBO admin/program staff 1. Desk Study 2. KIIs (purposive) 3. FGDs (purposive) 4. Mini-survey (stratified random) Quantitative: • Un/bivariate analysis Qualitative: • Content analysis • Pattern analysis Q2b) How effective was CSSP at increasing youth and women’s participation in target communities? • Component 1 outcome indicator: Percentage of CSOs which improve in Pact’s Organizational Performance Index; Component 1 outcome indicator: Percentage change in averaged advocacy index scores in target CSOs; • Component 1 outcome indicator: Percentage change in ONA scores for network connections; • Component 2 outcome indicator: Percentage increase in community participation in political processes Component 2 outcome indicator: Enhanced capacity of civil society actors 1. CSSP reports; Monitoring data; Evaluation reports 2. USAID; Pact admin/program staff; CSO/CBO admin/program staff; Traditional and/or faith leaders 3. Community members (youth/women/men) 4. CSO/CBO admin/program staff; Community members (youth/men/women) Case Study Approach (In- depth focus on 2-3 CSOs): 1. Desk Study 2. KIIs (purposive) 3. FGDs (purposive) 4. Mini-survey (stratified random) Quantitative: • Un/bivariate analysis Qualitative: • Content analysis • Pattern analysis Q2c) To what extent did the CSSP intervention contribute to increasing sustainable institutional and technical capacity of targeted local CSOs? In what areas do they still need support? • Component 2 outcome indicator: Percentage increase in community participation in political processes Component 2 outcome indicator: Enhanced capacity of civil society actors; • Component 3 outcome indictor: Percentage increase in organizational performance index score for supported organizations. 1. CSSP reports; Monitoring data; Evaluation reports 2. USAID; Pact admin/program staff; CSO/CBO admin/program staff 3. CSO/CBO admin/program staff 1. Desk Study 2. KIIs (purposive) 3. Mini-survey (stratified random) Quantitative: • Un/bivariate analysis Qualitative: • Content analysis • Trend analysis Q2d) What recommendations to improve methods or approaches can be made? N/A 1. CSSP reports; Evaluation reports 2. USAID; Pact admin/program staff; CSO/CBO admin/program staff; Traditional and/or faith leaders 3. Community members (youth/women/men) 1. Desk Study 2. KIIs (purposive) 3. FGDs (purposive) Quantitative: • Un/bivariate analysis Qualitative: • Content analysis • Pattern analysis MONITORING AND EVALUATION Q3a) M&E: How effective was the program’s M&E system in providing quality data for decision making and reporting? Component 4 outcome indictor: Increased data use for decision-making and programming 1. CSSP reports; Monitoring data; Data Quality Assessment report; Evaluation reports 2. USAID; Pact M&E staff; Pact admin staff; CSO/CBO M&E staff; CSO/CBO admin staff 3. DevResults system 1. Desk Study 2. KIIs (purposive) 3. Mini-survey (stratified random) Quantitative: • Un/bivariate analysis Qualitative: • Content analysis • Pattern analysis Q3b) What challenges and successes were realized? N/A 1. CSSP reports; Monitoring data; Data Quality Assessment report; Evaluation reports 2. USAID; Pact M&E staff; Pact admin staff; CSO/CBO M&E staff; CSO/CBO admin staff 3. DevResults system 1. Desk Study 2. KIIs (purposive) 3. Mini -survey (stratified random) Quantitative: • Un/bivariate analysis Qualitative: • Content analysis • Pattern analysis 38 ANNEX C: DATA COLLECTION PROTOCOLS KEY INFORMANT INTERVIEWS USAID/Zimbabwe Civil Society Strengthening Program (CSSP) Evaluation Coversheet for All Key Informant Interview (KII) Protocols Interview Details Key Informant Details Date of Interview: Name, Title: Time Start: Time End: Affiliation: Team: Sex: Male / Female (circle one) Interviewer(s): Location: Introduction Good morning/afternoon and thank you for taking the time to speak with us today. As mentioned during our interview request, we are working with USAID/Zimbabwe to conduct a final evaluation of the Civil Society Strengthening Program (CSSP) implemented by Pact. The evaluation is intended to provide an informed assessment of progress to date and recommend potential modifications, if needed. Our team has reviewed some background documents to get a better sense of the design and implementation of the project. However, documents can only tell us so much. We would like to speak with you today to hear about your experience, in your own words, to help us better understand how CSSP operates “on the ground.” Confidentiality Protocol • We will take note of the people we speak with, but for security precautions, we will not provide a list of the names of any key informants in our final evaluation report. • We may include quotes from respondents in the evaluation report, but we will not link individual names, organizations, or personally identifiable information to those quotes, unless written consent is granted by the respondent. Should the team desire to use a particular quote, photograph, or identifiable information in the report, the evaluators will contact the respondent(s) for permission to do so. • All data gathered will be used for the sole purposes of this evaluation and will not be shared with other audiences or used for any other purpose. • Your participation in this interview is voluntary and if you do not feel comfortable answering a question, please let us know and we will simply go on to the next question. Once again, thank you for taking the time to speak with us today. Do you have any questions for us before we get started? Inform interviewee we may follow-up with additional questions at the end of fieldwork 39 USAID/Zimbabwe CSSP KIIs (Est. Time ~45 min) USAID Staff Introduction and Project Scope • What has been your involvement with CSSP? Appropriateness and effectiveness of CSSP program design (EQ1) • Can you describe the goal of the CSSP project to date, as you understand it? (EQ1a) • How has Pact’s integrated approach of technical and financial support affected the project’s outcomes? (EQ1a) • How could this model, in your opinion, be improved upon in future iterations? (EQ1b) Grants Program (EQ1a) • Can you describe the grant component of CSSP? • What were the challenges to the grant program? What could be done differently? Interviewer Note: Extra EQ1 information if there is time to ask • Have there been any changes to the scope or activities since you have been involved with CSSP? If yes, please describe any and all changes, including contextual factors, intervening circumstances, or other conditions that may have affected implementation. • Are there any contextual factors, such as Zimbabwe’s political environment, that have affected this project? Achievement of Intended Results (EQ2) • What project outcomes have you observed to date? o Have you observed any unintended/unexpected results (positive or negative)? If yes, please describe. • What do you think are the project’s high points/successes? • Can you describe the technical support that you observed between Civil Society Organizations (CSOs) and Pact that took part in CSSP? o How could that support be further strengthened or expanded? • Did CSSP increase the institutional and technical capacity of partner CSO/Community-based Organizations (CBOs)? • If yes, which capacities did CSO/CBOs develop as a result of CSSP? • If yes, do you believe that increased capacity is sustainable? Why, or why not? (EQ2c) • In what areas do CSO/CBOs need additional support? • Has the democratic needle moved forward as a result of CSSP? • If yes, can you provide examples? • If no, why not? • Are there any barriers, in your estimation, standing in the way of the project achieving its stated objectives? • Has the project encountered any challenges? If so, please describe (program design challenges, implementation challenges, or other). o If challenges did occur, how did they affect the project? o How did the project address and overcome the challenges? • What recommendations do you have for the project? (EQ2d) 40 Gender and Youth (EQ2b) • What are the barriers to the meaningful participation of women and youth in your areas of work (in terms of locations and issues)? • Can you describe any observations about women and youth participation in CSSP? • Did Pact integrate women and youth into its program design in a way that produced meaningful participation by both? Alternatively, could they have done something differently to increase impact? • Can you describe changes to women and youth capacity as a result of their involvement in CSSP? Monitoring and Evaluation (EQ3) • Was Pact’s monitoring and evaluation (M&E) reporting on their project and grantee activities sufficient for you to understand project outputs, outcomes and results? • Can you describe how Pact utilized M&E data to adjust project activities over the life of the project? • Can you describe any positive outcomes of such course corrections? • What were the challenges in using M&E data to inform project decisions? What could be done differently? • How did Pact utilize analytical products – reports, assessments and briefs – to improve the CSSP project? • Did USAID utilize M&E data to inform decisions related to CSSP? Utilization • Is there any specific information that you would like the ET to know that we did not ask about? 41 USAID/Zimbabwe CSSP KIIs (Est. Time ~60 min) Pact/CSSP Staff Introduction and Project Scope • Describe your role in CSSP. How long have you been with the project? Appropriateness and effectiveness of CSSP program design (EQ1) • Can you describe the goal of the CSSP project to date? • How has Pact’s integrated approach of technical and financial support affected CSSP’s outcomes? (EQ1a) • Have there been any changes to the scope or project activities since you have been involved with CSSP? If yes, please describe any and all changes, including contextual factors, intervening circumstances or other reasons that may have affected the project. • What affect did the programmatic shift after the elections have on CSSP outcomes? (EQ1b) Grants Program (EQ1a) • Can you describe the grant component of CSSP? • How did the different types of financial support – rapid response grants and institutional support grants – help CSSP achieve its goal? (EQ1a) • What were the challenges to the grant program? What could be done differently? Achievement of Intended Results (EQ2a) • Based on your interaction with CSSP, please describe your assessment of the project to date. Can you describe the technical support that Pact provided to CSOs that took part in CSSP? o How could that support be further strengthened or expanded? • What types of requests for support/training have you received from participating CSO/CBOs? • Did CSSP increase the institutional and technical capacity of partner CSO/CBOs? (EQ2c) • If yes, which capacities did CSO/CBOs develop as a result of CSSP? • If yes, do you believe that increased capacity is sustainable? Why, or why not? • In what areas do CSO/CBOs need additional support? • What project outcomes have you observed to date? o Have you observed any unintended/unexpected results (positive or negative)? • What do you think are the project’s high points/successes? • Has the democratic needle moved forward as a result of the CSSP project? • If yes, can you provide examples? • If no, why not? • Are there any barriers, in your estimation, standing in the way of the project achieving its stated objectives? • Has the project encountered any challenges? If so, please describe (program design challenges, implementation challenges, or other). o If challenges did occur, how did they affect the project? o How did the project address and overcome the challenges? • What do you think the project should do more of? • What do you think the project should do less of? • What recommendations do you have for the project? Gender and Youth (EQ2b) • What are the barriers to the meaningful participation of women and youth in your areas of work (in terms of locations and issues)? 42 • Can you describe any observations about women and youth participation in the CSSP project? • Did Pact integrate women and youth into program design and implementation in a way that produced meaningful participation by both? Alternatively, could Pact have done something differently to increase impact? • Can you describe changes to women and youth capacity as a result of their involvement in CSSP? Monitoring and Evaluation (EQ3a) • Please describe what support you provided to grantees regarding M&E. • What M&E products and systems were put in place through the CSSP project? • What was the quality of grantees’ reporting? • Can you describe how Pact utilized M&E data to make adjustments to project activities over the life of the project? • Can you describe any positive outcomes of such course corrections? • What were the challenges in using M&E data to inform project decisions? What could be done differently? • How did Pact develop the analytical products – reports, assessments, and briefs? • How were these products used to inform program design and implementation? • What were some of the challenges to using M&E and the analytical products? (EQ3b) Contextual Issues • Are there any contextual factors, such as Zimbabwe’s political environment, that have affected this project? (EQ1) Utilization • Is there any specific information that you would like the ET to collect that will be helpful for the remainder of your implementation period? 43 USAID/Zimbabwe CSSP KIIs (Est. Time ~60 min) CSO/CBO Grantees Appropriateness and effectiveness of program design (EQ1) Grants Program (EQ1) • Please briefly describe your project that is being funded through the CSSP grant. • Please describe the assistance CSSP provided through the grant you received. • Please describe your interaction with Pact in terms of grant program implementation/grant management. • What were the challenges to the grant program? What could be done differently? Capacity Building Support (EQ1) • How many trainings have you participated in with CSSP? List as many as you can. • Did you find the information/relationships gained through your participation in the project useful for your work as a CSO? If yes, please describe in more detail. • How have you used the knowledge you gained from your participation in CSSP trainings/partnership in your organization’s work? • Since participation with CSSP, what do you see as the most significant improvement within your organization? • If additional training or support could be provided, what type of training/support would you find useful? Interviewer note: Try to dig past the obvious financial support request; try to urge answers that fit into scope of the CSSP project) Achievement of Intended Results (EQ2) • Can you describe the technical support that you received from Pact as a part of the CSSP project? o How could that support be further strengthened or expanded? • If the CSSP project would end tomorrow, would your organization be able to continue its work? Please describe why or why not. • You received CSSP support to engage within your community on specific issues. Can you describe how these activities have contributed to making your community more participatory and effective in working with local decision-makers? • If yes, can you provide examples? • What do you think are the high points/successes of your involvement with CSSP? • Has the project encountered any challenges? If so, please describe (program design challenges, implementation challenges, or other). o If challenges did occur, how did they affect the project? o How did the project address and overcome the challenges? • What do you think the project should do more of? • What do you think the project should do less of? • Do you have any recommendations for the project going forward? Women and Youth (EQ2) • What are the barriers to the meaningful participation of women and youth in your areas of work (in terms of locations and issues)? • Have your activities included both male and female participants? How did you ensure activities were made available to both men and women equally? 44 • Have your activities included youth participants? How did you ensure activities were made available to youth? • How did Pact/CSSP help you integrate women and youth into your project? Alternatively, could you (or CSSP) have done something differently to increase impact? • Can you describe changes to women and youth capacity as a result of their involvement in the activities you conducted with CSSP support? • Interviewer note: be sure to ask men about women's participation and non-youth about youth participation. Don't ignore questions even if respondent isn't of that demographic Monitoring and Evaluation (EQ3) • Can you describe CSSP support to your organization in developing M&E plans, systems, etc.? • Can you describe whether you utilized M&E data to make adjustments to your activities? • If yes, did CSSP staff help to facilitate this process? • Can you describe any positive outcomes of such course corrections? • What were the challenges in using M&E data to inform project decisions? What could be done differently? Contextual Issues • Are there any contextual factors, such as Zimbabwe’s political environment, that have affected your ability to carry out all of your activities and reach your desired outcomes? (EQ1) Additional comments • Is there any specific information that you would like the ET to collect that we have not asked you? 45 USAID/Zimbabwe CSSP KIIs (Est. Time ~60 min) External Actors Introduction and Project Scope • What type of work is your organization doing in the civic space? • What has been your experience with CSSP? Program Design and Effectiveness (EQ1) • Can you describe the goal of the project to date, as you understand it? • Have you worked with any CSO/CBOs that took part in CSSP? • CSSP takes an integrated approach to CSO/CBO support (explain this integrated approach if necessary). Have you seen any evidentiary outcomes of this approach in your work with CSOs that may have also taken part in the CSSP project? • Do you think this is an effective alternative approach for working with civil society in Zimbabwe? How do you see the future of the civil society landscape within Zimbabwe and within your organization? Women and Youth (EQ2) • What are the barriers to the participation of women and youth in Zimbabwe? • Have you seen any change within these communities (in terms of capacity or community/civic engagement) with groups receiving CSSP support? Monitoring and Evaluation (EQ3) • Are you aware of any of the analytical products CSSP has produced – environment scans, issue briefs, etc.? o If yes, how have you seen them be used? Recommendations (EQ1b) • What recommendations do you have for USAID going forward—for future work within this space? 46 FOCUS GROUP DISCUSSIONS USAID/Zimbabwe Civil Society Strengthening Program (CSSP) Evaluation Coversheet for Focus Group Discussion (FGDs) Protocols Interview Details Key Informant Details Date of Interview: Name, Title: Time Start: Time End: Affiliation: Team: Sex: Male / Female (circle one) Interviewer(s): Location: Introduction Good morning/afternoon and thank you for taking the time to speak with us today. As mentioned during our interview request, we are working with USAID/Zimbabwe to conduct a final evaluation of the Civil Society Strengthening Program (CSSP) implemented by Pact. The evaluation is intended to provide an informed assessment of progress to date and recommend potential modifications, if needed. Our team has reviewed some background documents to get a better sense of the design and implementation of the project. However, documents can only tell us so much. We would like to speak with you today to hear about your experience, in your own words, in order to help us better understand how CSSP operates “on the ground.” Confidentiality Protocol • We will take note of the people we speak with, but for security precautions, we will not provide a list of key informants in our final evaluation report. • We may include quotes from respondents in the evaluation report, but we will not link individual names, organizations, or personally identifiable information to those quotes, unless express written consent is granted by the respondent. Should the team desire to use a particular quote, photograph, or identifiable information in the report, the evaluators will contact the respondent(s) for permission to do so. • All data gathered will be used for the sole purposes of this evaluation and will not be shared with other audiences or used for any other purpose. • Your participation in this interview is voluntary and if you do not feel comfortable answering a particular question, please let us know and we will simply go on to the next question. • Once again, thank you for taking the time to speak with us today. Do you have any questions for us before we get started? Inform interviewee we may follow-up with additional questions at the end of fieldwork. 47 Focus Group Discussion Guide Direct CSSP Beneficiaries (female cohort, male cohort, youth cohort) Time: up to 90 minutes Be sure to:  Set up the room to facilitate a participatory discussion  Introduce yourselves  Introduce SI and explain why you are conducting the focus group session  Ask the participants to do the same  Give a verbal agenda and length of the meeting  Set goals  Clarify your role as a facilitator Evaluation’s Objective To better understand what the accomplished, document any challenges and identify opportunities for program improvement. Purpose of Discussion To hear from CSSP beneficiaries about their experience in the CSSP Program. Focus Group facilitator’s guidelines for effective discussions:  Everyone is clear on the topic  Everyone participates; no one dominates the discussion - No speeches!  One person talks at a time  Comments and discussion stay on the topic  Comments should be to the whole group - no side discussions  Respect time limits  Write down unanswered questions  No divisive or confrontational language or tone  Take notes of discussion, comments and observations so that you can write report Agenda 1. Introduction (5 min)  Reaffirm points of the meeting  Welcome participants  Set 90 minute timeline  Introduce the evaluation, give verbal agenda, objectives (as stated above)  2. Setting rules (5 min) Before the participants begin dealing with issues and ideas, the participants should agree on a set of rules that define how a group will function and how the participants will interact. 48 Sample rules  Each group member has the right to participate.  The opinion of each group member is important and should be respected.  Group members should be tolerant of different ideas.  Each group member is important. 3. Tell us about your involvement with the CSSP project (20 min) (EQ1) Ask participants:  What activities did you participate in through [local organization] and CSSP?  How long was your involvement with this activity?  Did you receive any training?  If yes, what types of training and what did you learn?  Did you find these trainings helpful? Facilitator note: explore with group why trainings were helpful or not. Try to get concrete examples of how they utilized the information or skills they learned. 4. Tell us about the changes you saw as a result of participating in the CSSP project (20 min) (EQ2a) Ask participants:  What were the results of the activities that you took part in?  What changes did you see in your community as a result of these activities?  Have these activities carried on even after the CSSP project ended?  If not, what type of additional support is needed to make these activities continue? Facilitator note: Try to explore beyond requests for additional financial support. 5. Gender and Youth Participation (20 min) (EQ2b) Ask participants (adjust wording as needed for if participants are male or female, youth or non-youth):  What efforts does CSSP take to promote women’s involvement your organization’s activities?  Did you feel this was a good approach? If no, how might they have done this better?  What efforts does CSSP take to promote youth involvement your organization’s activities?  Did you feel this was a good approach? If no, how might they have done this better?  What could CSSP do differently to encourage more women and youth to become involved your organization’s work? 6. Contextual Factors (10 min) Ask participants:  Are there any contextual factors, such as Zimbabwe’s political environment, that have affected this project? 7. Conclusion / Close (5 min)  Summarize the session  Ask if everyone agrees to what was discussed, offer any final comments and thank all those who participated. 49 MINI-SURVEY USAID/Zimbabwe Civil Society Strengthening Program (CSSP) Evaluation Coversheet for Mini-Survey Protocol Introduction Hello. I am calling from Social Impact, an independent research company based in Washington, DC. We are working on behalf of USAID/Zimbabwe to collect data about USAID’s Civil Society Strengthening Program (CSSP) implemented by Pact. Our evaluation is intended to provide an informed assessment of progress to date and recommend potential modifications, if needed. Pact provided your name as a person who is informed on this topic. I have a few questions for you. Could I speak with you for no more than twenty (20) minutes? Confidentiality Protocol • We will take note of the people we speak with, but for security precautions, we will not provide a list of survey respondents in our final evaluation report. • All data gathered will be used for the sole purposes of this evaluation and will not be shared with other audiences or used for any other purpose. • Your participation in this survey is voluntary and if you do not feel comfortable answering a particular question, please let us know and we will simply go on to the next question. Once again, thank you for taking the time to speak with us today. Do you have any questions for us before we get started? 50 USAID/Zimbabwe Mini-Survey Protocol (Est. Time ~20 min) CSO/CBO Grantees Date: Position Organization Gender Locality (city/village, district) Rapid Response Grantee or Institutional Support [_] Rapid Response [_] Institutional Support 1. How many grants did you receive from Pact? 2. How long (in months) was your grant(s)? 3. What was the main activity you conducted under the award you received from CSSP? [Interview instruction read out options - Have respondent pick ONE answer.] a. Community engagement b. Social accountability c. Advocacy d. Voter registration/voter mobilization e. Community mobilization f. Dialogue with local authorities g. Awareness raising h. Media/information sharing i. Women j. Youth k. Justice/human rights l. Other __________________ m. Don’t Know 4. How do you rate your organization’s inclusion of women in the above activities? (EQ2b) [Interviewer reads out options] 1-very poor 2-poor 3-satisfactory 4-good 5-very good 5. How do you rate your organization’s inclusion of youth in the above activities? (EQ2b) [Interviewer reads out options] 1-very poor 2-poor 3-satisfactory 4-good 5-very good 6. Did you receive any trainings under CSSP? Yes / No / Don’t Know 7. [If yes to Q6], What was the quality of trainings? (EQ1a) [Interviewer reads out options?] 1-very poor 2-poor 3-satisfactory 4-good 5-very good 8. [If yes to Q6], To what extent did your organization use and apply knowledge gained in the training? (EQ1a) [Interviewer reads out options] 1-none 2-unimportant 3-moderately important 4-very important 5-extremely important 51 Interviewer note: Whenever respondents answer “to a large extent,” please request an example of use: ________________________________________________________________________________ ________________________________________________________________________________ 9. What is the most important type of support provided to your organization by CSSP/Pact? (EQ1a) [Interviewer Note: Please reads out options. Ask respondent to only choose ONE answer] n. Training o. Funding p. Advocacy to promote civil society participation in public interest topics q. Support to networking and coalitions r. Other ________________________ s. Don’t know 10. How would you rate the overall relevance of CSSP services or activities in helping your organization achieve its goals? (EQ2a) [Interviewer reads out options] 1-irrelevant 2-somewhat relevant 4-very relevant 5-extremely relevant 11. Did you use M&E data to adjust your activities? (EQ3a) Yes / No / Don’t Know 12. [If yes to Q11], How did you use your M&E data to adjust your activities? (EQ3a) [Interviewer Note: reads out option. Record only one response] a. Requested additional technical support b. Requested additional capacity building support c. Requested additional financial resources d. Combined activities e. Added/Removed activities f. Changed target beneficiaries g. Other _________________________ h. Don’t know 13. To what extent do you agree with this statement: The CSSP support has made a huge difference in terms of helping with organizational sustainability. 1= strongly disagree, 2= disagree, 3= neither agree nor disagree, 4= agree, 5= strongly agree 14. What top challenge have you faced in implementing your CSSP-funded activity? (EQ1a) 52 ANNEX D: BIBLIOGRAPHY OF DOCUMENTS REVIEWED Activity Schedules 1. IS 011 June 2013 Work Plan. Work Plan. Harare: IDAZIM, 2013. Print. 2. IS 011 November 2013 Work Plan. Work Plan. Harare: IDAZIM, 2013. Print. 3. IS 011 December 2013 Work Plan. Work Plan. Harare: IDAZIM, 2013. Print. 4. IS 011 January 2014 Work Plan. Work Plan. Harare: IDAZIM, 2014. Print. Analytical Documents 5. A Compendium of CSSP Programming Lessons 2012-2016. Rep. Harare: Pact Zimbabwe, 2016. Print. 6. Alert - No. 2015/1. 6 July 2015. Pact Zimbabwe Alert. Zimbabwe, Harare. 7. Alert - 2015/2. 15 July 2015. Dzamara Prayer Meeting. Zimbabwe, Harare. 8. Alert - No. 2015/3. 20 Oct. 2015. "Munhu-Wese-Kuna-Amai": ZANU-PF Power Contestations & Implications for Civil Society Programming in 2016 & Beyond. Zimbabwe, Harare. 9. Beyond the 2013 and 2014 Zimbabwean Deluge: Zimbabwe Scenario Analysis. Rep. Harare: Pact Zimbabwe, 2014. Print. 10. Civil Society Providing National Leadership. 25 July 2014. Brainstorming Session 3. Zimbabwe, Harare. 11. Context Analysis for 2015/16 RFA Design, Summary of Key Findings of Provincial Political Economy Analysis, January 2015. Rep. Harare: Pact Zimbabwe, 2015. Print. 12. Integrating the Informal and Formal Sectors of the Zimbabwean Economy. Issue brief. Harare: Pact Zimbabwe, Print. 13. Pact Brief: No. 2014/3. 7 Aug. 2015. Zimbabwe Civil Society's Community & Citizen Engagement Processes and Approaches: Issues for Strategic Communication. Zimbabwe, Harare. 14. Pact Brief: No. 2015/2. 15 June 2015. Zimbabwe's Informal Economy and the Dilemma of Around Street Vending: Preliminary Discussion Thoughts? Zimbabwe, Harare. 15. Pact Discussion Paper: A Case for Zimbabwean Civil Society Collaboration. Tech. Harare: Pact Zimbabwe, 2015. Print. 16. Pact Research Paper: Rethinking Democracy, Rights, and Governance Programming in Zimbabwe: An Integrated Approach. 2015/2. Harare: Pact Zimbabwe, 2016. Print. 17. Prospects for Rebuilding Vibrant Social Movements in Contemporary Zimbabwe. Discussion Paper. Harare: Pact Zimbabwe, 2015. Print. 18. Reference Group Meeting Report of Proceedings and Strategies, October 7, 2015. Rep. Harare: Pact Zimbabwe, 2015. Print. 19. Rethinking Democracy, Rights, and Governance Programming in Zimbabwe. Rep. no. 2015/2. Vol. 2. Harare: Pact Zimbabwe, 2016. Print. 20. SROI for the Civil Society Strengthening Project, A Proposal from NEF Consulting. Rep. Cape Town: NEF Consulting, 2015. Print. 21. Sythesis Across the 6 Study Provinces, Context Analysis Report, 18 December 2014. Rep., 2014. Print. 53 CD Tools 22. Capacity Solutions Platform. 2013. OPI Result. Zimbabwe. 23. Capacity Development Framework and Theory of Change. 2 Nov. 2016. PPT. 24. Copy of OCA Tool. 2 Nov. 2016. Excel. Close Out 25. Grant Closeout Checklist. Checklist. Harare: Cotrad, 2014. Print. 26. Grant Closeout Checklist. Checklist. Harare: IDAZIM, 2012. Print. 27. Grant Closeout Checklist. Checklist. Harare: IDAZIM GPA TASP, 2013. Print. 28. Grant Closeout Checklist. Checklist. Harare: MAZ, 2016. Print. 29. Maisiri, Trevor. "Initial Close-out of Pact Grant No. OS-IS-037." Letter to Nigel James Nyamutumbu. 7 Jan. 2015. MS. Zimbabwe, Harare. 30. Mangere, Tatenda Jane. "Asset Disposition Request - MAZ IS 037 Grant." Letter to T. Munsaka. 20 Apr. 2016. MS. Zimbabwe, Harare. 31. Modification No. 001. Modification. Harare: Pact Zimbabwe, 2013. Print. 32. Modification No. 001. Modification. Harare: Pact, 2014. Print. COTRAD. 33. Modification No. 003. Modification. Harare: Pact Zimbabwe, 2014. Print. 34. Modification No. 04. Modification. Harare: Pact Zimbabwe, 2016. Print. 35. Phute, Thembile. "Final Close Out of Pact Grant No. OS-IS-007." Letter to Director, IDAZIM. 27 Apr. 2016. MS. Zimbabwe, Harare. 36. Phute, Thembile. "Final Close Out of Pact Grant No. OS-IS-008." Letter to Davie Malungisa. 27 Apr. 2016. MS. Zimbabwe, Harare. 37. Phute, Thembile. "Final Close-Out of Pact Grant No. OS-IS-002." Letter to Davie Malungisa. 24 June 2016. MS. Zimbabwe, Harare. 38. Phute, Thembile. "Final Close-out of Pact Grant No. OS-IS-011." Letter to Gamuchirai Makura. 13 June 2016. MS. Zimbabwe, Harare. 39. Phute, Thembile. "Final Close Out of Pact Grant No. OS-IS-037." Letter to Nigel James Nyamutumbu. 25 Apr. 2016. MS. Zimbabwe, Harare. 40. Phute, Thembile. "Final Close-Out of Pact Grant No. OS-IS-015." Letter to D Nkomo. 13 June 2016. MS. Zimbabwe, Harare. 41. Trust, Habakkuk. Grant Closeout Checklist. Checklist. Harare: Pact Zimbabwe, 2014. Print. 42. Zimbabwe. USAID. Grant Closeout Checklist. Harare: USAID, 2013. Print. CSSP Grantee Program Documents 43. 2012 RFA Civil Society Voices Mobilized November. 19 Dec. 2012. Civil Society Strengthening Project: Small and Community Based Organizations Grants Facility. 44. Request for Applications (RFA). Mar. 2014. Civil Society Strengthening Project: Institutional Capacity Building of Small and Community Based Organizations. Zimbabwe, Harare. 45. Request for Applications (RFA). 2015. Civil Society Strengthening Project: Institutional Capacity Building of Small and Community Based Organizations. Zimbabwe, Harare 54 CSSP RFAs 46. Request for Applications (RFA). 19 Dec. 2012. Civil Society Strengthening Project: Small and Community Based Organizations Grants Facility. Zimbabwe, Harare. 47. Request for Applications (RFA). 2015. Civil Society Strengthening Project: Small and Community Based Organizations Grants Facility. Zimbabwe, Harare. 48. Request for Applications (RFA). Mar. 2014. Civil Society Strengthening Project: Small and Community Based Organizations Grants Facility. Zimbabwe, Harare. 49. Request for Applications (RFA). 2016-17. Civil Society Strengthening Project: Small and Community Based Organizations Grants Facility. Zimbabwe, Harare. Consultants 50. Board Orientation and Strategic Planning Proposal for PACT - COTRAD. Rep. Harare: Victor Mhizha-Murira, 2013. Print 51. Community Tolerance Reconciliation and Development (COTRAD). Rep. Harare: COTRAD, 2013. Print. 52. Masters, Reyhana. PAC Report Writing Training 2013. Rep. 2013. Print. 53. Mukundu, R. Partner Capacity Building on Report and Proposal Writing. Rep. Harare: 2013. Print. 54. Rashweat Makundu. Resume. Zimbabwe, Harare. 55. Reyhana Masters. Resume. Zimbabwe, Harare. 56. Scope of Work, Capacity Building Support on Board Orientation and Strategic Planning. Scope of Work. Harare: COTRAD, 2013. Print. 57. Simon Matsvai. Resume. Zimbabwe, Harare. 58. Strategic Plan for Period 1 January 2014 to 31 December 2018. Rep. Harare: COTRAD, 2013. Print. Grant Documents 59. Certification Regarding Drug-Free Workplace Requirements. 1 July 2014. Certification. Zimbabwe, Harare. 60. Compliance Certification Checklist - Step 1b. Checklist. Washington, DC: Pact, 2012. Print. 61. Ellwood, Christian. "Grant Agreement No. OS-IS-002, with IDAZIM under Contract NO. AID-613-C￾12-00001." Letter to Otto Saki. 12 Sept. 2012. MS. Zimbabwe, Harare. 62. Ellwood, Christian. "Grant Agreement No. OS-IS-007, with IDAZIM under Contract No. AID-613-C￾12-00001." Letter to Otto Saki. 27 Nov. 2012. MS. Zimbabwe, Harare. 63. Ellwood, Christian. "Modification to Grant Agreement No. OS-IS-002, with IDAZIM under Contract No. AID-613-C-12-00001." Letter to Otto Saki. 19 Aug. 2013. MS. Zimbabwe, Harare. 64. Hansen, Stig M. "Grant Agreement No. OS-IS-008, with IDAZIM under Contract No. AID-613-C-12- 00001." Letter to Otto Saki. 14 Jan. 2013. MS. Zimbabwe, Harare. 65. Hansen, Stig M. "Grant Agreement # CSSP/IS 015 to Habakkuk Trust, under Contract No. AID-613-C￾12-00001." Letter to Otto Saki. 19 Mar. 2013. MS. Zimbabwe, Harare. 66. Maisiri, Trevor. "Grant Agreement No IS 036, with Media Alliance of Zimbabwe (MAZ) under Contract No. AID-613-C-12-00001." Letter to Monique Murad, COR, USAID/Zimbabwe. 24 June 2014. MS. Zimbabwe, Harare. 67. Neg Memo - New Grant Award. Grant Award. Harare: Pact Zimbabwe, 2014. Print. 55 68. "Non-Compliance with Income Tax Act and National Social Security Authority." Letter to The Board Chair, Media Alliance of Zimbabwe. 8 Feb. 2016. MS. Zimbabwe, Harare. 69. In-kind Grant Award. Grant Award. Harare: Pact, 2012. Print. 70. Zimbabwe. USAID. Modification NO 1. Harare: Pact Zimbabwe, 2013. Print. 71. Zimbabwe. USAID. Neg Memo - New Grant Award. Harare: IDAZIM, 2012. Print. 72. Zimbabwe. USAID. Neg Memo - Modification to Grant Award. Harare: Pact Zimbabwe, 2012. Print. 73. Zimbabwe. USAID. USAID-Funded In-Kind Grant Award. Harare: Pact, 2010. Print. 74. Zimbabwe. USAID. USAID-Funded In-Kind Grant Award. Harare: Pact Zimbabwe, 2012. Print. Implementation Plans 75. Phute, Thembile. "CSSP Annual Implmentation Plan." Message to Monique Murad. 10 Feb. 2014. E-mail. 76. Zimbabwe. USAID. Civil Society Strengthening Program (CSSP). Annual Implementation Plan.: Pact, 2012. Print. 77. Zimbabwe. USAID. Civil Society Strengthening Program (CSSP). Annual Implementation Plan FY 2015. N.p.: Pact, 2015. Print. 78. Zimbabwe. USAID. Civil Society Strengthening Program (CSSP). Annual Implementation Plan FY 16. Harare: Pact, 2015. Print. Indicator Data 79. Zimbabwe. USAID. Civil Society Strengthening Program (CSSP). Copy of Pact USAID CSSP Quarterly Progress Report - Annex 2 - Indicator Data - 31 Jan 2014. Harare: Pact, 2014. Print. 80. Zimbabwe. USAID. Civil Society Strengthening Program (CSSP). AID-613-C-12-00001-CSSP Quarterly Progress Report - Annex 2- Indicator Data - 30 April 2015. Harare: Pact, 2015. Print. 81. Zimbabwe. USAID. Civil Society Strengthening Program (CSSP). AID-613-C-12-00001-CSSP Quarterly Progress Report - Annex 2 - Indicator Data - 31 January 2015. Harare: Pact, 2015. Print. 82. Zimbabwe. USAID. Civil Society Strengthening Program (CSSP). Annex 3: Table 2: CSSP Quarterly Report Indicator Results Analysis- January- March 2015. Harare: Pact, 2015. Print. 83. Zimbabwe. USAID. Civil Society Strengthening Program (CSSP). AID-613-C-12-00001-CSSP Quarterly Progress Report July - September 2015- Indicator Data. Harare: Pact, 2015. Print. 84. Zimbabwe. USAID. Civil Society Strengthening Program (CSSP). CSSP Quarterly Report Indicator Results Analysis. Harare: Pact, 2015. Print. 85. Zimbabwe. USAID. Civil Society Strengthening Program (CSSP). Indicator Data for CSSP April - June 2013. Harare: Pact, 2013. Print. 86. Zimbabwe. USAID. Civil Society Strengthening Program (CSSP). Q2 2014 Indicator Data. Harare: Pact, 2014. Print. ISP 87. Habakkuk Trust Priority Areas 2014. Harare: Habakkuk, 17 June 2014. Excel. 88. Habakkuk Trust, Re-OCA Report. Strategy. Harare: Habakkuk, 2014. Print. 89. Habakkuk Trust Strategy 2014-2017. Strategy. Habakkuk, 2014. Print. 90. Organizational Capacity Assessment & Organizational Development Roadmap. Harare: Habakkuk, 6 Mar. 2014. Excel. 56 91. Organizational Development and Effectiveness. Harare: Habakkuk, 2014. Print. 92. Organizational Capacity Assessment & Organizational Development Roadmap. Harare: Habakkuk, 15 July 2014. Excel. Pre-Award 93. Draft Report, Pact Zimbabwe Sub-grant Compliance Report. Rep. Harare: Habakkuk Trust, 2013. Print. 94. Habakkuk Trust Draft Desk Audit Report. Audit Report. Harare: Habakkuk Trust, 2014. Print. 95. Habakkuk Trust Draft Desk Audit Report. Audit Report. Harare: Habakkuk Trust, 2015. Print. 96. Habakkuk Trust Draft Desk Audit Report. Harare: Habakkuk Trust, 2015. Print. 97. Habakkuk Trust Profile. Profile. Harare: Habakkuk Trust, 2013. Print. 98. List of Board Members. List. Harare: Habakkuk Trust, 2013. Print. 99. Pre-Award Financial & Administrative Assessment. Assessment. Harare: Habakkuk Trust, 2013. Print. Proposal 100. Civic Society Strengthening Project Budget. Harare: Habakkuk, 19 Mar. 2013. Excel. 101. Grant OS-IS-008 Narrative Report for May 2013. Monthly Plan. Harare: Institute for a Democratic Alternative for Zimbabwe, 2013. Print. 102. June Monthly Plan. Monthly Plan. Harare: 2013. Print. 103. Malungisa, Davie. "Request for Direct Payment of Technical Assistance Newsletter: Request for Direct Payment to Document Support Centre." Letter to Christian Ellwood. 12 Sept. 2012. MS. Zimbabwe, Harare. 104. USAID Budget Final. Harare: IDAZIM, 16 Jan. 2013. Excel. 105. Planned Activities for FY13 Q2 (January - March 2013). Monthly Plan. Harare: IDAZIM, 2013. Print. 106. Title of Application: Enhancing and Protecting Citizen Participation Project. Rep. Harare: Habakkuk Trust, 2013. Print. 107. Zimbabwe. USAID. GPA Implementation Technical Assistance Report Program. Harare: 2012. Print. 108. Zimbabwe. USAID. Sub-grant Site Visit Worksheet. Harare: IDAZIM, 2013. Print. Quarterlies+ Annual Report 109. AID-613-C-12-00001-CSSP Quarterly Progress Report - Annex 4 - Indicator Data - 31 October 2014.: Pact, Inc., Excel. 110. AID-613-C-12-00001-CSSP Quarterly Progress Report - Annex 5 - Human Rights Fund Update 31 October 2014. Annex. Harare: P, 2014. Print. 111. Annex 1: Summary of Grant Implementation and Appraisal. Rep. Harare: Pact Zimbabwe, 2013. Print. 112. Annex 1: Table 1: CSSP Implementing CSOs, Objectives, Activities and Results (RR and IS Grants). Annex. Harare: Pact, Print. 113. Annex 3: Table 2: CSSP Quarterly Report Indicator Results Analysis - April - June 2015. Annex. Harare: Pact, 2015. Print. 114. Annex 4: ERC Summary of Community Meetings Which Were Held to Review the Electoral Amendment Bill. Annex. Harare: Pact, Print. 57 115. Annex 5: Issues Raised at ZOYPwar Meetings. Annex. Harare: Pact, Print. 116. Annex 5: Victim of Torture Fund Program Update. Annex. Harare: Pact, Print. 117. Annex 6: Human Rights Fund Update. Annex. Harare: Pact, Print. 118. Annex 7: Information Safety and Capacity Project Update. Annex. Harare: Pact, Print. 119. Annual Report, October 1, 2012 - September 30, 2013. Rep. Harare: Pact Zimbabwe, 2013. Print. 120. Annual Report October 1, 2014 - September 30, 2015. Rep. Harare: Pact, 2015. Print. 121. Annual Report October 1, 2015 - September 30, 2016. Rep. Harare: Pact, 2016. Print. 122. Annual Report July 1, 2015 - September 30, 2016. Rep. Harare: Pact, 2016. Print. 123. Community Dialogue That Focused on Women and Youth Issues. Harare: Pact, Inc., Excel. 124. Copy of CSO Contact Details 2015 - UPDATED. Harare: Pact, Inc., 2015. Excel. 125. Copy of Copy of AID-613-C-12-00001-CSSP Annual Report. Harare: Pact, Inc., 2016. Excel. 126. Copy of Copy of AID-613-C-12-00001-CSSP Annual Report - Indicator Data. Harare: Pact, Inc., 31 Oct. 2016. Excel. 127. CSSP Close-Out Report August 1, 2012 - June 30, 2016. Rep. Harare: Pact, 2016. Print. 128. CSSP PPR Results - October 2015 - September 2016. Results. Harare: Pact Zimbabwe, 2016. Print. 129. Cumulative Data. Harare: Pact, Inc., 2015. Excel. 130. Cumulative Data AID-C-12-00001-CSSP Quarterly Progress Report - Indicator Data. Harare: Pact, Inc., 31 July 2015. Excel. 131. "Empty Buckets Campaign Bears Fruits." Success Story (31 Oct. 2014): n. pag. Pact, Inc.. Print. 132. "Goromonzi Youths Plough Back to the Community." Success Story: n. pag. Print. 133. Maisiri, Trevor. "Annual Report Submission." Message to Tsungai Kokerai. 6 Nov. 2015. E-mail. 134. "MISA's Constitutional Court Victory over Defamation Law." Success Story: n. pag. Print. 135. Number of Public Forum Feature Women and Youth Speakers. Harare: Pact, Inc., Excel. 136. Number of Targeted Communities. Harare: Pact, Inc., Excel. 137. Pact Indicators. Annex. Harare: Pact, Print. 138. Pact USAID CSSP Quarterly Progress Report - Annex 2 - Indicator Data. Harare: Pact, Inc., 30 Apr. 2014. Excel. 139. Partner Location FY14. Harare: Pact, Inc., 2014. Excel. 140. Public Gatherings Shut Down. Harare: Pact, Inc., Excel. 141. Quarterly Progress Report June 15, 2012 - September 30, 2012. Rep. Harare: Pact Zimbabwe, 2012. Print. 142. Quarterly Progress Report October 1, 2012 - December 31, 2012. Rep. Harare: Pact Zimbabwe, 2012. Print. 143. Quarterly Progress Report January 1, 2013 - March 31, 2013. Rep. Harare: Pact Zimbabwe, 2013. Print. 144. Quarterly Progress Report July, 2013 - September 30, 2013. Rep. Harare: Pact Zimbabwe, 2013. Print. 145. Quarterly Progress Report October 1, 2013 - December 31, 2013. Rep. Harare: Pact, 2014. Print. 146. Quarterly Progress Report January 1, 2014 - March 31, 2014. Rep. Harare: Pact, 2014. Print. 147. Quarterly Progress Report April, 2014 - June 30, 2014. Rep. Harare: Pact Zimbabwe, 2014. Print. 148. Quarterly Report July 1, 2014 - September 30, 2014. Rep. Harare: Pact, 2014. Print. 149. Quarterly Report October 1, 2014 - December 31, 2014. Rep. Harare: Pact, 2015. Print. 150. Quarterly Report October 1, 2014 - December 31, 2014. Rep. Harare: Pact, 2016. Print. 58 151. Quarterly Report January 1, 2015 - March 31, 2015. Rep. Harare: Pact, 2015. Print 152. Quarterly Report April 1, 2015 - June 31, 2015. Rep. Harare: Pact, 2015. Print. 153. Reference Group, Civil Society Strengthening Program (CSSP), Terms of Reference. Reference Group. Harare: Pact Zimbabwe, Print. 154. Rethinking Democracy, Rights, and Governance Programming in Zimbabwe: An Integrated Approach. Research Paper. 2015 ed. Vol. 2. Harare: Pact, 2016. Print. 155. Youth Dialogue Clubs That Met with Elected Officials. Harare: Pact, Inc., Excel. 156. Zimbabwe. USAID. Transition Country Development Cooperation Strategy. Harare: 2014. Print 157. Zimbabwe. USAID. Zimbabwe Democracy and Governance Assessment. Washington, DC: Management Systems International, 2012. Print. Reports 158. Data Collection Reporting Form. Rep. Harare: Habakkuk Trust, 2013. Print. 159. Data Collection Reporting Form. Rep. Harare: Habakkuk Trust, 2014. Print. 160. Final Report, Pact Zimbabwe Sub-grant Compliance Report, 10 July 2013. Rep. Harare: Pact Zimbabwe, 2013. Print. 161. ISO15 April Report. Rep. Harare: Habakkuk Trust, 2013. Print. 162. ISO15 August Report. Rep. Harare: Habakkuk Trust, 2013. Print. 163. ISO15 December Report. Rep. Harare: Habakkuk Trust, 2013. Print. 164. ISO15 July Report. Rep. Harare: Habakkuk Trust, 2013. Print. 165. ISO15 June Report. Rep. Harare: Habakkuk Trust, 2013. Print. 166. ISO15 May Report. Rep. Harare: Habakkuk Trust, 2013. Print. 167. ISO15 November Report. Rep. Harare: Habakkuk Trust, 2013. Print. 168. ISO15 October Report. Rep. Harare: Habakkuk Trust, 2013. Print. 169. ISO15 September Report. Rep. Harare: Habakkuk Trust, 2013. Print. 170. ISO15 February Report. Rep. Harare: Habakkuk Trust, 2014. Print. 171. "Nkayi Action Team Meets Traditional Leaders." Habakkuk Times (6 Mar. 2014): n. pag. Print. 172. Revised Planned Activities for May. Monthly Plan. Harare: Habakkuk Trust, 2013. Print. 173. Technical Assistance Support Program Narrative Report: 28 February 2013. Rep. Harare: IDAZIM, 2013. Print. 174. TASP Narrative Report: Final Narrative Report 15 August 2012 to 30 September. Rep. Harare: IDAZIM, 2012. Print. Site Visits 175. Sub-grant Site Visit Worksheet. Work Sheet. Harare: IDAZIM, 2013. Print. 176. Sub-grant Site Visit Worksheet. Work Sheet. Harare: Pact Zimbabwe, 2013. Print. 59 ANNEX E: KEY INFORMANTS Organization /Affiliation Method Location Sex Youth M F Epworth Residents Development Association (ERDA) KII Harare 1 Reference Group Member Researcher Reference Group Member Harare 1 Combined Harare Residents Association (CHRA) KII Harare 1 PACT/CSSP Team KII (group) Harare 5 6 Election Resource Centre KII Harare 1 Self Help Development Foundation-Training and Advisory Services (SHDF-TAS) KII Harare 1 Youth Empowerment and Transformation Trust (YETT) KII Harare 1 USAID Reference Group Member KII Harare 1 Zimbabwe Coalition of Debt and Development (ZIMCODD) KII Harare Local Research Consultant KII Harare 1 Evangelical Fellowship of Zimbabwe KII Harare 1 Media Alliance of Zimbabwe KII Harare 1 Member of the USAID Reference Group KII Harare 1 African Self Help Assistance Program (ASAP) KII Mutare 1 African Self Help Assistance Program (ASAP) FGD Hauna (Mutare) 13 13 10 COTRAD FGD Zaka (Masvingo) 14 18 12 COTRAD KII Masvingo 1 TellZimbabwe KII Masvingo 1 Masakhaneni KII (group) Bulawayo 3 1 Bulawayo Progressive Residents Association (BPRA) KII Bulawayo 1 Rural Communities Empowerment Trust (RUCET) KII Bulawayo 1 Emthonjeni Women’s Forum KII Bulawayo 2 Bulawayo Progressive Residents Association (BPRA) FGD Bulawayo 19 13 15 Victory Siyanqoba KII Bulawayo 1 Radio Dialogue KII Bulawayo 1 National Youth Development Trust (NYDT) KII Bulawayo 1 Habbakkuk Trust KII Bulawayo 1 1 Silveira House KII Harare 1 Institute for Young Women Development (IYWD) KII Bindura 1 Institute for Young Women Development (IYWD) FGD Bindura 4 22 15 60 Voluntary Media Council of Zimbabwe KII Harare 1 USAID/Zimbabwe KII Harare 1 2 ZACRAS KII Harare 1 Wadzanai Trust FGD Domboshava (Harare) Former USAID Staff KII Harare 1 61 ANNEX F: MINI-SURVEY DATA Mini-Survey Findings (a) Grantees’ Perceptions of Inclusion of Women and Youth USAID’s Civil Society Strengthening Program (CSSP) grantees were asked to rate their organization’s inclusion of women and youths under the support they got from Pact. A large majority of CSSP beneficiaries (84%) rated their organizations’ inclusion of women in their activities as either ‘good’ or ‘very good’ while only 3% felt that it was ‘poor’(Figure 1). On the other hand, almost eight out of 10 CSSP grantees (78%) rated their organizations’ inclusion of youths in their activities as either “good” or “very good” while 16% felt that it was ‘satisfactory’ and 5% felt that it was ‘poor’ (Figure 1). Figure 1: Grantees’ perceptions of inclusion of women and youth in the CSSP activities Respondents were asked: • How do you rate your organization’s inclusion of women in the above activities? • How do you rate your organization’s inclusion of youths in the above activities? (b) Grantees’ Perceptions of Training Under the CSSP Grantees were also asked if they had received any training under CSSP support. A majority of them (68%) responded in the affirmative while almost a third (32%) said they had not received any form of training under the program (Figure 2). Asked to further rate the quality of the training that they received under the CSSP; more than three out of five (63%) described the quality of the trainings as ‘good’ or ‘very good’ while 5% described them as ‘satisfactory’(Figure 3). A third (32%) of the grantees could not answer that question since they had indicated that they had not received any form of training under the CSSP awards. Grantees were also asked if they had used or applied the knowledge that they had acquired through training. A majority, (68%) felt that their organizations had used and applied knowledge gained in the training to a ‘large’ or ‘very large extent’ (Figure 4). 84% 78% 13% 16% 3% 5% Inclusion of Women Inclusion of Youth Very Good - Good Satisfactory Poor - Very Poor 62 Figure 2: Recipients of training under the CSSP Respondents were asked: Did you receive any training under CSSP? Figure 3: Grantees’ ratings of trainings received under the CSSP Respondents were asked: How do you describe the quality of the trainings? 68% 32% Recipients of Training under CSSP Yes No 5% 32% 63% Satisfactory Not Applicable Good/Very Good % of respondents in mini-survey 63 Figure 4: Extent to which grantees have applied knowledge from CSSP related trainings Respondents were asked: To what extent did your organization use and apply knowledge gained in the training? (c) Grantees’ Perceptions of Relevance of the CSSP Grantees were asked to rate the overall relevance of the CSSP services or activities in terms of helping their organizations to achieve their goals. The mini-survey data supports findings from the Key Informant Interviews (KIIs) and documentary review in which grantees described the CSSP as relevant. An overwhelming majority (95%) of the CSSP grantees reported that CSSP activities were at least very relevant, while 3% reported the CSSP services or activities were irrelevant in terms of advancing their organizational goals. Figure 5: Grantees’ perceptions of relevance of CSSP activities Respondents were asked: How would you rate the overall relevance of CSSP services or activities in helping your organization achieve its goals 68% 32% Very Large/Large Extent Not Applicable % of respondents in mini-survey 3% 3% 95% Irrelevant Somewhat relevant Very/Extremely relevant % of respondents in mini-survey 64 (d) Grantees’ Use of Monitoring and Evaluation (M&E) Data The mini-survey data gives credence to claims by the KIIs that the CSSP program did much in terms of offering trainings to grantees with a huge majority of them (89%) compared to only 11% claiming to have used M&E to adjust their activities (Figure 7). A plurality (22%) of the mini-survey respondents said they had used the M&E data to add or remove activities, 19% had used M&E data to combine activities while a similar proportion had used it to request for additional capacity building support. A significant proportion (14%) had requested additional financial resources while a tenth (11%) did not respond to the question because they had indicated that they had not used any M&E data during the project implementation (Figure 8). Figure 7: Use of M&E data by CSSP grantees Respondents were asked: Did you use M&E data to adjust your activities? 89% 11% Yes No 65 Figure 8: Use of M&E data by CSSP grantees Respondents were asked: How did you use M&E data to adjust your activities? (e) Grantees’ Perceptions of CSSP Support Received under the CSSP A majority of the grantees concurred that the most important type of support that they have received under the CSSP support is funding (63%), while 21% said it was training, and 13% claimed that they benefitted most in terms of support on advocacy to promote civil society participation in public interest topics. Only 3% claimed that they got support to nudge networking and coalitions (Figure 9). Grantees were also presented with a statement on the extent to which they agree with the statement that the CSSP support that they received has a made a huge difference in terms of organizational sustainability. A huge majority (81%) agreed or strongly agreed that the CSSP support has made a huge difference on that aspect while 8% “disagreed’ or “strongly disagreed” with the statement (Figure 10). Figure 10: Most important support received by grantees under the CSSP Respondents were asked: What is the most important type of support provided to your organization by CSSP/Pact? 3% 3% 11% 11% 14% 19% 19% 22% Don’t know Requested additional technical support N/A Changed targeted beneficiaries Requested additional financial resources Requested additional capacity building support Combined activities Added/Removed activities 63% 21% 13% 3% Funding Training Advocacy to promote civil society participation in public interest topics Support to networking and coalitions 66 Figure 9: Grantees’ perceptions of CSSP support and effect on sustainability | 2016 Respondents were asked: To what extent do you agree with this statement: The CSSP support has made a huge difference in terms of helping with organizational sustainability. 8% 11% 81% Agree/Strongly agree Neither agree nor disagree Disagree/Strongly disagree 67 ANNEX G: DISCLOSURE OF ANY CONFLICTS OF INTEREST Name Kourtney Pompi Title Team Leader Organization Broderick Haight Consulting Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) USAID MOBIS Zimbabwe CSSP Performance evaluation I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1.Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2.Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3.Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4.Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5.Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6.Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. N/A I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 10/18/2016 68 Name Jack Zaba Title Senior DRG Specialist Organization Social Impact, Inc. Independent Consultant Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) USAID MOBIS Zimbabwe CSSP Performance evaluation I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 7.Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 8.Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 9.Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 10. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 11. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 12. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. N/A I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 10/18/2016 69 Name Cleophas Tavaya Title Mid-level M&E Specialist Organization Social Impact, Inc. Independent Consultant Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) USAID MOBIS Zimbabwe CSSP Performance evaluation I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 13. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 14. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 15. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 16. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 17. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 18. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. N/A I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 11/16/2016 70 Name Stephen Ndoma Title Research Specialist and Logistician Organization Social Impact, Inc. Independent Consultant Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) USAID MOBIS Zimbabwe CSSP Performance evaluation I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 19. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 20. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 21. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 22. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 23. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 24. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. N/A I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 11/16/2016 71 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523