GENDER EQUITY PROGRAM FINAL EVALUATION REPORT NOVEMBER 13, 2016 This publication was produced for review by the United States Agency for International Development. It was prepared by Judy Benjamin, Rehana Shaikh, Zehra Yasmin Zaidi, Zil-e-Huma, and Samar Nadeem, Management Systems International, A Tetra Tech Company. GENDER EQUITY PROGRAM FINAL EVALUATION REPORT Contracted under Order No. AID-391-C-15-00004 Performance Management Support Contract DISCLAIMER This report is made possible by the support of the American people through the United States Agency for International Development (USAID). The contents are the sole responsibility of Management Systems International and do not necessarily reflect the views of USAID or the United States Government. Gender Equity Program: Final Evaluation Report iii ACKNOWLEDGMENTS The evaluation team wishes to acknowledge and thank everyone who generously gave their time to participate in this final evaluation of the Gender Equity Program. The evaluation team greatly appreciates your willingness to contribute to the evaluation. We especially thank the implementing partners and sub-grantees for sharing their views and experiences, and all the people who kindly allowed us to interview them and ask endless questions. We are grateful for the support and cooperation of USAID/Pakistan staff. The team appreciates the tireless efforts of the MSI/PERFORM data analyst team, Muhammad Danish in particular, who patiently introduced us to MAXQDA software. Omaira Chaudhry deserves special thanks for her contributions as the evaluation assignment manager. As always, Omaira performed her job to the highest standards and always with a smile. Gender Equity Program: Final Evaluation Report iv CONTENTS Acknowledgments.............................................................................................................iii Acronyms...........................................................................................................................vi Project Summary .............................................................................................................. 1 Executive Summary.......................................................................................................... 3 Key Findings and Conclusions.......................................................................................................... 3 Key Recommendations....................................................................................................................... 4 Evaluation Purpose and Questions.................................................................................. 5 Evaluation Questions.......................................................................................................................... 5 Project Background .......................................................................................................... 5 Theory of Change and Intended Results........................................................................................ 6 Evaluation Methods and Limitations............................................................................... 7 Data Collection.................................................................................................................................... 7 Data Analysis......................................................................................................................................... 9 Data Limitations................................................................................................................................... 9 Findings ............................................................................................................................ 10 Findings for Evaluation Question 1................................................................................................10 Findings for Evaluation Question 2................................................................................................20 Findings for Evaluation Question 3................................................................................................26 Overarching Conclusions................................................................................................ 36 Recommendations .......................................................................................................... 37 Lessons Learned.............................................................................................................. 38 Annexes............................................................................................................................ 40 Annex 1: Evaluation Statement of Work.....................................................................................40 Annex 2: Assignment Work Plan...................................................................................................51 Annex 3: Summary of GEP Objectives and Thematic Areas...................................................69 Annex 4: Data Collection Detail....................................................................................................71 Annex 5: Data Collection Instruments.........................................................................................84 Annex 6: List of Project Documents Reviewed .......................................................................119 Annex 7: Detailed Responses to Hypothesis Statements......................................................120 Annex 8: Current Literature on Women’s Access to Justice; Gender-Based Violence (GBV); Women, Peace, and Security (WPS); and Child, Early, and Forced Marriage (CEFM) in Pakistan....................................................................126 Annex 9: Donor Activities on Gender (Reflections from the Field Work)......................130 Annex 10: Conflict of Interest Statements................................................................................135 Gender Equity Program: Final Evaluation Report v List of Tables Table 1: Project Summary................................................................................................................................. 1 Table 2: Summary of Interviews...................................................................................................................... 8 Table 3: Advantages and Disadvantages of Grant Characteristics........................................................22 Table 4: Number of Grants by Objective and Thematic Area ..............................................................69 Table 5: Data Collection Summary by Respondent Type, Location, and Method............................71 Table 6: Sampled Grants by Objective and Theme..................................................................................72 Table 7: Sub-grantees Interviewed ...............................................................................................................76 Table 8: Stakeholders Interviewed ...............................................................................................................82 List of Figures Figure 1: Location of Project Grantees by Cycle........................................................................................ 2 Figure 2: Sub-grantees’ Perceptions of Capacity Building .......................................................................13 Figure 3: HUB Seamless Service Model.......................................................................................................16 Figure 4: Sub-Grantees’ Perceptions of Constraints................................................................................17 Gender Equity Program: Final Evaluation Report vi ACRONYMS AF Aurat Publication and Information Service Foundation AOR Agreement Officer’s Representative AWP Assignment Work Plan CAO Civic Advocacy Organization CBO Community-Based Organization CEFM Child, Early, and Forced Marriage CNIC Computerized National Identity Card COR Contracting Officer’s Representative CSC Civil Society Coalition CSO Civil Society Organization CSW Commission on Status of Women DO Development Objective FATA Federally Administered Tribal Areas FIR First Information Report FOG Fixed Obligation Grant GBV Gender-Based Violence GE Gender Expert GEP Gender Equity Program GI Group Interview (used only in tables and data collection instruments) GIZ Deutsche Gesellschaft für Internationale Zusammenarbeit (German Society for International Cooperation) GO Government Official GRAP Gender Reform Action Plan GST General Sales Tax GUCA Grants-Under-Cooperative Agreement HBW Home-Based Worker ICT Information and Communications Technology IDP Internally Displaced Person IEC Information, Education, and Communication IP Implementing Partner IT Information Technology KFW Kreditanstalt Für Wiederaufbau (German Development Bank) Gender Equity Program: Final Evaluation Report vii KII Key Informant Interview (used only in tables and data collection instruments) KP Khyber Pakhtunkhwa M&E Monitoring and Evaluation MSI Management Systems International NAF National Advisory Forum NCSW National Commission on the Status of Women NGO Non-Governmental Organization OAPA Office for Afghanistan and Pakistan Affairs (USAID) OSG Office of Stabilization and Governance (USAID) P&D Planning and Development PERFORM Performance Management Support Contract PGCN Pakistan Gender Coalition Network SDG Sustainable Development Goal (United Nations) SG Sub-grantee SMEs Small and Medium Enterprises SOW Scope of Work SRAP Special Representative for Afghanistan and Pakistan (U.S. State Department) TAF The Asia Foundation UN United Nations UNDP United Nations Development Program UNFPA United Nations Population Fund UNICEF United Nations International Children’s Emergency Fund USAID United States Agency for International Development WDD Women Development Department WPS Women, Peace, and Security Gender Equity Program: Final Evaluation Report 1 PROJECT SUMMARY Table 1 summarizes basic information about the Gender Equity Program (GEP). TABLE 1: PROJECT SUMMARY Title/Field Project/Activity Information Contract/agreement numbers 391-A-00-10-01162-00 Start date August 18, 2010 Completion date August 15, 2015 – Extended through August 2017 Location Nationwide Implementing partner(s) Prime: Aurat Publication and Information Service Foundation (AF) Subcontractor: The Asia Foundation (TAF) USAID/Pakistan Mission Strategic Framework objectives addressed DO3: Increasing Stability in Focused Areas Gender Equity Program: Final Evaluation Report 2 The map in Figure 1 illustrates the locations of the GEP grants in each grant cycle. FIGURE 1: LOCATION OF PROJECT GRANTEES BY CYCLE Gender Equity Program: Final Evaluation Report 3 EXECUTIVE SUMMARY The United States Agency for International Development (USAID) Mission in Pakistan’s Gender Equity Program (GEP) is designed to positively affect behavior and attitudes towards women and contribute to greater gender equity. Its specific objectives focus on expanding women’s access to justice and human rights, increasing women’s empowerment, combating gender-based violence (GBV), and strengthening the capacity of Pakistani organizations that advocate for gender equity. GEP is a grant program that uses a local implementing partner, the Aurat Foundation (AF), to disburse grants to non-governmental (NGO) and other organizations addressing gender issues under a grants-under-cooperative agreement (GUCA) authorization. The evaluation addressed three questions related to the design and implementation of GEP. 1. What are the key lessons learned (successes, challenges, opportunities, and weaknesses) from implementation of GEP that could inform the design of a follow-on project? 2. To what extent, and how, did the grants-under-cooperative agreement mechanism affect implementation and results? 3. What are the areas of greatest need, priority, or opportunity for new gender programming going forward? Answers to these questions will help the Mission design future gender-focused programming. Key Findings and Conclusions GEP is an effective program across most dimensions measured in this evaluation. 1. Key lessons learned from the implementation of GEP that could inform the design of a follow-on program – GEP’s focus on raising awareness of women’s social, economic, legal, and political rights, and of women’s issues more broadly, effectively underpinned almost all areas of GEP programming. Complementary work supporting women’s access to justice, by strengthening legislation and the capacity of justice institutions, contributed to the effectiveness of GEPs work on GBV. The ability to leverage the reputation, experience, local connections, and sensitivities of a capable local implementing partner contributed to GEP’s success, especially in the legislative and policy arenas. The size and duration of GEP; its exclusive focus on gender issues; and its ability, enabled by the GUCA approach, to disburse grants to a variety of small NGOs spread across Pakistan strengthened organizations working on gender issues and created pathways and space for women’s empowerment in the economic and social spheres. However, respondents noted that a smaller geographic reach may have resulted in a larger and more sustainable impact. The findings suggest that GEP’s success in empowering women stemmed from raising awareness of women’s rights, enhancing their ability to exercise their rights, and expanding their economic opportunities. Activities to empower home-based workers and facilitate internship opportunities for women in information and communications technology (ICT) and other areas were particularly effective. 2. The grants-under-cooperative agreement (GUCA) model’s effect on implementation and results – The GUCA model allowed USAID/Pakistan, through AF, to disburse grants to many small NGOs, and other organizations, that may not otherwise have Gender Equity Program: Final Evaluation Report 4 obtained funding for gender projects. The grants strengthened many of the recipient organizations, and their organizational and geographic spread created multiple entry points for addressing gender issues. Although GUCA was an appropriate and effective approach, AF sometimes struggled to disburse milestone payments on time, showed some rigidity in managing grantees, and created cumbersome grant management processes that frustrated sub-grantees, delayed implementation, and created financial hardships for small organizations. 3. Areas of greatest need, priority, and opportunity for new gender programming – Eighty-two percent of key informants and others identified access to justice as a key priority for future programming and felt that programming should include a strong policy and legislation component. Seventy percent identified women’s economic empowerment as another priority need for addressing women’s condition and position within society and the home. Forty percent of respondents noted that GBV is a critical area for future programming since it represents a major impediment to women’s being able to realize their rights and is directly linked to women’s empowerment. Twenty-nine percent of respondents believed that promoting women’s leadership is a need and priority at all levels, but particularly for grass-roots women in local government. Respondents also identified ensuring women’s representation in decision-making in government as a key priority area for future gender programming. Key Recommendations Based on the findings and conclusions, the evaluation team recommends the following: 1. Future USAID/Pakistan gender programming should continue to focus on four programmatic areas of GEP: access to justice, women’s economic empowerment, ensuring women’s representation in decision-making in government, and GBV prevention and response. Additionally, it should consider emphasizing policy and legislative advocacy for promoting gender-responsive socioeconomic change as a specific program objective. In light of the 18th Amendment, these activities should target national and provincial legislatures, ministries, and departments. 2. In future similar programming, AF or other stakeholders should implement an integrated approach to access to justice programming that addresses both demand and supply side interventions (communities as well as law enforcement agencies). 3. GUCA is a responsive, flexible model, and USAID should consider it for future similar programming. As it did with AF, it must account for the capacity of the implementing partner to design and manage the sub-grant process and provide capacity building or other support if necessary. A flexibly implemented fixed obligation grant (FOG) will be a more suitable sub-grant mechanism than a simplified grant. 4. USAID should review branding requirements in light of security risks and the increasing resistance in Pakistan to gender programming. Consider waiving branding requirements if necessary, training grantees in branding requirements, and pre-approving designs for banners and other materials to reduce delays in implementation. 5. Future USAID programming should address strengthening the Pakistan Gender Coalition Network (PGCN) at the provincial and sub-provincial levels and consider supporting a strong and well-resourced provincial secretariat for the PGCN. The PGCN has a strong potential for sustaining the achievements of GEP and continuing the engagement of sub-grantees, thereby creating the space and drive for more gender-responsive development. Gender Equity Program: Final Evaluation Report 5 EVALUATION PURPOSE AND QUESTIONS The USAID/Pakistan Office of Stabilization and Governance (OSG) will use the evaluation to inform decisions about future gender equity programming. Although the evaluation will address a number of issues, OSG is most interested in learning which aspects of the GEP design and implementation worked well, which did not, and why—particularly as they relate to the thematic area focus and the GUCA mechanism. Audiences for the evaluation include USAID/Pakistan OSG, the U.S. Congress, the Special Representative for Afghanistan and Pakistan (SRAP), the USAID Office for Afghanistan and Pakistan Affairs (OAPA), U.S. Government agencies in Pakistan, and other USAID Missions interested in gender equity programming. Evaluation Questions The Scope of Work (SOW) (Annex 1) posed three evaluation questions that emphasize the evaluation objective of informing future gender programming. The specific questions from the SOW are: 1. What are the key lessons learned (successes, challenges, opportunities, and weaknesses) from implementation of GEP that could inform the design of a follow-on project? 2. To what extent, and how, did the grants-under-cooperative agreement mechanism affect implementation and results? 3. What are the areas of greatest need, priority, or opportunity for new gender programming going forward? MSI/PERFORM elaborated on the questions in the Assignment Work Plan (AWP) (Annex 2) to identify the specific issues the evaluation would address to answer the evaluation questions. The elaborated questions focus broadly on the relevance and appropriateness of the project design, the project’s effectiveness relative to achieving stated objectives, the likely sustainability of project results, and the extent to which the project’s design and implementation addressed gender issues. PROJECT BACKGROUND USAID/Pakistan awarded the cooperative agreement for the five-year Gender Equity Program (GEP) to AF in 2010. The program supports the Government of Pakistan’s gender policies, including the National Plan of Action for Women, National Policy for Empowerment and Development of Women, and Gender Reform Action Plan (GRAP). GEP was originally slated to end in August 2015 but was extended until August 2017. AF provided a subcontract to TAF to manage sub-grants to local organizations. In 2013, USAID conducted an assessment of GEP and determined that AF had acquired the capacity necessary to manage the grants. It recommended that AF assume the grant making responsibilities and gradually phase out TAF. USAID and AF modified the cooperative agreement in preparation for the transition, but a delay in receiving FY 2014 funding necessitated extending TAF’s subcontract through May 2016 to complete its remaining tasks. Gender Equity Program: Final Evaluation Report 6 GEP is a grants program that provides grants to local organizations1 under a GUCA authorization from USAID/Pakistan. The grants support work on gender issues that addresses thematic areas determined by GEP and USAID. GEP conducts at least three grant cycles each year and awards grants to address 23 key thematic areas, 15 of which USAID identified as priority themes for the evaluation. Annex 3 summarizes the thematic areas associated with each objective and the number of grants GEP awarded in each thematic area as of November 2016. Sub-grants fall into two distinct categories:  Non-competitive grants are provided to key government departments and institutions working on women’s empowerment issues and/or policy research. These grants are designed to enlist public-private partnerships to encourage cooperative approaches to work with public institutions at the federal, provincial, and district levels.  Competitive grants are awarded to NGOs on a biannual basis. The majority of GEP sub￾grants are competitive. The grants fall into three tiers: Tier 1: less than $75,000; Tier 2: $75,000 to $150,000; and Tier 3: Over $150,000. This structure is a modification of the original four-tier design with Tier 1 below $25,000 and Tier 4 up to a maximum of $500,000. GBV prevention and response is a major focus of GEP. In addition to GBV awareness raising and prevention campaigns, GEP also approaches GBV by awarding “seamless service” grants that support provision of GBV-related services to survivors of violence. The services include helplines, police and court assistance, counseling, and rehabilitation support provided in a sequential manner. Implementation requires trained personnel and collaboration and coordination of institutions and processes to provide integrated assistance to GBV survivors. As part of the grant process, GEP also conducts capacity assessments and provides networking and capacity-building initiatives for all government and non-government sub-grantee organizations. In addition to providing grants, GEP also implements other activities to address specific gender issues and support organizations working on these issues. For example, GEP established a National Advisory Forum (NAF) consisting of prominent experts and leaders in gender and women’s empowerment in government, academia, civil society, and women’s organizations and established the PGCN to serve as a resource base for women at the district, provincial, and federal levels.2 GEP also set aside a gender fund which provides grants to universities, think tanks, and policy research institutions to conduct gender￾related research. GEP benefits by tapping into the research for guidance on strategies to help achieve the four objectives. Theory of Change and Intended Results GEP’s goal is to bring about positive behavioral change in Pakistan through a process of social change and governance that respects women’s rights, enables them to have greater control over their lives, and increases their access to information, resources, and institutions. GEP seeks to effect change in behaviors and attitudes towards women and in issues that affect the lives of women. GEP’s four key objectives are: 1. Enhancing gender equity by expanding women’s access to justice and human rights. 1 Eligible sub-grantees include government departments and institutions; policy think tanks; academic research and training institutions; professional and business associations; media; civic advocacy organizations (CAOs); civil society coalitions (CSCs); non-governmental organizations (NGOs); and community-based organizations (CBOs). 2 Cooperative Agreement, GEP Gender Equity Program: Final Evaluation Report 7 2. Increasing women’s empowerment by expanding knowledge of their rights in the workplace, community, and home and improving their opportunities to exercise these rights. 3. Combating gender-based violence (GBV). 4. Strengthening the capacity of Pakistani organizations that advocate for gender equity. By directly supporting activities that enhance women’s access to justice, creating an environment conducive to empowering women, reducing the incidence of GBV, supporting positive responses to GBV, and building local capacity to advocate for gender equity, GEP expects to begin effecting change in attitudes towards women and their role in society and, ultimately, to change behavior. As the theory of change suggests, GEP programming is intended to have a systemic effect on the system of social change with respect to gender equity in Pakistan that will play out over a period of time. EVALUATION METHODS AND LIMITATIONS The evaluation used a mixed methods framework consisting of individual interviews with key informants, group interviews with sub-grantee staff, and a mini-survey of all respondents that elicited information about future programming priorities and reactions to a set of hypothetical statements about GEP implementation. Data Collection The study’s analysis is based on secondary sources; quantitative and qualitative data obtained from 63 interviews, 34 key informant interviews (KIIs) and 29 group interviews (GIs) that included 153 individual respondents; and a mini-survey of 146 respondents. Table 2 summarizes the qualitative and quantitative interviews by respondent type, location, and data collection method. The evaluation team conducted interviews in Lahore, Peshawar, Karachi, Quetta, and Islamabad from May 23, 2016, through June 24, 2016, and administered the survey to 146 of the 153 individual respondents. The team interviewed a wide range of experts, stakeholders, and organizations that played various roles in GEP or had particular expertise or perspectives on GEP or on gender programming in Pakistan. These included representatives of 76 sub-grantees, 11 seamless service delivery “Hubs” for GBV survivors, and 13 donor agencies working on gender issues, 25 representatives of the two implementing partners, 5 gender experts who were not affiliated with GEP, 11 members of NAF, 7 government officials, and 5 USAID Pakistan staff members. The team worked with AF to identify a sample of 61 sub-grantees that had completed their grant-funded activities. The sample focused on the 15 priority themes identified by USAID but covered other themes as well. To manage travel costs, the team selected grantees within a practical travel distance of the cities in which they would lodge, i.e., Lahore, Karachi, Islamabad, Peshawar, and Quetta. To the extent possible within the thematic and location parameters, the team randomly selected sub-grantees for interviews. Annex 4 provides additional detail about the structure of the interviews. Individual tables summarize the data collection by respondent type and data collection method (i.e., key informant interview, group interview, and survey); interviews with sub-grantees; interviews with sub-grantees by thematic area; and interviews with stakeholders. The team developed interview instruments collaboratively during a two-week team planning meeting and pretested the instruments in the Islamabad area. The team developed separate instruments for key informant interviews with Hub grantees, gender experts, government officials, implementing partners, Gender Equity Program: Final Evaluation Report 8 sub-grantees, USAID personnel, and other donors. It developed group interview guides for Hub grantees and NAF members. Finally, it developed customized mini-survey instruments for all types of respondents, customized to include only hypothetical statements relevant to the respondents’ role in GEP. Annex 5 contains the final instruments. Secondary data sources consisted of project documents, reports, and data, including the cooperative agreement, the midterm evaluation report, the Office of the Inspector General’s performance audit of GEP, GEP’s grants database, GEP annual and quarterly reports, and GEP studies. The team also reviewed relevant documents and literature from other sources such as the World Bank, Asian Development Bank, UNDP, UNFPA, and UNICEF, as well as USAID’s guidelines and tools. Annex 6 contains a complete list of documents the evaluation team reviewed. TABLE 2: SUMMARY OF INTERVIEWS Respondent Type Locations of Interviews Data Collection Method Number of Interviews Number of Respondents Sub-grantee Peshawar, Karachi, Quetta, Lahore, and Islamabad KII 6 6 GI 17 70 Survey 23 71 Provincial and federal government officials Peshawar, Karachi, Quetta, Lahore, and Islamabad KII 4 4 GI 1 3 Survey 5 7 Donors Islamabad KII 8 8 GI 2 5 Survey 10 13 Gender experts Islamabad, Lahore, Peshawar, Karachi, and Quetta KII 5 5 Survey 5 5 Implementing partners (AF and TAF) Islamabad, Lahore, Peshawar, Karachi, and Quetta KII 8 8 GI 5 17 Survey 13 23 Hub for GBV survivors Lahore and Karachi GI 2 11 Survey 2 11 National Advisory Forum (NAF) Islamabad GI 1 11 Survey 1 11 USAID/Pakistan staff Islamabad and Lahore (by telephone) KII 3 3 GI 1 2 Totals KII 63 153 GI Survey 63 146 USAID also asked that the evaluation team compile a list of current research on Pakistani women’s access to justice; GBV; women, peace, and security (WPS); and child, early, and forced marriage (CEFM). Annex 8 contains this list. Gender Equity Program: Final Evaluation Report 9 Data Analysis The evaluation used a mixed methods approach, applying primarily qualitative research techniques. Data analysis employed content and thematic analysis based on deductive and inductive coding. Prior to collecting primary data, the evaluation team identified deductive codes based on the final interview guides.3 After collecting the data, the team analyzed the interview transcripts to identify the inductive codes, i.e., the themes that emerged from the interviews. The team used MAXQDA software to apply the deductive and inductive codes to the interview transcripts. The team identified themes that occurred in multiple data sources across more than one respondent category as key findings, and these became the basis for conclusions and recommendations. The report presents the evidence for each finding in terms of how often it appeared in discussions disaggregated by respondent type. 4 Since not all respondents spoke of exactly the same issues, lower numbers of respondents do not necessarily imply that a topic is less significant, but rather that fewer groups brought it up. The respondents, especially in group interviews, were not necessarily homogenous groups, especially in the case of the sub-grantees that represented a variety of sizes of organizations with different experiences, exposure, expertise, and interests. Further, not all group interview respondents repeated what had already been said even when they concurred (nodding their heads in agreement or stating that their input built on what the last speaker said). The frequencies of individual responses from group interviews may therefore understate the true frequencies. The findings sections present frequencies of coded qualitative responses, disaggregated by respondent type, in parentheses following a stated finding, e.g., 13/25 IPs. The numerator represents the number of times the theme emerged and the denominator is the total number of respondents, e.g., 34/76 SGs means 34 of 76 representatives of sub-grantees. Since the analysis tracked individual respondents within group interviews, the denominator represents individual respondents and not the number of organizations. This is most apparent in the interviews with the implementing partners, where 17 individual responses collected in 5 interviews represent 2 partners, and in the sub-grantee interviews, where 70 responses represent only 17 sub-grantee organizations. Data Limitations  Grant recipients may have biased views on the effectiveness or importance of the thematic areas on which they worked relative to other thematic areas. The team mitigated this potential bias by collecting corresponding data from a variety of types of respondents, including those who had no vested interest in GEP.  The implementing partners did not release all of the project documents the evaluation team requested, in particular some of the grant documents for the sub-grantees. Without these documents, the team could not triangulate details the sub-grantees provided about their grants against the grant documents. This created more work for the evaluation team in tracking down information but is unlikely to have substantially affected evaluation findings or conclusions. 3 Deductive codes are those the team has in mind at the start of data collection. Deductive codes often emerge from the team’s prior knowledge or experience or from a theoretical framework. 4 Respondent types included implementing partners, sub-grantees, government officials, gender experts, donors, NAF members, and Hub grantees. Gender Equity Program: Final Evaluation Report 10  Security issues limited the team’s access to some areas. To the extent that sub-grantees in these areas had different views on or experiences with gender programming or GEP than sub-grantees in areas the team was able to visit, the limitation will bias results. To counter this limitation, the team triangulated data from multiple sources.  The evaluation team planned to conduct mostly key informant interviews. In some cases, however, multiple individuals presented for the interviews. Rather than send some people away, the team elected to conduct small group interviews. Because the members of these group interviews often represented the same organization, the team does not feel that individual interactions within the group substantively affected responses relative to an individual interview.  The team tracked and coded individual responses in group interviews. This allowed the analysis to determine the number of individuals that mentioned a particular theme instead of treating a group as a single homogeneous unit of analysis. Consequently, the analysis reports frequencies and percentages of individual respondents instead of organizations (with which individual respondents were affiliated).  Much of the data came from qualitative interviews that relied largely on open-ended questions. When responses to these sorts of questions are disaggregated to a fine enough level to be useful, they often do not yield majority responses, i.e., no individual theme garners responses from a majority of respondents. In this context, the relative frequencies with which various themes emerge is more relevant than the actual frequencies. In other words, the failure of a theme to emerge from a majority of responses does not imply that it is unimportant. FINDINGS Findings for Evaluation Question 1 Evaluation question: What are the key lessons learned (successes, challenges, opportunities, and weaknesses) from implementation of GEP that could inform the design of a follow-on project? This question explores aspects of GEP’s implementation to learn about the program’s effectiveness and shortcomings with specific interest in gaining insights useful to designing future projects. To answer the question, the evaluation team asked GEP sub-grantees (SGs), implementing partners (IPs), National Advisory Forum (NAF) members, and USAID staff to share their experiences as implementers, grant managers, policy advisors, and donors. The evaluation also sought the views of outsiders such as gender experts (GEs) and government officials (GOs) who were familiar with, but not directly involved in, GEP. The evaluation team’s analysis of the findings generated conclusions on achievements, challenges, and learning around the four GEP objectives and the priority thematic areas of programming. Successes The team identified successes as the areas in which interview data suggested that GEP had been particularly effective. These included supporting women’s empowerment, building the capacity nationally to address gender issues, engaging with government, working with the private sector, raising awareness of gender issues, GBV, networking, and seamless service delivery. This section explores the evidence for each of these successful strategies or implementation approaches separately. Gender Equity Program: Final Evaluation Report 11 Women’s Empowerment The findings suggest that GEP’s success in empowering women stemmed from raising awareness of women’s rights, enhancing their ability to exercise their rights, and expanding their economic opportunities. Fifty-six percent of respondents noted that GEP’s engagement with multiple stakeholders over a large geographic area, a direct result of the GUCA approach, created pathways and space for enhancing women’s empowerment in the economic and social spheres (11/11 NAF; 40/76 SGs; 12/25 IPs; 6/11 Hub).5 The evidence suggests that GEP-supported activities helped raise women’s awareness of their rights, which contributed to their empowerment. A majority (72 percent) of respondents believed that GEP contributed to enhancing awareness of women’s social, economic, legal, and political rights (11/11 NAF; 55/76 SGs; 18/25 IPs; 3/5 GEs; 2/7 GOs). While most respondents believed that GEP had sensitized women to their rights, 28 percent believed that GEP-supported activities did not contribute to sensitizing men to the extent articulated in the cooperative agreement and project design documents (23/76 SGs; 7/25 IPs; 2/11 NAF; 1/7 GOs).6 In locations where women faced significant cultural barriers to participating in social and political activities, i.e., the Dir and Bannu districts of KP, sub-grantees found activities that helped women obtain Computerized National Identity Cards (CNICs) particularly effective in helping women exercise their rights. Seventy-five percent of eight respondents from KP noted the effectiveness of CNICs compared to only 33 percent across all provinces. Forty-one percent of respondents also specifically mentioned that GEP emphasized women’s economic empowerment (34/76 SGs; 5/11 NAF; 7/25 IPs). Some GEP grants focused on enhancing economic opportunities for women, and an entire grant cycle focused on home-based workers (HBWs), who are among the main contributors to the informal economy of Pakistan which comprises mostly women.7 Grants under this cycle sensitized HBWs to their rights and enhanced linkages between HBWs and markets for their products. Additionally, the NAF drafted and presented an HBW policy to the Government of Pakistan. Forty-one percent of respondents reported that these activities contributed to empowering HBWs; they offered examples including trade facilitation centers’ providing space for collective work and increasing the visibility of HBWs’ work (33/112 total; 16/76 SGs; 12/25 IPs; 5/11 NAF), increasing their knowledge of labor laws, raising their confidence level to access markets, improving bargaining skills with middlemen for advantageous pricing of their work, and upgrading their skills in material design (34/76 SGs; 5/11 NAF; 7/25 IPs). The evaluation provides evidence that GEP also improved women’s economic opportunities outside the home. Thirty-seven percent of sub-grantee respondents noted that women’s internship opportunities are usually restricted to traditional women-focused fields, and four cited examples of GEP’s facilitating women to join non-traditional professions such as the legal and ICT fields (3/76 SGs; 1/15 IPs). Additionally, 24 percent of sub-grantee and implementing partner respondents noted that since GEP’s intervention, women lawyers have begun practicing law in their respective areas (SGs 17/76; IPs 7/25). 5 The denominator indicates the number of respondents in a particular category while the numerator reflects the number of respondents that expressed the referenced opinion. 6 The cooperative agreement (pages 28 and 35) specifically notes that GEP was expected to “effect and measure gender￾responsive behavioral change and engage men.” Also, according to Objective 3, Combatting GBV: “Any meaningful initiatives aimed at sustainably reducing GBV and providing redress for victims must address these structures and engage men and children.” 7 International Labor Organization (ILO) Office for Pakistan-Islamabad, Recognising and supporting home-based workers: South Asian Regional consultation on national policy for home-based workers in Pakistan (Islamabad, Pakistan, 2010). Gender Equity Program: Final Evaluation Report 12 WOMEN’S ECONOMIC EMPOWERMENT “We are also working to change the behavior of middlemen towards women. Middlemen use abusive language and harass women. We sensitize women around these issues.” – Group interview, Cycle 7 Building Capacity Nationally to Address Gender Issues GEP’s national approach and its focus on gender equity enhanced capacity nationally to address gender issues. GEP grants supported geographically dispersed small organizations, many from remote rural areas, that were often not otherwise able to obtain funding for gender activities. GEP also focused solely on gender issues, as opposed to programs where the focus on gender may be diluted. Sixty-nine percent of respondents believed that this focus allowed GEP sub-grantees to view situations through a gender lens and to work exclusively on women’s issues (53/76 SGs; 10/11 NAF; 20/25 IPs; 1/7 GOs; 1/5 GEs). Some GEP-supported grant activities also pursued a broader human rights agenda, and a quarter of sub￾grantee respondents saw value in targeting other groups such as transgender, minorities, and internally displaced persons (IDPs), particularly in the context of GBV (18 of 76). WORKING WITH PARLIAMENTARIANS “AF has a good relationship with parliamentarians. The GEP program and AF complement each other and collaborate with others to create influence. However, being a leading organization we are concerned about laws being implemented and how they could become reality.” – Implementing partner interview When asked in an open-ended question about the types of capacities GEP had built in their organizations, implementing partners emphasized financial management, integrating gender policies in programming, and linkages with other donors. Fewer sub-grantees than implementing partners mentioned GEP’s contribution to capacity building but they emphasized improving their capacities in human resource management and monitoring and evaluation documentation (Figure 2). Engaging with Government GEP built on AF’s experience and connections to engage with government primarily in advocating for pro-women legislation, supporting legislation, and directly supporting (through grants) national and provincial women’s machineries. Just over half of respondents (52 percent) believed that GEP effectively leveraged the past legislative and advocacy work of AF to gain unprecedented access to parliamentarians and government departments (10/11 NAF; 19/25 IPs; 28/76 SGs; 3/5 GEs; 5/7 GOs). Forty-six percent of respondents believed that through its advocacy work (e.g., by hosting large-scale advocacy events and building consensus among important stakeholders), GEP influenced pro-women legislation at the provincial level (10/11 NAF; 17/25 IPs; 26/76 SGs 2/5 GEs; 2/7 GOs). As examples, they noted that GEP, in collaboration with other civil society organizations (CSOs), contributed to passage of the Sindh Child Marriages Restraint bill, preparing domestic violence bills in Sindh and Punjab, and drafting the Sexual Harassment Act in Balochistan, all of which will benefit women at large. Gender Equity Program: Final Evaluation Report 13 FIGURE 2: SUB-GRANTEES’ PERCEPTIONS OF CAPACITY BUILDING ROLE OF AF IN POLICY LEGISLATION “During the execution of GEP, the Child Marriage Restraint bill, Domestic Violence bill, and Provincial Commission bill were passed; and advocacy for the Acid Throwing bill is underway with AF. The strong lobby and networking made us reach these achievements, and all credit goes to AF.” – Sub-grantee, Karachi Additionally, 57 percent of respondents noted that NAF, as an advisory forum with inter-provincial representation from government, political parties, and civil society, facilitated GEP in cross-fertilizing ideas around policy issues and initiated discussions on laws (18/25 IPs; 7/11 NAF; 2/5 USAID; 2/7 GOs; 1/5 GEs). Respondents explained that NAF has one central committee and seven provincial committees and provides a platform where representatives of ministries, secretaries, and academics can come together to formulate practical approaches to addressing gender issues. NAF endorsed the grant cycle themes, thereby providing GEP with some assurance of buy-in from government and civil society. NAF members who belong to political parties and are able to generate political support for GEP projects and ideas. Thirty percent of respondents reported that AF staff in the regional offices supported legislation by providing parliamentarians with relevant information and input in drafting and reviewing bills (7/11 NAF; 13/25 IPs; 15/76 SGs; 1/5 GEs; 1/7 GOs). Sub-grantees from Cycle 7 also noted that they had translated the protection against sexual harassment laws and policies into regional languages to enable more people to understand the laws. Additionally, 56 percent of respondents believed that GEP’s support for advocacy projects facilitated discussion and debate around pro-women laws and policies (11/11 NAF; 20/25 IPs; 6/11 Hub; 30/74 SGs; 4/7 GOs; 3/5 GEs). 25% 32% 37% 33% 42% 20% 40% 44% 20% 44% 36% 24% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% Linkages to other donors Integrating gender policies in programming M&E documentation Financial management Human resources Administrative Percent of respondents Implementing parnters (N=25) Sub-grantees (N=76) Gender Equity Program: Final Evaluation Report 14 Thirty-six percent of respondents noted that GEP supported Women Development Departments (WDDs) in three provinces—Sindh, Khyber Pakhtunkhwa (KP), and Balochistan—where the WDDs are under-resourced and marginalized. These respondents suggested that GEP should have engaged more with the Planning and Development (P&D) departments because they have the authority to approve projects and are positioned to facilitate mainstreaming gender in government planning and processes. GEP also supported national and provincial Commissions on Status of Women (CSWs) by providing non-competitive grants to strengthen their institutional capacity in the areas of research and service delivery on GBV (9/11 NAF; 14/25 IPs; 4/7 GOs; 16/76 SGs; 2/5 GEs). Working with the Private Sector GEP worked with the private sector to find internship opportunities for women and to support legislation. Thirty-five percent of implementing partner and sub-grantee respondents reported that GEP helped grantees establish linkages with private sector organizations through mechanisms such as ICT internship programs (17/25 IPs; 18/76 SGs). However, 20 percent of implementing partner respondents (5/25) believed that more networking was necessary, and a quarter believed it was important for GEP to devise a mechanism to help sub-grantees network more effectively with the private sector. GEP also supported the private sector to back the Protection Against Sexual Harassment in the Workplace bill and the mandatory committees to be set up under the Act of the same name (15/25 IPs; 19/76 SGs). Two sub-grantee respondents reported problems working with private sector organizations at the district level because they required approvals from their urban-based head offices, which delayed implementation of project activities and made it difficult for the sub-grantees to follow up. Increasing Awareness of Gender Issues GEP increased sub-grantees’ knowledge of gender issues by supporting research studies and developing training modules and information, education, and communication (IEC) materials. Sixty-eight percent of respondents reported that GEP sub-grantees used these materials for capacity building and advocacy purposes (58/76 SGs; 18/25 IPs; 4/7 GOs; 3/5 GEs; 1/11 NAF). AF reported that GEP supported women and gender studies departments in 25 public universities (e.g., by working with them to develop curricula). Forty-four percent of 25 implementing partner staff members, including staff from three of the four universities the evaluation team interviewed, reported that the universities are using the curricula, and the GEP chief of party reported that GEP has standardized the curricula and distributed it to all the GEP-supported universities. RAISING AWARENESS OF GENDER ISSUES “There was only one gender studies department, in University of Balochistan, while the other universities, e.g., Balochistan University of Information Technology, Engineering, and Management Science (BUITMS) and Sardar Bahadur Khan Women’s University in Quetta, were not familiar with gender studies and its importance. GEP sensitized them and now they are planning to establish gender studies departments in their universities.” – AF field staff, Quetta Gender-Based Violence Respondents noted that GEP activities addressed both prevention and response to GBV (41/76 SGs; 14/25 IPs; 11/11 Hub; 1/7 GOs). They specifically mentioned activities aimed at awareness raising, sensitizing various groups to GBV, and teaching women how to obtain and register First Information Reports (FIRs) from police (11/11 Hub; 21/25 IPs; 31/76 SGs; 2/7 GOs; 1/5 GEs). Implementing partner Gender Equity Program: Final Evaluation Report 15 representatives explained that GEP addressed GBV through multiple initiatives with government and CSOs, including providing support to strengthen government-run shelters/Darulamans8 which were under-resourced; building the capacity of judicial commissions, lawyers, and bar councils (2/25 IPs); establishing waiting rooms in courts to facilitate women (4/76 SGs; 3/25 IPs); creating internship opportunities for women lawyers (3/25 IPs; 3/76 SGs); and providing legal aid (13/76 SGs; 6/25 IPs; 4/11 Hub), helpline (2/25 IPs; 2/11 Hub), psychosocial services (5/76 SGs; 3/25 IPs; 1/11 Hub), and medical aid (2/76 SGs; 1/25 IPs;1/7 GOs), among other services. Respondents validated the importance of activities focused on facilitating women’s access to justice in the context of GBV; 49 percent believed that training on gender and human rights should be institutionalized through regular training of the judiciary (11/11 Hub; 18/25 IPs; 28/76 SGs; 2/7 GOs; 2/5 GEs). Seamless Service Delivery Respondents noted that the seamless service delivery model (Hub) was a conceptually good model (27/76 SGs; 1/5 GEs; 5/25 IPs). As of July 2016, GEP had established Hubs in two provinces, Punjab (in Lahore) and Sindh (in Karachi). Hubs bring together various service providers to offer survivors of GBV access to a comprehensive range of services and referrals under one umbrella (Figure 3). The Lahore shelter was created to serve minority women who were discriminated against in the Darulamans. Hub grantees and AF explained that AF influenced the Punjab government to include a similar Hub model in the Women’s Protection Act by setting up one-stop GBV service centers. Two-thirds of the three GEP senior managers interviewed stated that because the Hub model was still in its nascent stage it was too soon to comment on its performance. However, all 11 Hub grantees interviewed noted a lack of effective coordination among different actors (Lahore) and the lack of a formal memorandum of understanding between the various partners (Karachi). Forty-two percent of implementing partners and Hub grantees (9/25 IPs; 6/11 Hub) noted the importance of including the police as key stakeholders in Hub activities. They explained that the police are normally the first point of contact for GBV survivors and play a vital role in case management and ensuring that the rights of the survivor are honored, as well as processing the investigation towards justice. Therefore, the police’s sensitization and understanding of GBV is critically important. However, forty-two percent of implementing partners and Hub grantees (6/11 Hub; 9/25 IPs) noted that USAID policy would not permit them to include the police.9 ENGAGING GOVERNMENT IN SEAMLESS SERVICE DELIVERY According to AF staff, Hub grantees, and implementing partners, Cycle 8 grant activities developed district women’s support groups and lawyers’ groups to provide legal aid to GBV survivors and build the capacity of government to address GBV issues. AF intends to link these groups with women’s protection centers that the government of Punjab is establishing, each of which will have a staff of about 100 and forensic laboratories, a helpline, and lawyers. The centers will link survivors to Benazir Income Support Program and Akhuwat for enterprise training and access to microcredit. 8 Government-run shelters for women. 9 The USAID cooperative agreement with AF mentions involvement with police in several sections, for example: related to Objective 1, access to justice (including the establishment of legal helplines, clinics, kiosks, women-friendly facilities in police stations and courts, and female staffing and employment opportunities); data capture and monitoring of institutional performance (covering police, courts, Federal Investigation Agency, and civil service); and training police/judicial officers in targeted jurisdictions to record the occurrence and status of GBV complaints. Therefore, a misunderstanding or miscommunication existed between AF and the sub-grantees, and possibly with USAID, about GEP’s working with police. Gender Equity Program: Final Evaluation Report 16 Networking Respondents explained that GEP established a network, the PGCN, 10 of sub-grantees and others representing a diverse geographic and issue-based spread and a mix of small and large NGOs (32/76 SGs; 15/25 IPs; 1/5 USAID). The 11 percent of sub-grantees and implementing partners that participated in the network reported that it was effective. However, the evaluation team learned that many respondents (56 percent) did not know about PGCN or had not participated in its activities. The GEP chief of party explained that GEP was responsible for coordinating PGCN activities and compiled a directory of all sub-grantees but that it provided no formal coordination mechanism or guidance for subgroups of the network, i.e., academia, universities, and helpline. Forty-two percent of implementing partners and Hub grantees (9/25 IPs; 6/11 Hub) noted the importance of including the police as key stakeholders in Hub activities. They explained that the police are normally the first point of contact for GBV survivors and play a vital role in case management and ensuring that the rights of the survivor are honored, as well as processing the investigation towards justice. Therefore, the police’s sensitization and understanding of GBV is critically important. However, forty-two percent of implementing partners and Hub grantees (6/11 Hub; 9/25 IPs) noted that USAID policy would not permit them to include the police.11 FIGURE 3: HUB SEAMLESS SERVICE MODEL Constraints In response to an open-ended question about constraints that affected sub-grantees’ and implementing partners’ implementation of GEP grants, a substantial number of respondents mentioned financial limitations, complying with USAID’s branding policy, poor communication, administrative issues, and external events (Figure 4). Both implementing partners and sub-grantees identified the main external constraints as: the worsening security situation; a growing resistance to gender and human rights promotion in some areas; and natural shocks, such as floods, that required immediate response and took priority over GEP activities. Implementing partners and sub-grantees identified constraints differently: implementing partners brought up those associated with USAID, such as delayed payments 10 The PGCN is a network of Pakistani CSOs, academic and research institutions, and media organizations working together to promote gender equity in Pakistan. 11 A misunderstanding or miscommunication existed between AF and the sub-grantees, and possibly with USAID, about GEP’s working with police. See footnote 9. Gender Equity Program: Final Evaluation Report 17 and compliance issues, while sub-grantees mentioned constraints related to their issues with GEP and AF in administrative and financial areas. FIGURE 4: SUB-GRANTEES’ PERCEPTIONS OF CONSTRAINTS Sub-grantee respondents described the specific nature of the various constraints as follows:  Delays in receiving payments from AF created problems for some sub-grantees. A respondent from one sub-grantee said they could not pay salaries for six months while they waited for AF to send their grant payment. As a result, several staff members left the organization. Another affected sub-grantee commented that failing to pay staff violated their human rights.  Forty-six percent of sub-grantees reported that USAID branding requirements increased security risks in some areas. Furthermore, 13 percent of sub-grantees reported that delayed branding approvals caused them to postpone or cancel events, which not only delayed the event but also negatively affected the sub-grantees’ sense of ownership of the project and events. They explained that it took time to meet exacting USAID requirements for logo formats, colors, and content which delayed USAID approval of banners and other material.  Seventy-eight percent of respondents noted that the reimbursement mode of payments did not work well because of complex internal procedures within GEP that delayed payments to sub￾grantees. Some small organizations explained that they require timely payments because they do not have financial reserves.  Lack of clarity from GEP regarding lines of authority and staff responsibilities resulted in poor communication with sub-grantees. Fifty percent reported that when they communicated with regional GEP staff, they learned that approvals and feedback originated from the AF head office in Islamabad and not in the regional offices. This system caused delays in obtaining answers and 56% 54% 51% 53% 43% 52% 32% 44% 64% 76% 0% 10% 20% 30% 40% 50% 60% 70% 80% Financial constraints USAID branding policy Poor communication Administrative constraints External constraints Percent of respondents Implementing parnters (N=25) Sub-grantees (N=76) Gender Equity Program: Final Evaluation Report 18 approvals. Half of the 76 sub-grantee respondents interviewed noted that the GEP regional office did not appear to have authority to take actions.  One sub-grantee complained that GEP’s selection process for grant awards did not pay sufficient attention to the expertise and past experience of the NGOs, and this sometimes resulted in awards to NGOs without sufficient expertise or capacity. Regional GEP offices had little role in the sub-grantee selection process despite the inherent differences within each provincial context.  The organizational structure of GEP and the compliance requirements created a complex process that required multiple steps before funds were released to sub-grantees. The lengthy process wasted time and frustrated sub-grantees. However, when sub-grantees raised this issue with AF and TAF, the implementing partners attributed the delays to the sub-grantees’ weak financial management capacity. This issue suggests that the implementing partners and sub￾grantees require greater capacity and signals that communication between the implementing partners and the sub-grantees was not effective. The implementing partners most often mentioned constraints involving interactions with USAID. For example:  Twenty percent of implementing partners reported that slow USAID approval of grants necessitated renegotiating with GEP partners and revising price proposals since prices tend to increase over time. Renegotiating budgets caused further delays. In addition, the delayed grant starts reduced the time for implementing grant activities.  Failure to resolve general sales tax (GST) 12 issues wasted senior staff time and caused delays in organizing program events.  Changing the financial reporting system from quarterly to monthly caused implementing partners to expend more resources on financial issues which they said took away from time spent on programmatic performance.  A reduction in GEP’s funding adversely affected the program, and GEP was unable to implement important planned initiatives, such as a TV serial on access to justice and capacity building of PGCN. Challenges While the GUCA model was effective in the sense that it made it practical for USAID to support a large number of small grants, sub-grantees and implementing partners mentioned several challenges related to how GEP implemented the grants under the GUCA model. In particular: 1. Seventy-four percent of sub-grantees highlighted administrative delays as a major challenge that wasted staff time and reduced the time available to implement grant activities. In two extreme cases, sub-grantees reported that they did not received funds until one year after they began implementing grant activities. Forty-eight percent of the 25 implementing partner respondents interviewed highlighted the issue of slow fund disbursements from USAID. Sub-grantees were 12 GST is chargeable on all locally produced and imported goods. USAID policy does not permit reimbursements for payment of taxes. Sub-grantees requested waivers from USAID for paying the sales tax. Gender Equity Program: Final Evaluation Report 19 more likely to blame poor communication and cumbersome administrative procedures within AF and between AF and GEP. 2. Forty-seven percent of sub-grantees and 48 percent of implementing partner respondents interviewed stated that the 6 to 18-month grant duration was too short to generate the desired impact, particularly for grants related to GBV prevention and economic empowerment. 3. All 11 Hub sub-grantees and 52 percent of the 25 implementing partner respondents interviewed stated that engaging relevant stakeholders in some grants remained a challenge, e.g., police in cases of GBV (because of the USAID-imposed prohibition), social media users, and the transgender community. 4. Twenty percent of sub-grantees and 44 percent of implementing partner respondents said that some grants lacked an integrated approach, e.g., social mobilization activities and service delivery activities were done by different groups at different times. Conclusions The evaluation team reached the following conclusions related to Question 1: 1. GEP raised awareness and visibility of gender issues, particularly GBV, and to a degree built the capacity of many small and geographically dispersed organizations to engage with these issues at various levels and deliver services. GEP’s scale, its exclusive focus on gender issues, and the ability under the GUCA agreement to disburse grants to many small organizations contributed to broad awareness raising and capacity building. GEP’s support for research and developing training modules and IEC materials was a particularly effective strategy for raising awareness of gender issues. GEP’s work to strengthen the women and gender studies curricula in 25 public universities was also an effective approach for raising awareness. 2. GEP’s integrated approach to combating GBV was particularly effective. Respondents noted the value of activities focused on raising awareness of women’s rights, supporting women to exercise their rights, building the capacity of the legal system to enforce women’s rights, and providing the full continuum of support that victims of GBV require to seek and obtain justice and deal with the physical, social, and psychological consequences of GBV. More than a quarter of respondents, however, believed that GEP activities in GBV could have been more effective if they had engaged men and boys to a greater extent. Respondents saw the value of the seamless service delivery approach, but since it is in a nascent stage, there is little evidence yet of its effectiveness. 3. GEP’s approach to women’s empowerment encompassed activities designed to raise women’s awareness of their rights, enhance their ability to exercise their rights, and expand their economic opportunities. Raising women’s awareness of their social, economic, legal, and political rights cut across many thematic areas, and 72 percent of respondents found these activities effective. GEP grants that focused on HBWs and internship opportunities, developed by forming linkages with the private sector, seemed particularly effective in expanding women’s economic opportunities and empowerment. In particularly remote and conservative areas, GEP activities to help women obtain CNICs helped women exercise their rights, an important step toward empowerment. 4. By leveraging the experience and networks of its national implementing partner AF, GEP was particularly effective in supporting sub-grantee advocacy and lobbying efforts to promote gender legislation. The NAF was also an effective body in this regard. While GEP-supported activities Gender Equity Program: Final Evaluation Report 20 supported advocacy and legislation, the result of this work is not clear because it is difficult in most instances to attribute results solely to GEP activities. 5. GEP’s support to three under-resourced and marginalized government departments, the Women Development Department (WDD), the National and Provincial Commissions on the Status of Women, and the Ministry of Human Rights, was important. However, it would have been useful to engage with other departments with greater influence, e.g., the P&D departments that have the authority to approve all projects at the provincial level and could facilitate mainstreaming gender in government planning and processes. 6. Networks within GEP—such as sub-grantees, universities, and Hubs—served a useful purpose. However, networks need clear coordination mechanisms to function effectively, and coordination was not as strong as it could have been. For example, the potential of the PGCN was not realized, partly due to inadequate planning and coordination by AF. More direction and coordination by AF to PGCN could have enhanced PGCN’s effectiveness. 7. Insufficient grant size and duration limited some sub-grantees’ ability to achieve meaningful results. The maximum duration of sub-grants under GEP was 18 months, which was insufficient in some cases to adequately develop project activities, especially given the delays in grant approvals and processing without a corresponding extension of the end date. Findings for Evaluation Question 2 Evaluation question: To what extent, and how, did the grants-under-cooperative agreement mechanism (GUCA) affect implementation and results? This question examines the GUCA model in terms of its effect on GEP implementation and results. The GUCA model authorizes the implementing partner of a cooperative agreement to disburse sub-grants to other organizations. In GEP, USAID awarded a cooperative agreement to AF and TAF. The terms of the cooperative agreement authorized these implementing partners to disburse sub-grants (GUCAs) to other organizations. The implementing partners then chose specific grant mechanisms through which to award sub-grants. AF reported that it used simplified grants and fixed obligation grants (FOGs) as sub￾grant mechanisms. The answer to this question distinguishes between the GUCA model and the specific grant mechanisms AF used to award grants to sub-grantees. To answer this question, the evaluation team asked interview respondents about their experiences with the GUCA model and about the pros and cons of various mechanisms for awarding sub-grants. Because of their diverse roles, the various types of stakeholders were able to answer different aspects of the question. The implementing partners were familiar with the GUCA model and could answer questions about GUCA and about the alternative grant mechanisms they used to disburse sub-grants. Sub-grantees generally knew little about the GUCA model but knew more about the characteristics of alternative sub-grant mechanisms. The evaluation team asked all categories of respondents about alternative sub￾grant mechanisms and all respondents except donors and government officials about the GUCA model. The evaluation team also asked respondents the extent to which they agreed or disagreed with a series of hypothetical statements, many of which addressed aspects of the GUCA model and sub-grant mechanisms. The GUCA Model Among other requirements, USAID authorizes grant programs under a GUCA authority in cases where “Executing a number of small grant activities is particularly difficult for the responsible USAID Mission or Gender Equity Program: Final Evaluation Report 21 office.”13 GEP’s strategy of engaging a large number of grass-roots organizations nationwide to work on selected gender themes thus provided a strong rationale for using the GUCA model. GEP’s ability to provide grants to a large number of geographically dispersed organizations under the GUCA model strengthened civil society’s role in addressing gender issues and built the capacity of AF. Thirty-two percent of respondents from AF, the sub-grantees, and USAID believed that the organizational and geographic spread of GEP grants created new entry points for addressing gender issues with small rural organizations (14/25 IPs; 1/5 USAID; 22/87 SGs). One of the five USAID respondents specifically noted that GEP’s ability to disburse grants to a large number of organizations under the GUCA model strengthened civil society and enhanced grass-roots organizations’ capacity to work on gender issues. Respondents believed that GEP developed the capacity of AF in grant making (63 percent: 18/25 IPs; 1/5 GEs), establishing new linkages with other donors (50 percent: 14/25 IPs; 1/5 GEs), financial management (47 percent: 13/25 IPs; 1/5 GEs), and procurement (32 percent: 8/25 IPs). One of the five USAID respondents, however, believed that GUCA was not an effective model to achieve the overall objectives of GEP because it did not give USAID sufficient latitude to intervene directly in design and implementation. The GUCA model provided AF the latitude to design a grant program that was sensitive to local needs and flexible enough to adapt to emerging priorities. The model did not prescribe a specific programmatic approach (2/5 USAID) and gave AF a great deal of flexibility in designing the grant program. The GUCA approach gave AF the ability to specify grant-funded activities and thematic areas (5/25 IPs; 1/5 USAID; 9/87 SGs) and decision-making authority over the granting process (12/25 IPs; 2/5 USAID; 16/87 SGs). Twenty percent of respondents believed the flexibility in design enhanced GEP’s ability to respond to emerging priorities (9/25 IPs; 3/5 USAID; 1/5 GEs; 11/87 SGs). GUCA Administration The strategy of pairing an international organization with a Pakistani NGO had both strengths and drawbacks. Twenty-six percent of respondents mentioned that international organizations bring needed financial and administrative management expertise (17/25 IPs; 3/5 USAID; 1/13 Donors; 1/5 GEs; 13/87 SGs). On the other hand, 44 percent of sub-grantee respondents said that Pakistani organizations bring an understanding of the culture and communities, established networks, and relationships with local NGOs and private companies (38/87 SGs). Forty-seven percent of respondents agreed that established Pakistani organizations generally have rapport with the government and with parliamentarians which is an asset when advocating for gender-friendly policy and laws (20/25 IPs; 3/5 USAID; 3/13 Donors; 2/5 GEs; 4/11 NAF; 38/87 SGs; 2/7 GOs). The findings for question 1 highlighted a number of issues related to AF’s ability to effectively manage the GUCA grants program. Slow disbursement of grant funds emerged as the most frequently mentioned issue (56/76 SGs; 12/25 IPs). While implementing partner respondents were most likely to blame delays on USAID’s administrative requirements, sub-grantees were more likely to attribute delays to poor communication between AF and GEP and AF’s cumbersome administrative processes. Grant Mechanisms This section examines respondents’ experiences with GEP grants and with alternative grant mechanisms. Because few respondents had much experience with grant mechanisms, the data on the advantages and disadvantages of alternative mechanisms are quite thin. 13 ADS 302. https://www.usaid.gov/sites/default/files/documents/1868/302.pdf Gender Equity Program: Final Evaluation Report 22 While sub-grantees appreciated the grants they received through GEP, some believed the way in which AF designed and administered the grants was overly restrictive and rigid, noting in particular that the short time frame for implementation and limited funding stifled innovation. One said, “I believe that GEP grantees should be provided an option to submit project proposals individually [directly]. Being aware of ground realities, the proposals would be very context specific and achieve desired results and create impact. But with GEP the project design was already prepared by AF and the meeting on the proposal seemed nothing other than a management discussion plan.” Another sub-grantee believed that “GEP is too bureaucratic. There is no flexibility. They stop you from making amendments in the documents.” Sub-grantees also expressed frustration over delayed payments and a complicated system of milestone payments or reimbursements and felt they had to spend too much time on financial issues and not enough on improving and managing the program aspects of GEP. AF began using simplified grants that reimburse grant recipients for incurred costs. It found the mechanism difficult administratively (i.e., processing reimbursements) and almost immediately switched to using fixed obligation grants (FOGs). Sub-grantees’ direct experience with grants under GEP therefore largely reflects their experience with FOGs. The interviews asked about the grant mechanisms with which the respondents were familiar and their perceptions of those mechanisms. Many respondents spoke of characteristics of grant mechanisms rather than actual mechanisms and few provided any information about the advantages or disadvantages of alternative mechanisms. Of the 51 respondents who answered the question about grant mechanisms, most (64 percent) were familiar with directly implementing grants (i.e., without another organization between the grantee and the donor), two sub-grantees mentioned FOGs, and one donor and one sub-grantee mentioned contracts. All of the remaining 14 responses mentioned characteristics of grant mechanisms such as advance payments versus reimbursement approaches and grants to consortiums of organizations. Because most respondents were unfamiliar with specific grant mechanisms, they spoke instead about characteristics of grants with which they were familiar. Table 3 summarizes respondents’ perceptions of the advantages and disadvantages of various grant characteristics. Only 14 respondents (13/25 SGs; 1/5 Donors) mentioned advantages and disadvantages, so the evidence is very thin. The strongest evidence comes from the six sub-grantee respondents who preferred advance payments to reimbursements because they believed small grantees would not have the liquidity to wait for reimbursements. The findings from question 1 on delayed milestone payments provide concrete examples of the consequences of limited liquidity. TABLE 3: ADVANTAGES AND DISADVANTAGES OF GRANT CHARACTERISTICS Grant Characteristics Advantages and Disadvantages Fixed obligation grants (FOGs) The FOG mechanism awards a fixed amount and specifies a payment schedule for incremental disbursements tied to meeting specified deliverable milestones. While 2 of 43 sub-grantees were familiar with FOGs, none mentioned advantages or disadvantages of the mechanism. Direct implementation In this context, direct implementation refers to a grantee organization’s working directly with the donor instead of being one step removed as a sub￾grantee. Direct implementation is not a grant mechanism per se. Instead, it describes the nature of the relationship between the grantee and the donor. Sixty-four percent of respondents were familiar with direct implementation approaches. The six respondents (all sub-grantees) who mentioned advantages and disadvantages of the mechanism mentioned the advantages of direct contact with the donor and the reduced financial risk when payments were not channeled through another organization. Gender Equity Program: Final Evaluation Report 23 Grant Characteristics Advantages and Disadvantages Advance payments and reimbursements Advance payments are a characteristic of grant mechanisms that disburse funds to grantees in advance. Reimbursement models reimburse grantees based on actual expenditures. The simplified grant that AF first used with GEP was a reimbursement model. Four sub-grantees said they preferred advance payments over reimbursement models because small organizations did not have the liquidity to wait for reimbursements. Consortium model This model refers to a grantor’s awarding a grant to a consortium of grantees where each consortium member has its own mandate but is linked to common objectives. Two respondents mentioned that consortium approaches were strong because they leveraged the various areas of expertise of the consortium members. Respondents’ preferences for grant mechanisms hinged entirely on the risks associated with payment modalities; advance payment was less risky than a reimbursement approach, especially for small organizations with limited financial resources. This implies that, in general, respondents will prefer any mechanism that provides for advance payments over mechanisms based on reimbursing grant recipients for incurred expenses. Although FOGs to NGOs require milestone payments, the mechanism provides the flexibility to set milestone payments that more than cover the cost of achieving the milestone specifically to address recipients’ liquidity constraints. It also provides the flexibility to define milestones related to incurring costs required to begin activities (e.g., equipment or supplies).14 Simplified grants, since they are a cost reimbursement approach, do not address the liquidity concerns of sub-grantees. Given AF’s difficulty in managing the grants under GEP, ease of administration is also a relevant criterion for identifying appropriate grant mechanisms. AF’s own experience managing the simplified grant mechanism suggests that FOGs are preferable to simplified grants in this respect. Responses to Hypothesis Statements The interview instruments also contained a series of hypothetical statements and asked respondents whether they strongly agreed, agreed, neither agreed nor disagreed, disagreed, or strongly disagreed with each statement. Eight of the hypothesis statements addressed some aspect of the grants. Annex 7 contains the detailed responses for each statement. Hypothesis 1: There is a positive and synergistic impact from the umbrella sub-grants model.15 Eighty-eight percent of respondents overall and a majority of each type of respondent either agreed or strongly agreed with this statement (23/24 IPs; 3/5 USAID; 4/5 NAF). Representatives of the two implementing partners gave the example that the requirement to meet USAID’s reporting and accountability guidelines for grants improved and streamlined their systems and enabled them to expand their outreach. 14 USAID, Fixed Amount Awards to Non-Governmental Organizations: An Additional Help Document for ADS Chapter 303 (2014). Accessed from: https://www.usaid.gov/sites/default/files/documents/1868/303saj.pdf 15 “Umbrella sub-grant” is the term by which most respondents referred to the GUCA model. Gender Equity Program: Final Evaluation Report 24 Hypothesis 2: AF and USAID bureaucratic requirements reduced the efficiency of GEP implementation. Sixty-four percent of respondents overall, and a majority of each type of respondent, either agreed or strongly agreed with this statement (37/65 SGs; 18/24 IPs; 5/5 USAID). Sixty percent of sub-grantee respondents acknowledged that it was sometimes difficult to meet USAID’s bureaucratic requirements but believed that doing so was part of the project and that sub-grantees must plan for and follow the requirements. Sub-grantees also noted that the grant process was long and tedious with the main issues being budgetary realignment (13/76 SGs), inflexibility (7/76 SGs), and accounting for GST (11/76 SGs). Sub￾grantees were more likely to attribute these delays to AF than to USAID. A few sub-grantees reported that the lengthy grant process delayed implementation of project activities (7/76 SGs). Hypothesis 3: The umbrella sub-grants model is more effective than direct implementation by implementing partners for gender equity programs in Pakistan. Seventy-two percent of respondents either agreed or strongly agreed that the GUCA model is more effective than direct implementation by implementing partners because it builds the capacities of the grantee (AF) and a large number of sub-grantees. Respondents mentioned that having an Islamabad￾based organization leading the project was not very effective because it did not have the local knowledge in the provinces to select the most qualified sub-grantees. These respondents suggested that having an umbrella organization at the provincial level would work better. Hypothesis 4: The umbrella sub-grants model enables coordination among Hub grantees. Ninety-three percent of respondents either agreed or strongly agreed with this statement and believed that the seamless service delivery concept of the Hubs was sound. They noted that each sub-grantee had its own competencies in the Hub and GBV survivors could benefit from all the facilities at one location. Hypothesis 5: A Pakistani organization is a better choice than an international organization for awarding grants to Pakistani organizations. Of 116 respondents, 71 percent either strongly agreed or agreed that a Pakistani organization is a better choice for awarding grants because a Pakistani organization knows the realities on the ground and can better understand the local systems and environment, build the capacity of smaller organizations, and enhance its own expertise. Through grant-making, a Pakistani organization can be more effective than an international organization and can communicate more easily with government, other organizations, and beneficiaries. One respondent noted that the programming component of AF was exceptional but the financial component was weak. Hypothesis 7: The specific grants received under GEP responded to the needs of the sub￾grantees. Of 106 respondents, 83 (78 percent) either strongly agreed or agreed with this statement. Specific examples included supporting small organizations that had previously had difficulty obtaining funding. The grants provided them the means to pursue their objectives, while the opportunity to work with AF built their capacities. Gender Equity Program: Final Evaluation Report 25 Hypothesis 8: The umbrella sub-grants model is the most effective model to promote gender equity. Twenty-four sub-grantee respondents (71 percent) either strongly agreed or agreed with this statement. Because most respondents were not familiar with the GUCA model, however, probes about why they agreed produced no meaningful responses. Conclusions  The evidence, including respondents’ reactions to the hypothesis statements, suggests that the GUCA approach was effective in allowing USAID to (indirectly) disburse grants to many small organizations that would have found it difficult to obtain funding otherwise, thereby disseminating gender programming widely. While these organizations sometimes struggled to meet USAID requirements (e.g., branding, reporting, and accounting), learning to meet them built their capacities and those of AF. The GUCA approach of disbursing grants through a strong local organization also allowed USAID to leverage local experience and knowledge to implement a more relevant and effective small-grants program. While the GUCA approach had its advantages, AF appears to have struggled to manage it well. Sub-grantees bridled at the rigidity of AF’s process which gave them little latitude to employ their local knowledge or innovate and adapt. More importantly, delays in milestone payments— which sub-grantees attributed to poor communication within AF and between AF and GEP and overall cumbersome administrative procedures—created financial hardships for small organizations with limited financial resilience. Given the capacity of many local organizations, a grant mechanism that is easy to manage will be most appropriate, and even then, the organization may require capacity building or other support to manage the grant process. Although few sub-grantees knew about specific grant mechanisms, the evidence indicates a preference for mechanisms that do not reimburse sub-grantees for incurred costs. The limited financial resources of small organizations make a mechanism that provides funding in advance of the organization’s incurring costs more appropriate. If administered appropriately, FOGs provide this flexibility, although AF seems to have chosen not to implement FOGs in this way.  The flexibility of GEP’s design under the GUCA model permitted GEP to adapt its programming to emerging priorities. The GUCA model did not dictate to the implementing partners; it instead provided flexibility in selecting and awarding sub-grants to numerous small organizations across Pakistan, leveraging their connections and local expertise, building their capacities, and extending the geographic reach of GEP programming. GEP’s administration of the GUCAs, however, did not give sub-grantees much flexibility to design or adapt their grant activities.  Placing the GUCA model within a large and experienced women’s organization was a viable strategy as the program was able to leverage AF’s social capital, visibility, credibility, countrywide network of partners and volunteers, and connections and experience in advocacy with political parties and legislators.  Pairing a Pakistani organization with an international organization was an effective strategy. The international organization brought financial and administrative expertise. AF has been active in the gender space for years and provided deep understanding of culture and communities, experience working at the grass-roots level, policy awareness, advocacy experience, and gender linkages to GEP. The GEP experience, including working with TAF, improved AF’s capacities in financial management, human resources, administration, documentation, reporting, and grant making. Gender Equity Program: Final Evaluation Report 26  AF demonstrated an ability in grant-making and managing numerous sub-grantees at the grass￾roots level, while continuing to advocate for gender policies and practices. The transition from advocacy to grant making, however, brought management challenges, particularly in the areas of financial management, disbursing grant payments to sub-grantees, and developing a system for making milestone payments to sub-grantees (who found the payment system complex and time consuming). Findings for Evaluation Question 3 Evaluation Question: What are the areas of greatest need, priority, or opportunity for new gender programming going forward? The evaluation team collected data to answer this question from open-ended questions and from a set of semi-structured questions that elicited opinions on needs, constraints, and priorities for future programming. The questions asked respondents to think of three game changer interventions for advancing women’s empowerment and gender equity that were relatively easy to take up. The evaluation team also asked respondents to choose the 3 top programming priorities from a set of 13. The analysis compared responses to both of these questions to validate the evidence. The instruments also asked respondents how they would engage with men given the importance of men’s participation in programs for empowering women—a strategy endorsed by UN Women16 and GIZ’s gender specialist, among others. The findings for Question 3 are organized under needs, opportunities, and priorities for future programming as identified by the respondents. Identified needs fell into categories of working with government and women’s machineries, access to justice, women’s economic empowerment, prioritizing gender, and collaboration between CSOs and effective networks. The final section summarizes a donor mapping showing the gender-related activities of nine donor organizations and provides a segue to the recommendations of this report. Key Findings: Needs Working with Government and Women’s Machineries Ninety-two percent of respondents identified working with government and women’s machineries17 as a key need for gender programming going forward (75/87 SGs; 25/25 IPs; 7/7 GOs; 5/5 GEs; 13/13 Donors; 11/11 NAF; 4/5 USAID). The responses identified four specific issues on which programming could focus: (1) under-resourced ministries and line departments directly focused on women; (2) inadequate awareness of gender issues among other government departments; (3) hamstrung local governments; and (4) the need to build women’s leadership capacity at all levels. Women’s machineries at federal and provincial levels are in place but need resources and the authority to function effectively and adequately. These include the National Commission on the Status of Women 16 “UN Women’s work on engaging men and boys for gender equality is anchored in the belief that achieving gender equality is about transforming unequal power relations between men and women. This involves challenging notions of masculinity and traditional perceptions of manhood. It requires men to question power dynamics in their actions or their words at the personal, interpersonal, and societal levels and to take responsibility for change. Men need to be engaged as gender advocates— speaking out as active agents and stakeholders who can transform social norms, behaviours and gender stereotypes that perpetuate discrimination and inequality.” http://www.unwomen.org/en/news/in-focus/engaging-men#sthash.YemE3NVK.dpuf 17 National women’s machineries are a key institutional mechanism for the advancement of women and play a critical role in promoting and ensuring implementation of the Convention on the Elimination of All Forms of Discrimination Against Women (CEDAW) at the national level. Since devolution, women’s machineries operate at both the federal and provincial levels. Gender Equity Program: Final Evaluation Report 27 (NCSW), the two provincial Commissions on the Status of Women in Punjab and KP, and the Women Development Departments (WDDs) in each of the provinces. However, 69 percent of donors and 80 percent of gender experts interviewed believed that each of these entities, tasked with ensuring that the government fulfills its constitutional obligations towards women as full citizens and meets its national and international commitments, is largely marginalized and without sufficient technical capacity or resources. Thirty-six percent of other respondents agreed that these entities need to be strengthened (2/5 USAID; 24/87 SGs; 15/25 IPs; 3/7 GOs; 4/11 NAF). Thirty-five percent of respondents believed that government departments, for the most part, are not gender sensitized and that there is a need to work with ministries and departments that are often not included as key stakeholders in gender programming, e.g., the Planning Commission and provincial P&D departments (26/87 SGs; 9/25 IPs; 4/7 GOs; 2/5 GEs; 4/13 Donors; 5/11 NAF; 3/5 USAID). The P&D Department, together with the Ministry of Finance, has the authority to approve and prioritize development projects and allocate resources. Over the years, several donor initiatives,18 the most recent being the Gender Reform Action Plan (GRAP),19 have attempted to engage these departments, but the frequent turnover of staff results in limited understanding of gender programming and a need for continuous and sustained engagement. Local governments (union council, tehsil, and district) are in place in all provinces, but many are not yet fully functional. A government representative in Lahore noted, “It is important to engage women in local government, but also men, to build their capacities and strengthen them.” A donor respondent reported, “Low institutional capacity of government departments, low staff, low budget, and low skills are the combined reason for no actions at the provincial level and are the constraints for future programming.” Local governments include reserved seats for women councilors, and these, along with other members of the local government at each of the three tiers, are the first level of contact with citizens and communities.20 A government official interviewed in Lahore stressed the importance of engaging with local government, particularly in terms of gender gains. She emphasized that local governments have the potential to improve the lives of women but lack the capacity to do so at this stage. Partly this is because in some provinces the rules of business for local government have not been drawn up. Devolution at the sub-provincial level is a sensitive area for local and provincial government representatives, particularly women and others on reserved seats such as youth, farmers, and minorities. Overall, 16 percent of respondents noted these issues and added that local governments require gender sensitization and awareness raising of their roles and duties as well as resources (11/87 SGs; 5/25 IPs; 2/7 GOs; 2/5 GEs; 2/13 Donors; 1/5 USAID). Twenty-nine percent of respondents agreed there is a dearth of women’s leadership skills whether in parliament, local government, civil service, or NGOs. These respondents endorsed promoting women’s leadership at all levels but particularly for grass-roots women in local government (18/87 SGs; 9/25 IPs; 3/7 GOs; 1/5 GEs; 4/13 Donors; 5/11 NAF; 4/5 USAID). Jabeen and Iqbal21 explain that the presence of women representatives on reserved seats at all tiers of governance has led to the realization that just having women present is not enough. Often these women have entered public service for the first time and are intimidated by the male-dominated environment of 18 UNDP: Gender Mainstreaming in P & D Departments (2004-2007) and the Gender Responsive Budget Initiative with the Ministry of Finance (2005-2007) among others. 19 GRAP was initially funded through a loan by the Asian Development Bank in 2002; the loan remained underutilized and lapsed in 2007. The Government of Pakistan incorporated GRAP into the regular budget as a five-year project in 2005. 20 World Bank, Pakistan: Devolution in Pakistan, Report No 29912-PK Vol 2 (2004). 21 Nasira Jabeen and Muhammad Zafar Iqbal, “Gender and Local Governance in Pakistan: Promoting Participation through Capacity Building,” South Asian Studies 25, no. 2 (2010): 255-281. Gender Equity Program: Final Evaluation Report 28 the national and provincial legislatures as well as the district governments. They need not only technical information to understand their roles and functions and perform them efficiently, but also the soft skills of negotiation, assertiveness, and public speaking to be effective. Similarly, women civil servants are also often not gender sensitive and have had little exposure to gender rights. They struggle to find their way in the system. As Dr. Jabeen notes, there is a clear need to build the capacity of elected women to enable them to perform their governance roles effectively. Access to Justice Three notable findings relating to access to justice emerged from interviews with 153 respondents: (1) the need to improve access by strengthening, and in some cases putting in place, implementation mechanisms for existing women-friendly laws (5/7 GOs); (2) lobbying and advocating for laws and policies that protect women and safeguard their rights (3/5 GEs; 4/5 USAID); and (3) access to justice tied in with strengthening local governments and the judiciary to respond to violations of women’s rights at the district level (25/25 IPs; 13/13 Donors; 10/11 NAF). The progressive women-friendly laws enacted by the national and provincial assemblies have not been translated into actions because the rules of business for each law and the related mechanisms for implementation are not in place. Law enforcement and the judiciary at the district level are not fully aware of the laws, and procedures for redressal are cumbersome, especially for women.22 Eighty-nine percent of respondents, including 100 percent of sub-grantees and implementing partner respondents, noted the need to strengthen mechanisms for implementing existing laws and improve access to justice at the provincial and sub-provincial levels (87/87 SGs; 25/25 IPs; 3/7 GOs; 2/5 GEs; 10/13 Donors; 6/11 NAF; 3/5 USAID). Seventy-two percent of implementing partner respondents noted a need for new legislation in areas such as the protection of home-based workers and criminalizing domestic violence. Further, supportive legislation may have been enacted in some provinces but not in others. Two examples are the Domestic Violence bill and the Child Marriage Restraint Act passed by the Sindh Assembly. Those bills have been drafted for KP as well, but the KP Assembly has not legislated on them (1/5 GEs; 4/11 NAF). Moreover, the National Assembly has not legislated on the Domestic Violence bill (3/13 Donors). Twenty-six percent of respondents saw the need for some cross-provincial push and lobbying to have these laws taken up by all the provinces and by the National Assembly (23/87 SGs; 2/7 GOs; 1/5 USAID). Women’s Economic Empowerment Seventy percent of respondents identified women’s economic empowerment is an area of critical need for addressing women’s conditions and positions within society and the home (48/87 SGs; 25/25 IPs; 4/7 GOs; 4/5 GEs; 11/13 Donors; 11/11 NAF; 4/5 USAID). Much of the current women’s economic empowerment programming in Pakistan still focuses on traditional skills such as sewing and handicrafts, or adding value to these skills through embellishment. Respondents noted the need for women’s economic empowerment programs to identify and promote market demands for skilled labor, and also incorporate a human rights-based approach into all such interventions and trainings. Prioritizing Gender Almost all (90 percent) respondents identified the need to prioritize gender in both government and donor programming, and the need to back the rhetoric with adequately funded interventions (76/87 SGs; 25/25 IPs; 7/7 GOs; 5/5 GEs; 13/13 Donors; 7/11 NAF; 5/5 USAID). Donors’ funding priorities tend 22 Noreen Naveeda and Razia Musarrat, “Protection of women rights through legal reforms in Pakistan,” Journal of Public Administration and Governance 3, no. 4 (2013): 119-142. Gender Equity Program: Final Evaluation Report 29 to shift as the policies of foreign governments change and the demands on the limited pool of aid funds increase. Further, based on 13 interviews with donors, most have specific geographic priorities, a practice that may leave unaddressed the development needs of some disadvantaged regions. The federal and provincial governments in Pakistan also fall short on their commitments to women’s empowerment. Since the devolution in 2010, there is a gap at the federal level for gender equity that requires attention, noted a donor. Respondents expressed opinions that, in general, government institutions are ambivalent towards NGOs, especially those that work on women’s rights or have a rights-based agenda. Often these organizations are lauded as partners in development, and as frequently they are charged with promoting values that go against the prevalent culture and gender norms. Most recently NGOs are required by the government to get clearance for every field activity and intervention, and those who receive funds from international organizations are subjected to greater scrutiny. Continuity of Gender Programming Twenty percent of implementing partner respondents (5/25) pointed out that it was imperative that both the government and international donors sustain resources and attention for continuity of gender programming. Sub-grantee (5/87) and implementing partner (5/25) respondents noted the need for greater coordination between and among donors, government departments, and CSOs to maximize scarce resources and to complement and create synergy in donor efforts. Collaboration Seventy percent of respondents also emphasized the need for collaboration between CSOs and effective networks (66/87 SGs; 25/25 IPs; 7/7 GOs; 4/5 GEs; 11/13 Donors; 3/11 NAF; 2/5 USAID). A quarter of respondents noted that collaboration between large and small NGOs and across sectors and geographic areas is needed to increase capacity and expand outreach. Given Pakistan’s young demographic profile, thirty-two percent of respondents also noted a need to involve youth. Twenty percent of respondents identified the need to proactively promote women’s rights and to fully utilize the potential of media for innovative gender programming and awareness-raising. Electronic mass media, social media, and alternate media need to be used efficiently to promote behavior change, sensitize, and inform, especially among youth. Key Findings: Opportunities Policies and Laws The federal and provincial governments have enacted laws and policies to promote gender equality such as the Women’s Empowerment Package (Punjab), the Women’s Empowerment Policy (Sindh, KP, and Balochistan), and the National Policy on the Development and Empowerment of Women (federal government). Additionally, the last ten years have seen a spate of laws passed that have consolidated women’s constitutional rights. These include amendments to the Pakistan penal code and laws on domestic violence, harassment of women at the workplace, and harmful customary practices that infringed on women’s rights.23 The laws have been possible in part because women won 17 to 22 percent of the seats in national and provincial parliaments in the general elections of 2008 and 2013, the majority nominated by their political parties against the reserved seats for women. The women parliamentarians worked effectively across party lines as members of the Women’s Political Caucus to table bills and push through laws.24 Sixty-nine percent of respondents, including large majorities of implementing partner respondents, government officials, gender experts, and donors spoke of policies 23 Anita M. Weiss, Moving Forward with the Legal Empowerment of Women in Pakistan (Special Report No. 305), (Washington, D.C.: U.S. Institute of Peace, 2012). 24 Noreen and Musarrat, “Protection of women rights through legal reforms in Pakistan.” Gender Equity Program: Final Evaluation Report 30 and laws that addressed the status of women (46/87 SGs; 24/25 IPs; 7/7 GOs; 5/5 GEs; 13/13 Donors; 7/11 NAF; 4/5 USAID). This context is an opportunity to continue the work with women parliamentarians, building their capacities and strengthening the various parliamentarian groups and committees that are relevant for advancing gender equity. The laws passed are not implemented fully.25 Forty-five percent of the 153 respondents interviewed saw this as an opportunity for future gender programming to support and strengthen implementation mechanisms. Similarly, 26 percent of respondents noted that women’s rights advocates have identified other areas where laws are lacking, and this is an opportunity to support the advocacy of civil society with parliamentarians. Thirty-seven percent of respondents also mentioned opportunities to enhance the capacity of government departments directly tasked with implementing women-specific policies to deliver on gender sensitive policies and laws. Relevant departments include the WDD, the P&D Department, and the Finance Department. Respondents also identified the Commissions on the Status of Women, which respond to the concerns of women and also serve as watchdogs, monitoring the progress of the government on national and international gender commitments, as additional targets for capacity building. Women’s Economic Empowerment Women’s economic empowerment is an important pathway to ensuring gender equity with the well￾documented spillover benefits of improving women’s decision-making in the family and enhancing the nutritional and educational outcomes of their children.26,27 More women in Pakistan are entering the workforce, driven by poverty but also by the rise in female literacy and educational attainment. Unfortunately, most of this employment growth is concentrated in the informal economy and in agriculture, areas that offer low-paid and insecure employment. Seventy-six percent of 153 respondents interviewed mentioned opportunities to improve the capacity of women by enhancing non-traditional but high-value skills, improving access to markets, growing women’s enterprises, supporting women’s entry into non-traditional sectors (e.g., the IT and science, technology, engineering, and math fields), adding value to traditional sectors (e.g., agriculture), and improving women’s access to affordable transportation so they can access jobs. Vibrant Civil Society and Strong Women’s Movement The growing acceptance of women’s rights and a supportive women’s movement have strengthened civil society, noted senior management of AF. More young people have joined CSOs and are active in making their voices heard. These managers saw a larger civil society with a more visible presence in diverse sectors and geographic regions as an opportunity to advance the women’s rights and gender equality agenda. They also believed that the growing use of social media offers an opportunity to expand outreach and awareness among diverse groups, including young women and men. Eighty-seven percent of respondents viewed the government’s recently increasing willingness to promote women’s empowerment, coupled with donor commitment, as an opportunity to leverage and 25 Ibid. 26 World Bank, World Development Report: Gender Equality and Development (2012): http://wdronline.worldbank.org/includes/imp_images/book_pdf/WDR_2012.pdf 27 Afiya S. Zia, A Policy Framework for Women’s Equal Rights, Study for the National Commission on the Status of Women (Pakistan, 2010). Gender Equity Program: Final Evaluation Report 31 engage civil society. The Sustainable Development Goals28 (SDGs) agreed to by all UN member states in September 2015 include gender as a cross-cutting component, with a specific stand-alone goal (Goal 5). Countries, including Pakistan, are integrating the SDGs into their development plans, and will soon be setting targets to monitor their progress against each goal. Seventy-eight percent of respondents believed an opportunity exists for sustained programming that promotes synergy between the three stakeholder groups (government, donors, and CSOs), strategically advocating for closer collaboration between government and civil society and extending coverage to areas most in need of development (84/87 SGs; 21/25 IPs; 6/7 GOs; 3/5 GEs; 9/13 Donors; 5/11 NAF; 5/5 USAID). “There is more acceptance of women’s rights in our society now. For future gender programming there is a lot of space and opportunities to work on gender issues.” – Implementing partner, Quetta Key Findings: Priorities for Future Programming The evaluation explored priorities for future programming and triangulated priorities through a variety of questions. This section reports on findings collated and analyzed from questions that asked about key game-changing options for promoting women’s empowerment, engaging men as supporters of gender equity, changing the narrative around gender issues, and selecting the 3 top priorities from 13 options. The 153 responses to these questions were largely consistent and complemented each other. Eighty-six percent of respondents identified access to justice as one of the three most important game changers for gender programming, 72 percent cited women’s economic empowerment, 44 percent identified women’s representation in decision-making bodies, and 39 percent selected GBV. The remainder of this section elaborates the findings for these top four priorities. Priority 1: Access to Justice A large majority (82 percent) of respondents, and a majority of each respondent type, identified access to justice as a key priority for future programming (73/87 SGs; 19/25 IPs; 5/7 GOs; 5/5 GEs; 13/13 Donors; 6/11 NAF; 4/5 USAID). Policy and legislation enactment, reform, and implementation: Fifty-three percent of respondents (58 percent of women and 46 percent of men) identified policy and legislation enactment, reform, and implementation as a priority area for gender programming to address access to justice (44/87 SGs; 13/25 IPs; 2/7 GOs; 4/5 GEs; 9/13 Donors; 4/11 NAF; 2/5 USAID). Surprisingly, a smaller proportion of NAF members and government officials than other types of respondents saw this as a top priority area for gender programming. The evaluation team was not able to go back to these respondents to explore the rationale for this result. Respondents noted that there has been little progress in enacting legislation and reforming existing laws (such as amendments to the infamous Hudood Ordinance29 or ensuring stricter penalties for crimes 28 In 2015, 190 world leaders committed to 17 Sustainable Development Goals (SDGs) seeking to end extreme poverty and inequality and fix climate change. 29 Zia-ul-Haq initiated a process of Islamization by introducing discriminatory legislation against women such as the set of Hudood Ordinances and the Qanun-e-Shahadat Order (Law of Evidence Order). He banned women from participating and from being spectators of sports and promoted purdah. He suspended all fundamental rights guaranteed in the Constitution that had been adopted in 1973, including the right to be free of discrimination on the basis of sex. Gender Equity Program: Final Evaluation Report 32 against women that are covered by the Pakistan Penal Code).30 Partly this is due to the inadequate capacity of legislators. More work is required to institutionalize and formalize these processes so that new legislators can also be sensitized and supported to further the legislative agenda for women’s rights. Sub-grantees, gender experts, NAF, donors, and implementing partner respondents were keen to see existing laws implemented now that many of the major hurdles to accessing rights, i.e., lack of relevant policies or legislation, have been removed. The wave of women-friendly legislation enacted over the last decade needs supporting mechanisms so that women at provincial and sub-provincial levels can avail themselves of their rights and access justice. Respondents agreed that the people responsible for implementing laws were often unaware of existing laws because information, especially recent legislation, does not filter down to the law enforcement agencies or the judiciary, especially at the provincial and sub-provincial levels. The law enforcement agencies and the judiciary are therefore unable to register complaints under these laws. Similarly, many lawyers and the district bar councils remain unaware of the new laws and are not able to offer legal support and guidance to women who approach them for advice and help. A corresponding lack of awareness and information on the part of communities, especially women, creates a situation where there is no demand for invoking these laws to seek justice. Improving information and awareness of these laws among seekers and dispensers of justice is one step in improving access to justice.31 “More advocacy is needed to inform women about legislation to improve their access to the right instruments and information.” – Donor “Pressure groups are needed to push government to implement the rules and laws already made.” – Group interview, sub-grantees, Karachi Enhanced access to justice: Thirty-three percent of respondents (31 percent of women and 36 percent of men) identified enhancing access to justice in addition to ensuring that laws are enacted and implemented as a key priority. While the two categories are similar, the survey presented them separately. Respondents saw them both as key drivers of women’s empowerment and redressal for GBV (29/87 SGs; 19/25 IPs; 3/7 GOs; 5/5 GEs; 1/13 Donors; 2/11 NAF; 2/5 USAID). In addition to the findings noted above, respondents saw a need to improve the rules and procedures for accessing justice which are cumbersome and complicate women’s access to justice. While this is true of all legal processes in Pakistan, respondents, especially sub-grantees who worked with the law enforcement and justice systems, were well aware that for female litigants these are unfriendly if not hostile spaces. In addition to dealing with time-consuming delays, women unfamiliar with public spaces are vulnerable to being misguided, misinformed, and exploited. Beyond the need for sensitization and awareness, a donor respondent noted that increasing the numbers of women on the police force and improving implementation mechanisms, such as having one window of operations for women to access legal services, would go a long way in enabling women to get the services they need and removing the discriminatory barriers that they struggle against in their search for justice. 30 The Pakistan Penal Code 1860 addresses a number of forms of violence against women including assault (sections 354 and 354A), rape (section 375), criminally detaining or intimidating women (sections 496A and 506), and insulting a woman’s modesty (section 509). 31 Weiss, Moving Forward with the Legal Empowerment of Women in Pakistan. Gender Equity Program: Final Evaluation Report 33 Priority 2: Women’s Economic Empowerment Seventy-two percent of respondents (68 percent of men and 80 percent of women) identified women’s economic empowerment as a priority for future programming (59/87 SGs; 18/25 IPs; 5/7 GOs; 8/13 Donors; 2/5 GEs; 11/1 NAF; 3/5 USAID). This priority area combines 5 of the 13 alternative priority areas presented to respondents:  Increased employment of women in government and in the private sector (identified by 22 percent of 146 respondents)  Enhance value added and off-farm participation of women in agriculture (identified by 18 percent of 146 respondents)  Supporting women’s small, medium, and large enterprises (financial and physical capital) (identified by 15 percent of 146 respondents)  Enhanced market-driven technical and vocational education (identified by 9 percent of 146 respondents)  Ownership of physical assets, including property (identified by 8 percent of 146 respondents) Respondents noted that few women are employed in the public sector despite the 10 percent quota for women.32 Employment conditions will improve when more women join the workforce, as many hesitate to join the public or the private sector since the workplaces are heavily male-dominated and a female employee may often be the only woman in the room. Respondents noted that the majority of women in the workforce work in the agriculture sector. Therefore, strategies for increasing their incomes will include improving their ability to generate more income from their on-farm activities or gaining the skills to engage in off-farm economic activities. Respondents also noted that women’s entrepreneurship has the potential for growth. As owners of businesses, women have flexibility to manage their reproductive and productive roles. Respondents noted that the low employment among women with college degrees33 also points to a need for promoting their involvement in small and medium enterprises (SMEs) where they would not only earn income, but also become owners of physical and capital assets. Whether it is through supporting women’s SMEs, adding value to their work, ensuring employment in the public or private sectors, or enhancing skills that offer employability and returns, women’s financial empowerment is seen as a key factor in empowering women in their social lives as well. It offers stability and security and creates opportunities for women to change their lives and those of their families. Priority 3: Ensuring Women’s Representation in Decision-Making in Government Forty-eight percent of respondents identified ensuring women’s representation in decision-making in government (in ministries, departments, committees, and autonomous bodies) as an important priority area for future programming (30/87 SGs; 15/25 IPs; 4/5 GEs; 9/13 Donors; 3/7 GOs; 6/11 NAF; 3/5 USAID). Respondents acknowledged the presence of women in the national and provincial assemblies, but emphasized the need to enhance women’s presence at all decision-making levels in government. They 32 Jawad Syed, “Women and small business entrepreneurship in Pakistan,” in International Research Handbook on Successful Women Entrepreneurs, eds. Sandra L. Fielden and Marilyn J. Davidson (Cheltenham & New York: Edward Elgar Publishing, 2010), 117-131. 33 Yasmin Zaidi and Shujaat Farooq, Women’s Economic Participation and Empowerment in Pakistan – Status Report 2016 (Islamabad: UN Women Pakistan, 2016). Gender Equity Program: Final Evaluation Report 34 attributed the success in passing progressive legislation for women in recent years to the presence of a large number of women in Parliament and also as chairs and members of parliamentary committees who collaborated across party lines. Women’s representation in all decision-making bodies is important not only for addressing women’s concerns, but also for adding women’s perspectives to polices and decisions. Such representation will have a trickle-down effect along several dimensions, especially if there is more than a single token woman on a decision-making forum: it will encourage more women to aspire and join such forums and make decision-making more gender-responsive by incorporating the needs and concerns of women. Respondents suggested that the 33 percent representation of women on all public sector committees mandated (but not yet implemented) by the government of Punjab was a good example. Similarly, the Balochistan government has 33 percent representation of women in local government, and that has contributed to many women becoming members of district and sub-district councils. Respondents were also aware that such representation should go beyond numbers and should result in effective representation of women’s issues and concerns. Quotas fulfill an important need in bringing women to the fore, but there should be transparent criteria so that women who are nominated are informed, active, and ready to play an effective role. Of course they may require training and capacity enhancement, but again this needs to be specific and targeted to the roles they play. Ensuring women’s increased representation in decision-making is strategic, since only by being part of these processes will women gain experience and learn how to be effective. Respondents cited the example of the 33 percent of seats reserved for women in the Local Government Act of 2002—which led to approximately 36,000 women joining local councils—as a game-changing intervention since many of these women went on to join the legislative assemblies and also to contest elections as independent candidates or as members of a political party. Priority 4: Gender-Based Violence against Women and Girls Forty percent of respondents (36 percent of women and 46 percent of men) identified GBV as a critical area for future programming (31/87 SGs; 10/25 IPs; 4/7 GOs; 3/5 GEs; 4/13 Donors; 5/11 NAF; 1/5 USAID). Respondents noted that GBV was a major impediment to women’s being able to realize their rights and linked prevention and response to GBV directly to women’s empowerment. Women’s empowerment can be ensured only if GBV is addressed at all levels, at home, in public spaces, and at work. Respondents appreciated that GEP properly addressed GBV and saw this emphasis as correct because it is a crucial contributor to achieving the ultimate objective of empowering women. These respondents were emphatic that the scale of GBV—not only its physical manifestation but also the psychological, sexual, and emotional violence—makes it an important area for continued programming attention. Respondents noted that one act of GBV is enough to send a message to all women, and one GBV survivor who is denied justice is also an example of the futility of trying to change the social attitudes that allow such acts to occur with impunity. Even the 20 percent of respondents who did not identify GBV as one of the three most important priorities also recognized its importance, but they believed that programming to address access to justice, economic opportunities, education, and implementation of laws addressed the factors that can prevent GBV and shape responses to it. For example, women who have a secure means of income may be better placed to take a stand against GBV. Similarly, implementing and enforcing laws so GBV survivors can expect an efficient and effective response from relevant authorities will deter perpetrators of GBV. “If there is justice, there will be no GBV.” – Sub-grantee, female, Islamabad Gender Equity Program: Final Evaluation Report 35 Respondents also noted that GBV prevention and response are among the most important challenges in the human rights arena. Access to justice, social protection, and ensuring representation of women in decision-making bodies address GBV by creating an environment that does not tolerate such violence. Addressing the root causes of violence—such as poverty, patriarchal norms, and gender bias in curricula—and improving policy and legislation can indirectly address GBV. They noted that creating awareness among youth, especially young men and adolescents, was an important strategy for preventing GBV. Donor Mapping USAID/Pakistan’s OSG provided the evaluation team with a matrix that summarized results of a gender/social inclusion donor mapping exercise it conducted just prior to the evaluation. The mapping shows that most donors either address gender as a cross-cutting component of their larger programs or mainstream gender in their programs to an extent. Few donors, if any, have dedicated programs for enhancing gender equity and women’s empowerment, and none of these programs reach the scale of GEP in terms of funding. Interventions focus on women-friendly legislation and policies and improving women’s access to livelihoods, income, education, and skill enhancement. The evaluation team interviewed 13 donors. The findings from these interviews correspond to the donor mapping (Annex 9). Respondents explained that gender programming competes with other areas for scarce development funding. Nevertheless, the SDGs raised the profile of gender programming by defining a stand-alone gender goal—Goal 5: achieve gender equality and empower all women and girls— and integrating gender across many of the other SDGs. Pakistan will be preparing country-specific plans to achieve these goals, and the government may well be more amenable to adopting good practices and allocating resources for the advancement of women’s rights as a result. The international funding/lending entities such as the Asian Development Bank and the World Bank fund large infrastructure and social projects that have mandatory gender components, but these components are often small and not well resourced. It may be possible to collaborate with government and CSOs to support interventions that leverage such large projects (such as roads, power, energy, and communications) to promote gender equity. Conclusions 1. Pakistan has put in place many of the women’s machineries necessary to address gender equity and women’s empowerment. To make these machineries effective, however, ministries and line departments focused on women need to be better resourced; government departments need to be effectively sensitized to gender issues; and both women’s and men’s skills and leadership capacities need to be enhanced so they can more effectively engage in local governments and advocate for action to improve the lives of women. 2. Pakistan has also passed many women-friendly laws that can potentially enhance women’s access to justice. Additional effort is required, however, to advocate for additional laws and policies; strengthen, and in some cases establish, implementation mechanisms for existing laws; and strengthen the capacities of local governments and the judiciary to understand the law and respond to violations of women’s rights at the district level. 3. Respondents also believed that donors and the government need to go beyond rhetoric to meaningfully prioritize gender programming; commit to a sustained and consistent effort; and promote coordination among actors in the gender space (donors, government, civil society, and NGOs) to increase capacity and expand outreach across sectors and geography. Gender Equity Program: Final Evaluation Report 36 4. The growing acceptance of women’s rights and a supportive women’s movement, coupled with the SDGs that emphasize gender issues, create an opportunity for donors and others to allocate scarce funding to gender programming. 5. Enhancing women’s access to justice emerged as a high priority for future programming. Programming priorities related to access to justice include sensitizing and building the capacity of legislators, men and women, to develop appropriate policies, enact legislation, reform existing laws, and develop rules for implementation. It is also important to educate communities, lawyers, law enforcement agencies, and the judiciary about the laws protecting women’s rights and access to justice and create an environment where women can easily access justice and feel safe exercising their rights. Increasing women’s representation and decision-making in government, and building their skills and leadership capacities to be effective, will help advance access to justice. 6. Women’s economic empowerment is also an important priority with substantial positive spillover effects on women’s social lives, stability, security, and the wellbeing of their families. Appropriate programming responses include improving women’s skills to add value to current economic activities and move into new areas as well. Promoting entrepreneurship, especially among the large numbers of underemployed college-educated women, may be a promising avenue. 7. Programming to combat GBV is a high programming priority not only as an end in itself but also because it contributes to other important priorities, such as women’s economic empowerment. However, programming to address GBV need not be direct since addressing the gender dimensions of access to justice, policy and law, economic opportunity, and education can indirectly contribute to preventing GBV and shaping appropriate responses. OVERARCHING CONCLUSIONS 1. Other than GEP, there is no stand-alone, well-funded, gender-focused program in Pakistan with flexibility and a five-year duration. Other donors address gender through smaller components embedded in other programs. The evaluation findings demonstrate the need and potential for well-funded and sustained programming focused on women’s empowerment. 2. The needs and opportunities for future programming identified by the respondents are relevant and account for the Pakistan context and the existing capacities of potential implementing partners, CSOs, government, and other stakeholders that can be leveraged for greater impact. The need to work with government departments, to expand collaboration with government beyond the WDDs and work with others that may have a larger impact on women’s lives, is valid. Similarly, the need to address the many problems associated with accessing justice and prioritizing gender so that it receives the attention and resources it needs to make change visible and sustainable is relevant. Multiple opportunities exist that can support efforts for women’s empowerment. These include the presence of the WDDs (in all the provinces) and CSWs, as well as women parliamentarians and a vibrant civil society that is increasingly more aware and ready to take on discriminatory behaviors and attitudes that impact women. Devolution of power to the provinces has created opportunities for sustained donor programming at the provincial and sub-provincial levels. 3. The four future programmatic priorities identified by respondents, i.e., access to justice, women’s economic empowerment, women’s representation in decision-making, and GBV, offer Gender Equity Program: Final Evaluation Report 37 a comprehensive framework for advancing women’s empowerment in Pakistan. As noted by a leading advocate of women’s rights, effectively addressing just the first two priorities would change women’s lives dramatically in Pakistan. 4. GBV remains a priority for future work. It can be addressed directly through response mechanisms such as helplines, medical and legal aid, and shelters. The factors that contribute to GBV—such as justice delayed or denied; misogynistic and patriarchal attitudes and norms; reinforcement of gender stereotypes, especially negative stereotypes, in media and curricula (in educational institutions and training institutes); and poverty and lack of economic opportunities for women—are areas that require attention for GBV prevention. 5. Findings suggest a number of areas as cross-cutting components that would support the four programmatic areas noted. These include: o Changing educational curricula to incorporate gender and human rights. Respondents saw this as particularly important to influence the minds of young boys and men to become more gender sensitive and aware.34 o Focusing on enhancing women’s leadership and capacities and incorporating appropriate activities in all interventions. Women should be supported to take charge of the change that improves their lives, whether big or small. o Including men as allies and champions, but without having them take leadership away from women. Men have an important role to play in supporting women as they move forward. However, it is necessary to strike a balance between engaging them as allies and allowing them to speak, act, and make decisions on behalf of women. o Working with diverse stakeholders (e.g., government; religious leaders, including women and religious minorities; and women with disabilities) to support gender programming across sectors (e.g., trade, finance, private sector, and non-traditional sectors). RECOMMENDATIONS 1. USAID should include in future gender programming the four programmatic areas of GEP: access to justice, women’s economic empowerment, ensuring women’s representation in decision-making in government, and GBV prevention and response. 2. USAID should include policy and legislative advocacy for promoting gender-responsive socioeconomic change as a specific program objective in future activities. 3. AF and other stakeholders should consider taking an integrated approach to access to justice programming, focusing on both demand and supply side interventions (communities as well as law enforcement agencies). 4. USAID’s future gender programming should tap into the potential of social media and mass media to raise awareness and understanding of gender, particularly among youth. 34 Changing educational curricula was particularly noted in response to the question of involving men as supporters for women’s empowerment and game-changing interventions. Gender Equity Program: Final Evaluation Report 38 5. GUCA is a responsive, flexible model that USAID should consider for future programming that involves disbursing many small grants to local organizations. Local knowledge, expertise, and networks will often make a local organization best suited to managing the grants. However, a local organization may need support in the form of targeted capacity building or mentoring by a partner with experience managing USAID grant programs to effectively manage the grant program. Especially when disbursing sub-grants to small organizations, grant mechanisms that provide up-front payment instead of reimbursing grantees for incurred expenses will be easier to manage and more consistent with sub-grantees’ financial capacities. A flexibly implemented FOG will be more suitable than a simplified grant mechanism. 6. National level implementing partners should devolve grant making administration and financial functions to the provincial level. This could potentially enhance program effectiveness since implementing partner staff would be more familiar with potential sub-grantee organizations’ expertise and capacities. 7. USAID should review branding requirements in light of security risks and the increasing resistance in Pakistan to gender programming. Consider waiving branding requirements if necessary, training grantees in branding requirements, and pre-approving designs for banners and other materials to reduce delays in implementation. 8. Future gender programming should focus on engaging men as allies and male champions without undermining women’s leadership. 9. PGCN should be strengthened at the provincial and sub-provincial levels. USAID should consider supporting a strong and well-resourced provincial secretariat for the PGCN. The PGCN has a strong potential for sustaining the achievements of GEP and continuing the engagement of sub-grantees, thereby creating the space and drive for more gender-responsive development. 10. USAID should include policy and legislative advocacy for promoting gender-responsive socioeconomic change as a specific program objective in future programming. In light of the 18th Amendment, this should be targeted to the national and provincial legislatures, ministries, and departments. Such support could go to organizations already working at these levels. 11. Future USAID gender programming should focus strongly on women’s economic empowerment because of its vital role in improving the social position of women at home and in society. LESSONS LEARNED 1. Achieving results in gender equity requires a long-term investment. Short-term activities with small funding do not yield good results. Behavior change and social change require repetitive activities with ongoing monitoring and community buy-in to show progress. 2. Gender-based violence prevention requires a long-term effort since it must influence social, attitudinal, and behavior change. Initiatives that respond to the needs of GBV survivors (such as shelters, medical and psycho-social treatment facilities, and helplines) can be established relatively quickly but should be paired with long-term prevention campaigns. 3. GEP’s geographic reach allowed it to support gender programming throughout the country but also diluted results. A less ambitious geographic outreach and more focused approach to the Gender Equity Program: Final Evaluation Report 39 type, duration, and size of sub-grants would likely have resulted in a more productive, measurable, and sustainable gender equity program. 4. Working with small organizations at the community level is an important aspect of gender programming, particularly for awareness raising, advocacy initiatives, and long-term buy-in and commitment. 5. Programmatic monitoring plays a crucial role in successful on-the-job capacity building of sub￾grantees. The monitoring aspects of GEP focused more on financial and administrative issues than on programmatic aspects. Capacity building efforts need well-defined goals and a structured monitoring plan starting with a baseline assessment. 6. The political and social environments are different in the various provinces and regions of the country. The differences in cultural and social views and practices, the degree of government support, and the levels of development and access to services are among the many factors to consider when designing and managing gender programming. The same design will not have equal results in all provinces or regions. Gender Equity Program: Final Evaluation Report 40 ANNEXES Annex 1: Evaluation Statement of Work Gender Equity Program: Final Evaluation Report 41 Gender Equity Program: Final Evaluation Report 42 Gender Equity Program: Final Evaluation Report 43 Gender Equity Program: Final Evaluation Report 44 Gender Equity Program: Final Evaluation Report 45 Gender Equity Program: Final Evaluation Report 46 Gender Equity Program: Final Evaluation Report 47 Gender Equity Program: Final Evaluation Report 48 Gender Equity Program: Final Evaluation Report 49 Gender Equity Program: Final Evaluation Report 50 Gender Equity Program: Final Evaluation Report 51 Annex 2: Assignment Work Plan Gender Equity Program: Final Evaluation Report 52 Gender Equity Program: Final Evaluation Report 53 Gender Equity Program: Final Evaluation Report 54 Gender Equity Program: Final Evaluation Report 55 Gender Equity Program: Final Evaluation Report 56 Gender Equity Program: Final Evaluation Report 57 Gender Equity Program: Final Evaluation Report 58 Gender Equity Program: Final Evaluation Report 59 Gender Equity Program: Final Evaluation Report 60 Gender Equity Program: Final Evaluation Report 61 Gender Equity Program: Final Evaluation Report 62 Gender Equity Program: Final Evaluation Report 63 Gender Equity Program: Final Evaluation Report 64 Gender Equity Program: Final Evaluation Report 65 Gender Equity Program: Final Evaluation Report 66 Gender Equity Program: Final Evaluation Report 67 Gender Equity Program: Final Evaluation Report 68 Gender Equity Program: Final Evaluation Report 69 Annex 3: Summary of GEP Objectives and Thematic Areas TABLE 4: NUMBER OF GRANTS BY OBJECTIVE AND THEMATIC AREA GEP Objectives Themes Number of Grants Status of Themes 1. Enhancing gender equity by expanding women’s access to justice and women’s human rights Rapid response for flood relief 4 Closed Institutional strengthening and capacity building of critical partners and stakeholders, and GBV-focused media campaigns 3 Closed Strengthening district bar associations and setting up facilitation kiosks in courts 13 Ongoing Social awareness to combat GBV 1 Supporting Ministry of Law, Justice and Human Rights in devising and putting forth women centered laws and capacity building of prosecutors, court officials and police on women centered laws 1 Ongoing Strengthening district bar associations and setting up facilitation kiosks in courts 10 Ongoing Engaging with minority groups on women’s rights 1 In process 2. Increasing women’s empowerment by expanding knowledge of their rights and opportunities to exercise their rights in the workplace, community and home Rapid response for flood relief 4 Closed Institutional strengthening and capacity building of critical partners and stakeholders, and GBV-focused media campaigns 3 Closed Promoting improved livelihoods of informal sector workers 12 Closed Advocacy campaign for allocation and rehabilitation of women-friendly spaces 8 Ongoing Supporting female graduates access into information technology (IT) sectors through internship programs 7 Ongoing Innovative approaches to promoting women’s economic empowerment 7 Ongoing Engaging with Chambers of Commerce and Industries to promote and advance women entrepreneurship 10 In Process Engaging with public and private institutes/NGOs/CSOs to promote and increase women economic empowerment (with different organizations) 8 Ongoing 3. Combating gender-based violence (GBV) Rapid response for flood relief 8 Closed Institutional strengthening and capacity building of critical partners and stakeholders, and GBV-focused media campaigns 2 Closed Combating GBV 8 Closed Support to private shelters 8 Closed Economic rehabilitation 6 Closed Provision of psycho-social support 2 Closed Advocacy for prevention of GBV 13 Closed Establishing private helplines 13 Closed Gender Equity Program: Final Evaluation Report 70 GEP Objectives Themes Number of Grants Status of Themes Combating GBV through provision of seamless service delivery hubs 23 Ongoing Countering GBV through research, data strengthening and standardization 1 Ongoing Engagement with 3 public universities 3 Ongoing 4. Strengthening the capacity of Pakistani organizations that advocates for gender equity, women’s empowerment and elimination of GBV Rapid response for flood relief 1 Closed Institutional strengthening and capacity building of critical partners and stakeholders, and GBV-focused media campaigns 7 Closed Engagement with 14 public universities 14 In process Training women & men lawyers to provide legal services to women, including minority women 8 In process Gender Equity Program: Final Evaluation Report 71 Annex 4: Data Collection Detail TABLE 5: DATA COLLECTION SUMMARY BY RESPONDENT TYPE, LOCATION, AND METHOD Respondent Type Respondent Locations Data Collection Method Sampling Number of Interviews Number of Respondents Grantee Grantee staff: interviews with grantee executive and group interviews with the balance of staff. Peshawar, Karachi, Quetta, Lahore, Islamabad KII Random 6 6 GI Random 17 70 Survey Purposive 23 71 Provincial and Federal Government Officials Staff in WDDs Peshawar, Karachi, Quetta, Lahore, Islamabad KII Purposive 4 4 GI Random 1 3 Survey Purposive 5 7 Donors Gender leads Islamabad KII Purposive 8 8 GI Random 2 5 Survey Purposive 10 13 Gender Experts Individual experts Islamabad, Lahore, Peshawar, Karachi, Quetta KII Purposive 5 5 Survey Purposive 5 5 Implementing Partners COPs of AF and TAF. Also from AF: COO, Grants Manager, M&E staff, financial staff and GEP field staff Islamabad, Lahore, Peshawar, Karachi, Quetta KII Purposive 8 8 GI Random 5 17 Survey Purposive 13 23 Hub for Survivors and Victims of GBV Grantee organizations Lahore, Karachi GI Purposive 2 11 Lahore, Karachi Survey Purposive 2 11 NAF Event NAF leadership, Cycle 7 grantees Islamabad GI Convenience 1 11 Survey Purposive 1 11 USAID/Pakistan staff GEP AOR, OSG representatives Islamabad KII Purposive 3 3 GI Purposive 1 2 Total KII 34 63 153 GI 29 Survey 63 146 Gender Equity Program: Final Evaluation Report 72 TABLE 6: SAMPLED GRANTS BY OBJECTIVE AND THEME Objectives / Themes / Sub-grantee Organizations Number of Grants Start Date End Date Objective 1 – Access to Justice and Rights 9 Expanding women’s access to justice (strengthening district bar associations and setting up facilitation kiosks in district court) 5 Aghaz Foundation Trust 1 1-Dec-15 30-Apr-17 Association for Integrated Development (AID) 1 16-Jan-15 31-May-16 Center for Peace & Justice for Pakistan (CPJP) 1 1-Dec-15 30-Apr-17 Social Justice & Development Institute (SJDI) 1 1-Dec-15 30-Apr-17 Taraqee Foundation (TF) 1 1-Nov-15 31-Mar-17 Institutional strengthening and capacity building of critical partners and stakeholders, and GBV focused media campaign 3 Change Thru Empowerment (CTE) 1 1-Aug-11 30-Sep-12 Legal Rights Forum (LRF) 1 1-Oct-11 31-Oct-12 Youth Resource Council (YRC) 1 1-Aug-11 31-Aug-12 Supporting ministries and institutions in devising and putting forth centered laws and capacity building of prosecutors, court officials, and police on women centered laws 1 Ministry of Law Justice and Human Rights (MoLJHR) 1 15-Dec-14 15-Aug-16 Objective 2 – Women’s Empowerment 26 Advocacy Campaign for Allocation & Rehabilitation of Women-Friendly Spaces 6 Balochistan Social Development Program (BSDP) 1 1-Feb-15 30-Jun-16 Community Initiatives for Development in Pakistan (CIDP) 1 16-Jan-15 31-May-16 Drugs and Narcotics Educational Services for Humanity (DANESH) 1 1-Feb-15 30-Jun-16 Ghazi Social Welfare Association (GSWA) 1 16-Jan-15 31-May-16 Nana Sahib Development Society (NSDS) 1 1-Nov-15 31-Mar-17 PIRBHAT Women Development Society 1 16-Jan-15 31-May-16 Combating gender-based violence (engagement with public sector universities) 2 Allama Iqbal Open University (AIOU) 1 1-Nov-15 31-Mar-17 Gender Equity Program: Final Evaluation Report 73 Objectives / Themes / Sub-grantee Organizations Number of Grants Start Date End Date Sardar Bahadur Khan Women’s University (SBKWU) 1 1-Nov-15 31-Dec-16 Enhancing women workers’ awareness of their rights and labor laws 4 HomeNet Pakistan 1 1-Apr-14 30-Oct-15 Human Resource Development Network (HRDN) 1 1-Jul-14 31-Dec-15 National Organization for Working Communities (NOWC) 1 1-Jul-14 31-Dec-15 Sewa Development Trust Sindh (SDTS) 1 1-Dec-14 30-Apr-16 Innovative approaches promoting women’s economic empowerment 4 AIDS Awareness Society (AAS) 1 16-Jan-15 31-May-16 College of Tourism and Hotel Management (COTHM) 1 1-May-15 30-Apr-16 JOBS International Pvt LTD 1 16-Jan-15 31-May-16 Management Development Foundation (MDF) 1 1-Dec-15 30-Apr-17 Institutional strengthening and capacity building of critical partners and stakeholders, and GBV focused media campaign 2 Black Box Sounds 1 25-May-11 31-May-12 Individualland Pakistan 1 1-Feb-11 29-Feb-12 Promoting improved livelihood for informal sector workers 5 Balochistan Environmental and Educational Journey (BEEJ) 1 1-Jul-14 29-Feb-16 Khwendo Kor (KK) 1 1-Jan-15 31-May-16 Pakistan Hoslamand Khawateen Network (PHKN) 1 1-Jul-14 31-Dec-15 Sindh Development Society (SDS) 1 1-Jul-14 29-Feb-16 Women Development Foundation (WDF) 1 1-Jul-14 29-Feb-16 Supporting female graduates’ access into information technology sector through internship programs 3 Attitude, Skills, Knowledge Development Pvt LMT. (ASK) 1 16-Jan-15 31-May-16 Empowerment Thru Creative Integration (ECI) 1 1-Feb-15 31-May-16 Virtual University 1 16-Jan-15 31-May-16 Objective 3 – Combating Gender-Based Violence 23 Gender Equity Program: Final Evaluation Report 74 Objectives / Themes / Sub-grantee Organizations Number of Grants Start Date End Date Combating GBV by supporting shelters, helpline, and providing other critical services 8 Acid Survivors Foundation (ASF) 1 15-Jan-13 14-Mar-14 Basic Education for Awareness, Reforms, and Empowerment (BEFARe) 1 1-Nov-12 31-Dec-13 Blue Veins (BV) 1 1-Nov-12 30-Sep-13 Center for Peace and Development 1 11-Jan-12 30-Sep-13 Human Dignity Society 1 16-Nov-12 31-Dec-13 Interactive Resource Center (IRC) 1 1-Nov-11 31-Dec-13 PANAH 1 1-Nov-12 31-Oct-13 War Against Rape (WAR) 1 1-Nov-12 31-Dec-13 Combating GBV through research, data strengthening and standardization 1 SEBCON 1 1-Jun-11 30-Sep-12 Combating GBV trough provision of seamless service delivery Hubs 9 Binte Fatima Old Home Trust 1 4-Feb-16 30-Jun-17 Center for Legal Aid Assistance & Settlement (CLAAS) 1 1-Dec-14 31-Jul-16 Dastak Charitable Trust (DCT) 1 16-Jan-15 31-Jul-16 National Maternal, Newborn and Child Health Program (NMNCH) 1 9-Feb-16 31-Mar-17 Panah Trust 1 4-Feb-16 30-Jun-17 Roshni Research & Development Welfare Organization (RRDWO) 1 8-Feb-16 30-Jun-17 Sindh Human Rights Commission (SHRC) 1 8-Feb-16 30-Jun-17 Sudhaar Society (SS) 1 1-Jan-15 31-Aug-16 War Against Rape (WAR) 1 8-Feb-16 30-Jun-17 Institutional strengthening and capacity building of critical partners and stakeholders 1 Ministry of Women Development, Shaheed Benazir Bhutto Center, Quetta 1 1-Feb-11 31-Aug-11 Strengthening public sector universities and departments/institutions 1 Women Development Department, Sindh 1 11-Mar-13 31-May-14 Supporting Four Months Campaign on GBV aimed at men 3 AHSAS PK 1 12-Dec-11 11-Jun-12 Gender Equity Program: Final Evaluation Report 75 Objectives / Themes / Sub-grantee Organizations Number of Grants Start Date End Date Grass-Root Organization for Human Development (GODH) 1 12-Dec-12 11-May-12 Sindh Development Society (SDS) 1 1-Jul-14 29-Feb-16 Objective 4 – Strengthening Capacity of Critical Partners 3 Institutional strengthening and capacity building of critical partners and stakeholders, and GBV focused media campaign 2 Human Resource Development Center, Institute of Management Science 1 1-Aug-11 30-Jan-12 Strengthening Participatory Organization (SPO) 1 2-Jul-12 28-Feb-15 Strengthening public sector universities and departments/institutions 1 Provincial Commission on Status of Women 1 11-Mar-13 10-Apr-14 Grand Total 61 Gender Equity Program: Final Evaluation Report 76 TABLE 7: SUB-GRANTEES INTERVIEWED Date of Interview Objective Organization Locations Organization Type June 3, 2016 Objective 2 – Women’s Empowerment Nana Sahib Development Society (NSDS) Balochistan NGO June 4, 2016 Objective 2 – Women’s Empowerment Drugs and Narcotics Educational Services for Humanity (DANESH) Balochistan NGO June 3, 2016 Objective 2 – Women’s Empowerment Balochistan Social Development Program (BSDP) Balochistan NGO June 3, 2016 Objective 3 – Combating Gender-Based Violence Human Dignity Society Balochistan NGO June 3, 2016 Objective 3 – Combating Gender-Based Violence Center for Peace and Development Balochistan NGO June 2, 2016 Objective 2 – Women’s Empowerment Sardar Bahadur Khan Women’s University (SBKWU) Balochistan Public Sector Organization/ Institution June 3, 2016 Objective 1 – Access to Justice & Rights Association for Integrated Development (AID) Balochistan NGO June 2, 2016 Objective 1 – Access to Justice & Rights Taraqee Foundation (TF) Balochistan NGO June 3, 2016 Objective 3 – Combating Gender-Based Violence Ministry of women development, Shaheed Benazir Bhutto Center, Quetta Balochistan Public Sector Organization/Institution June 3, 2016 Objective 1 – Access to Justice & Rights Change Thru Empowerment (CTE) Balochistan NGO June 3, 2016 Objective 3 – Combating Gender-Based Violence AHSAS PK Balochistan NGO Gender Equity Program: Final Evaluation Report 77 Date of Interview Objective Organization Locations Organization Type June 3, 2016 Objective 1 – Access to Justice & Rights Social Justice & Development Institute (SJDI) Balochistan NGO June 3, 2016 Objective 2 – Women’s Empowerment Balochistan Environmental and Educational Journey (BEEJ) Balochistan NGO June 8, 2016 Objective 3 – Combating Gender-Based Violence Acid Survivors Foundation (ASF) ICT NGO June 9, 2016 Objective 2 – Women’s Empowerment JOBS International Pvt LTD ICT Private Institution/ Organization June 9, 2016 Objective 2 – Women’s Empowerment Allama Iqbal Open University (AIOU) ICT Public Sector Organization/Institution June 14, 2016 Objective 1 – Access to Justice & Rights Ministry of Law Justice and Human Rights (MoLJHR) ICT Public Sector Organization/Institution June 3, 2016 Objective 3 – Combating Gender-Based Violence Blue Veins (BV) KP NGO June 3, 2016 Objective 2 – Women’s Empowerment Pakistan Hoslamand Khawateen Network (PHKN) KP NGO June 3, 2016 Objective 3 – Combating Gender-Based Violence Basic Education for Awareness, Reforms, and Empowerment (BEFARe) KP NGO June 2, 2016 Objective 1 – Access to Justice & Rights Youth Resource Council (YRC) KP NGO June 2, 2016 Objective 2 – Women’s Empowerment Khwendo Kor (KK) KP NGO June 3, 2016 Objective 4 – Strengthening Capacity of Critical Partners Provincial Commission on Status of Women KP Public Sector Organization/Institution Gender Equity Program: Final Evaluation Report 78 Date of Interview Objective Organization Locations Organization Type June 6, 2016 Objective 2 – Women’s Empowerment Black Box Sounds National Private Institution/ Organization June 8, 2016 Objective 4 – Strengthening Capacity of Critical Partners Strengthening Participatory Organization (SPO) National NGO May 25, 2016 Objective 2 – Women’s Empowerment Empowerment Thru Creative Integration (ECI) National Private Institution/ Organization June 9, 2016 Objective 3 – Combating Gender-Based Violence Interactive Resource Center (IRC) National Private Institution/ Organization June 9, 2016 Objective 2 – Women’s Empowerment Home Net Pakistan National NGO June 9, 2016 Objective 2 – Women’s Empowerment Individual and Pakistan National NGO June 9, 2016 Objective 4 – Strengthening Capacity of Critical Partners Human Resource Development Center, Institute of Management Science National Private Institution/ Organization June 20, 2016 Objective 3 – Combating Gender-Based Violence SEBCON National Private Institution/ Organization June 2, 2016 Objective 2 – Women’s Empowerment Attitude, Skills, Knowledge Development Pvt LMT. (ASK) Punjab Private Institution/ Organization June 1, 2016 Objective 3 – Combating Gender-Based Violence Center for Legal Aid Assistance & Settlement (CLAAS) Punjab NGO June 1, 2016 Objective 3 – Combating Gender-Based Violence Sudhaar Society (SS) Punjab NGO June 2, 2016 Objective 2 – Women’s Empowerment AIDS Awareness Society (AAS) Punjab NGO Gender Equity Program: Final Evaluation Report 79 Date of Interview Objective Organization Locations Organization Type June 3, 2016 Objective 2 – Women’s Empowerment Virtual University Punjab Public Sector Organization/Institution June 3, 2016 Objective 3 – Combating Gender-Based Violence Grass-Root Organization for Human Development (GODH) Punjab NGO June 7, 2016 Objective 1 – Access to Justice & Rights Aaghaz Foundation Trust Punjab NGO June 3, 2016 Objective 2 – Women’s Empowerment Human Resource Development Network (HRDN) Punjab NGO June 3, 2016 Objective 3 – Combating Gender-Based Violence Dastak Charitable Trust (DCT) Punjab NGO June 3, 2016 Objective 3 – Combating Gender-Based Violence PANAH Sindh NGO May 25, 2016 Objective 3 – Combating Gender-Based Violence Sindh Development Society (SDS) Sindh NGO June 3, 2016 Objective 3 – Combating Gender-Based Violence Roshni Research & Development Welfare organization Sindh NGO June 3, 2016 Objective 3 – Combating Gender-Based Violence War Against Rape (WAR) Sindh NGO June 3, 2016 Objective 3 – Combating Gender-Based Violence National Maternal, Newborn and Child Health Program (NMNCH) Sindh Public Sector Organization/Institution June 3, 2016 Objective 3 – Combating Gender-Based Violence Bint e Fatima Old Home Trust Sindh Private Institution/ Organization June 3, 2016 Objective 3 – Combating Gender-Based Violence War Against Rape (WAR) Sindh NGO Gender Equity Program: Final Evaluation Report 80 Date of Interview Objective Organization Locations Organization Type June 1, 2016 Objective 3 – Combating Gender-Based Violence Sindh Human Rights Commission (SHRC) Sindh Public Sector Organization/Institution June 3, 2016 Objective 3 – Combating Gender-Based Violence Panah Trust Sindh NGO May 25, 2016 Objective 2 – Women’s Empowerment National Organization for Working Communities (NOWC) Sindh NGO June 3, 2016 Objective 1 – Access to Justice & Rights Center for Peace & Justice for Pakistan (CPJP) Sindh NGO May 25, 2016 Objective 2 – Women’s Empowerment College of Tourism and Hotel Management (COTHM) Sindh Private Institution/ Organization June 3, 2016 Objective 1 – Access to Justice & Rights Legal Rights Forum (LRF) Sindh NGO June 2, 2016 Objective 2 – Women’s Empowerment Women Development Foundation (WDF) Sindh NGO June 2, 2016 Objective 3 – Combating Gender-Based Violence Women Development Department, Sindh Sindh Public Sector Organization/Institution May 25, 2016 Objective 2 – Women’s Empowerment Management Development Foundation (MDF) Sindh NGO May 25, 2016 Objective 2 – Women’s Empowerment Ghazi Social Welfare Association (GSWA) Sindh NGO May 25, 2016 Objective 2 – Women’s Empowerment PIRBHAT Women Development Society Sindh NGO May 25, 2016 Objective 2 – Women’s Empowerment Community Initiatives for Development in Pakistan (CIDP) Sindh NGO Gender Equity Program: Final Evaluation Report 81 Date of Interview Objective Organization Locations Organization Type May 25, 2016 Objective 2 – Women’s Empowerment Sindh Development Society (SDS) Sindh NGO May 25, 2016 Objective 2 – Women’s Empowerment Sewa Development Trust Sindh (SDTS) Sindh NGO Gender Equity Program: Final Evaluation Report 82 TABLE 8: STAKEHOLDERS INTERVIEWED Date of Interview Location Respondent Type Organization May 23, 2016 Islamabad Donor UNDP May 24, 2016 Islamabad Donor UNICEF UNICEF May 23, 2016 Islamabad Donor JICA June 10, 2016 Islamabad Donor World Bank June 14, 2016 Islamabad Donor EU June 7, 2016 Islamabad Donor Political Section, German Embassy Development Section, German Embassy June 14, 2016 Islamabad Donor DFID June 7, 2016 Islamabad Donor UN Women UN Women June 9, 2016 Islamabad Donor ADB June 15, 2016 Islamabad Donor UNFPA June 7, 2016 Islamabad Gender Expert N/A May 25, 2016 Islamabad Gender Expert N/A June 2, 2016 Lahore Gender Expert N/A June 2, 2016 Karachi Gender Expert N/A June 4, 2016 Quetta Gender Expert N/A May 25, 2016 Islamabad Implementing partner GEP May 25, 2016 Islamabad Implementing partner AF May 25, 2016 Islamabad Implementing partner TAF May 25, 2016 Islamabad Implementing partner GEP May 25, 2016 Islamabad Implementing partner GEP June 16, 2016 Islamabad Implementing partner GEP June 23, 2016 Islamabad Implementing partner GEP Gender Equity Program: Final Evaluation Report 83 Date of Interview Location Respondent Type Organization June 1, 2016 Lahore Implementing partner GEP June 3, 2016 Lahore Implementing partner GEP June 3, 2016 Lahore Implementing partner GEP June 3, 2016 Lahore Implementing partner GEP June 3, 2016 Lahore Implementing partner GEP June 1, 2016 Karachi Implementing partner GEP June 2, 2016 Karachi Implementing partner GEP June 2, 2016 Karachi Implementing partner GEP June 2, 2016 Karachi Implementing partner GEP June 2, 2016 Karachi Implementing partner GEP June 1, 2016 Quetta Implementing partner GEP June 3, 2016 Quetta Implementing partner GEP June 3, 2016 Quetta Implementing partner GEP June 3, 2016 Quetta Implementing partner GEP June 1, 2016 Quetta Implementing partner GEP June 1, 2016 Quetta Implementing partner GEP June 1, 2016 Quetta Implementing partner GEP June 1, 2016 Quetta Implementing partner GEP June 16, 2016 Islamabad USAID USAID June 16, 2016 Islamabad USAID USAID June 16, 2016 Islamabad USAID USAID June 17, 2016 Islamabad USAID USAID June 24, 2016 Islamabad USAID USAID Gender Equity Program: Final Evaluation Report 84 Annex 5: Data Collection Instruments Gender Equity Program (GEP) Final Evaluation Group Interview (GI) Guide – Hubs In August 2010, USAID/Pakistan awarded a US$40 million five-year Cooperative Agreement to the Aurat Foundation (AF) to implement its Gender Equity Program (GEP), which has been extended until August 15, 2017. GEP aims to close the gender gap in Pakistan by actively supporting women’s economic, political and social development. It gives grants to government and non-government entities. USAID/Pakistan is conducting an evaluation of GEP through its contractor, Management Systems International (MSI), to identify lessons learned, effective implementation modalities, and strategic opportunities for follow-on programming. Your insights are important to this study and will be invaluable in designing future gender programs. All your comments will be kept confidential. [If recording please ask interviewee: Do we have your permission to record the interview as it helps in recalling the discussion. Yes: No: ] With your permission I/we would like to proceed with the interview. [Oral consent] Questions Prompts Key Evaluation Question 1. What are the key lessons learned (successes, challenges, opportunities, and weaknesses) from implementation of GEP that could inform the design of a follow-on project? (Note: We are seeking here responses that can lead to lessons learned for future programming. So we need SIGNIFICANT responses.) 1.1 Please describe the activities of GEP in the areas of prevention and response to gender-based violence. Hub creation / Hub strengthening, health response, legal response, any other? 1.2 To what extent, if at all, has GEP made efforts to ensure the provision of seamless service delivery to victims of GBV? What, if anything, could be changed to make this more effective? Structure of Hub; diagram it if possible 1.3 Were there any external events that impacted gender equity or your ability to execute GEP? How, if at all, did GEP (or your gender programming) respond to these external factors? Floods, change of government, policy changes such as the 18th Amendment, violence, peace and security, terrorism, etc.? 1.4 In your opinion, did the hubs face any CRITICAL constraints? Explain. 1.5 Please describe the relationship, if any, the Hub has with the government? If supportive, please specify how and in what ways. Gender Equity Program: Final Evaluation Report 85 Questions Prompts 1.6 How can the sustainability of Hubs be ensured after GEP ends? Key Evaluation Question 2. To what extent, and how, did the grants-under-cooperative￾agreement mechanism affect implementation and results? 2.1 Please indicate for each of the following statements whether you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree. Then, please explain why you feel that way. We will do this for each statement, one-by-one. Hand out statement sheet to respondent(s) so they can read each statement and indicate their degree of agreement. Be sure to fill out ID information for each respondent and to record explanations for each statement to which you receive a response. Key Evaluation Question 3. What are the areas of greatest need, priority, or opportunity for new gender programming going forward? (taking into consideration the work of other donors) 3.1 What do you think are the CRITICAL opportunities and constraints at the national, provincial, or local level that can influence future gender programming? 3.2 If we had to choose three key recommendations on which to focus, what would be the key game changers that could be done relatively easily and would really send a strong signal about the importance of women’s empowerment? 3.3 Please describe the role you think men can play in successful women’s empowerment programs? In your opinion, how could we engage with men so that they become active supporters of gender equity and help change the narrative? Gender Equity Program: Final Evaluation Report 86 Questions Prompts 3.4 In your opinion, which 3 of the 13 gender programming areas listed below are the most important for achieving gender equity? Why? (For each of the three most important areas). a. Youth and adolescent sexual and reproductive health rights b. Policy and legislation: enactment, implementation and reform c. Enhance access to justice d. Enhance women’s participation and visibility in higher education especially in sciences and ICT e. Prevention and response to Gender-Based Violence (GBV); (Country specific: Violence against Women and Girls) f. Ensuring women’s representation in decision making bodies in all government ministries, departments, committees, and autonomous bodies g. Increased employment of women in government and in private sector h. Enhanced market driven technical and vocational education i. Ownership of physical assets including property j. Enhance value added and off-farm participation of women in agriculture k. Supporting women’s small, medium and large enterprises (financial and physical capital) l. Gender responsive budgeting m. Social protection of women Provide handout to each respondent in the group to indicate three priority areas 3.5 For those that selected GBV as one of their three priority areas: to what extent, if at all, do you feel that the GBV focus of GEP successfully contributed to gender equity? For those that did not include GBV as one of their three priority areas: As you may know, GBV was a major focus of GEP. Please describe why you have chosen not to include GBV in your top three priorities. Are there any other comments you would like to share with the evaluation team? “Thank you for your time and efforts to provide us with the information we need for the final evaluation of GEP.” Gender Equity Program: Final Evaluation Report 87 Gender Equity Program (GEP) Final Evaluation Statements – GI HUBs Respondent ID: Please read the following statements and check the appropriate box Statements Strongly Agree Agree Neither Agree or Disagree Disagree Strongly Disagree #4 The umbrella sub-grants model enables coordination among Hub grantees. 1 2 3 4 5 #5 A Pakistani organization is a better choice than an international organization for awarding grants to Pakistani organizations. 1 2 3 4 5 #7 The specific grants received under GEP responded to the needs of the sub-grantees. 1 2 3 4 5 Gender Equity Program: Final Evaluation Report 88 Gender Equity Program (GEP) Final Evaluation Group Interview (GI) Guide – National Advisory Forum (NAF) In August 2010, USAID/Pakistan awarded a US$40 million five-year Cooperative Agreement to the Aurat Foundation (AF) to implement its Gender Equity Program (GEP), which has been extended until August 15, 2017. GEP aims to close the gender gap in Pakistan by actively supporting women’s economic, political and social development. It gives grants to government and non-government entities. USAID/Pakistan is conducting an evaluation of GEP through its contractor Management Systems International (MSI) to identify lessons learned, effective implementation modalities and strategic opportunities for follow-on programming. Your insights are important to this study and will be invaluable in designing future gender programs. All your comments will be kept confidential. [If recording please ask interviewee: Do we have your permission to record the interview as it helps in recalling the discussion. Yes: No: ] With your permission I/we would like to proceed with the interview. [Oral consent] Questions Prompts Key Evaluation Question 1. What are the key lessons learned (successes, challenges, opportunities, and weaknesses) from implementation of GEP that could inform the design of a follow-on project? [Note: Here we are seeking responses that can lead to lessons learned for future programming decisions: So we seek SIGNIFICANT responses.] 1.1 What, if any, influence did GEP have on legislation/policy formation and/or policy reform/change for gender equity? Any specific examples? 1.2 Follow up Question: What role, if any, do you think, the National Advisory Forum (NAF) has in influencing policy and legislation in Pakistan? 1.3 What, if any, contributions has GEP made to social change in terms of gender equity? Has GEP produced any lessons to inform future programming? Explain. 1.4 How, if at all, has the National Advisory Forum (NAF) contributed to GEP? 1.5 Do you think the National Advisory Forum (NAF) should continue after GEP ends? If yes, how and why? 1.6 In your opinion, what if anything, will be the legacy of GEP Has GEP been successful? If yes, how? If not, why not? Gender Equity Program: Final Evaluation Report 89 Questions Prompts Key Evaluation Question 2. To what extent, and how, did the grants-under-cooperative￾agreement mechanism affect implementation and results? 2.1 GEP is led by a Pakistani NGO. What is your opinion on this choice? Having a Pakistani NGO or a foreign entity implementing a project? 2.2 Please indicate for each of the following statements whether you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree. Then, please explain why you feel that way. We will do this for each statement, one-by-one. Hand out statement sheet to respondent(s) so they can read each statement and indicate their degree of agreement. Be sure to fill out ID information for each respondent and to record explanations for each statement to which you receive a response. Key Evaluation Question 3. What are the areas of greatest need, priority, or opportunity for new gender programming going forward? (taking into consideration the work of other donors) 3.1 What do you think are the CRITICAL opportunities and constraints at the national, provincial, or local level that can influence future gender programming? 3.2 If we had to choose three key recommendations on which to focus, what would be the key game changers that could be done relatively easily and would really send a strong signal about the importance of women’s empowerment? 3.3 Please describe the role you think men can play in successful women’s empowerment programs? In your opinion, how could we engage with men so that they become active supporters of gender equity and help change the narrative? Gender Equity Program: Final Evaluation Report 90 Questions Prompts 3.4 In your opinion, which 3 of the 13 gender programming areas listed below are the most important for achieving gender equity? Why? (For each of the three most important areas). a. Youth and adolescent sexual and reproductive health rights b. Policy and legislation: enactment, implementation, and reform c. Enhance access to justice d. Enhance women’s participation and visibility in higher education, especially in sciences and ICT e. Prevention and response to Gender-Based Violence (GBV); (Country specific: Violence against Women and Girls) f. Ensuring women’s representation in decision making bodies in all government ministries, departments, committees, and autonomous bodies g. Increased employment of women in government and in private sector h. Enhanced market-driven technical and vocational education i. Ownership of physical assets including property j. Enhance value added and off-farm participation of women in agriculture k. Supporting women’s small, medium and large enterprises (financial and physical capital) l. Gender responsive budgeting m. Social protection of women Provide handout to each respondent in the group to indicate three priority areas 3.5 For those that selected GBV as one of their three priority areas: to what extent, if at all, do you feel that the GBV focus of GEP successfully contributed to gender equity? For those that did not include GBV as one of their three priority areas: As you may know, GBV was a major focus of GEP. Please describe why you have chosen not to include GBV in your top three priorities. Are there any other comments you would like to share with the evaluation team? “Thank you for your time and efforts to provide us with the information we need for the final evaluation of GEP.” Gender Equity Program: Final Evaluation Report 91 Gender Equity Program (GEP) Final Evaluation Statements – National Advisory Forum (NAF) Respondent ID: Please read the following statement and check the appropriate box Statements Strongly Agree Agree Neither Agree or Disagree Disagree Strongly Disagree # 1 There is a positive and synergistic impact from the umbrella sub￾grants model. 1 2 3 4 5 # 3 The umbrella sub-grants model is more effective than direct implementation by implementing partners for gender equity programs in Pakistan. 1 2 3 4 5 # 5 A Pakistani organization is a better choice than an international organization for awarding grants to Pakistani organizations. 1 2 3 4 5 # 8 The umbrella sub-grants model is the most effective to promote gender equity. 1 2 3 4 5 # 9 The umbrella sub-grants model encouraged implementation of innovative gender equity solutions. 1 2 3 4 5 # 10 The umbrella sub-grants model grants cycles and themes are the best ways to categorize and distribute grant funds. 1 2 3 4 5 Gender Equity Program: Final Evaluation Report 92 Gender Equity Program (GEP) Final Evaluation Key Informant Interview (KII) Guide - Donors In August 2010, USAID/Pakistan awarded a US$40 million five-year Cooperative Agreement to the Aurat Foundation (AF) to implement its Gender Equity Program (GEP), which has been extended until August 15, 2017. GEP aims to close the gender gap in Pakistan by actively supporting women’s economic, political and social development. It gives grants to government and non-government entities. USAID/Pakistan is conducting an evaluation of GEP through its contractor Management Systems International (MSI) to identify lessons learned, effective implementation modalities and strategic opportunities for follow-on programming. Your insights are important to this study and will be invaluable in designing future gender programs. All your comments will be kept confidential. [If recording please ask interviewee: Do we have your permission to record the interview as it helps in recalling the discussion. Yes: No: ] With your permission we would like to proceed with the interview. [Oral Consent] Questions Prompts Key Evaluation Question 1. What are the key lessons learned (successes, challenges, opportunities, and weaknesses) from implementation of GEP that could inform the design of a follow-on project? No sub-questions asked of donor for Key Evaluation Question 1 Key Evaluation Question 2. To what extent, and how, did the grants-under-cooperative￾agreement mechanism affect implementation and results? No sub-questions asked of donor for Key Evaluation Question 2 Key Evaluation Question 3. What are the areas of greatest need, priority, or opportunity for new gender programming going forward? (taking into consideration the work of other donors) 3.1 What do you think are the CRITICAL opportunities and constraints at the national, provincial, or local level that can influence future gender programming 3.2 If we had to choose three key recommendations on which to focus, what would be the key game changers that could be done relatively easily and would really send a strong signal about the importance of women’s empowerment? Gender Equity Program: Final Evaluation Report 93 Questions Prompts 3.3 Please describe the role you think men can play in successful women’s empowerment programs? In your opinion, how could we engage with men so that they become active supporters of gender equity and help change the narrative? 3.4 In your opinion, which 3 of the 13 gender programming areas listed below are the most important for achieving gender equity? Why? (For each of the three most important areas). a. Youth and adolescent sexual and reproductive health rights b. Policy and legislation: enactment, implementation, and reform c. Enhance access to justice d. Enhance women’s participation and visibility in higher education, especially in sciences and ICT e. Prevention and response to Gender-Based Violence (GBV); (Country specific: Violence against Women and Girls) f. Ensuring women’s representation in decision making bodies in all government ministries, departments, committees, and autonomous bodies g. Increased employment of women in government and in private sector h. Enhanced market-driven technical and vocational education i. Ownership of physical assets including property j. Enhance value added and off-farm participation of women in agriculture k. Supporting women’s small, medium, and large enterprises (financial and physical capital) l. Gender responsive budgeting m. Social protection of women Provide handout to each respondent in the group to indicate three priority areas Gender Equity Program: Final Evaluation Report 94 Questions Prompts 3.5 For those that selected GBV as one of their three priority areas: to what extent, if at all, do you feel that the GBV focus of GEP successfully contributed to gender equity? For those that did not include GBV as one of their three priority areas: As you may know, GBV was a major focus of GEP. Please describe why you have chosen not to include GBV in your top three priorities. Are there any other comments you would like to share with the evaluation team? “Thank you for your time and efforts to provide us with the information we need for the final evaluation of GEP.” Gender Equity Program: Final Evaluation Report 95 Gender Equity Program (GEP) Final Evaluation Key Informant Interview (KII) Guide – Gender Expert In August 2010, USAID/Pakistan awarded a US$40 million five-year Cooperative Agreement to the Aurat Foundation (AF) to implement its Gender Equity Program (GEP), which has been extended until August 15, 2017. GEP aims to close the gender gap in Pakistan by actively supporting women’s economic, political and social development. It gives grants to government and non-government entities. USAID/Pakistan is conducting an evaluation of GEP through its contractor Management Systems International (MSI) to identify lessons learned, effective implementation modalities and strategic opportunities for follow-on programming. Your insights are important to this study and will be invaluable in designing future gender programs. All your comments will be kept confidential. [If recording please ask interviewee: Do we have your permission to record the interview as it helps in recalling the discussion. Yes: No:] With your permission we would like to proceed with the interview. [Oral consent] Questions Prompts Key Evaluation Question 1. What are the key lessons learned (successes, challenges, opportunities, and weaknesses) from implementation of GEP that could inform the design of a follow-on project? [Note to facilitator: We are seeking significant responses that can lead to lessons learned for future programming] 1.1 What, if any, contributions has GEP made to social change in terms of gender equity? Has GEP produced any lessons to inform future programming? Explain. 1.2 What, if any, influence did GEP have on legislation/policy formation and/or policy reform/change for gender equity? Can you give specific examples? 1.3 How, if at all, has the National Advisory Forum (NAF) contributed to GEP? 1.4 What role, if any, do you think, the National Advisory Forum (NAF) has in influencing policy and legislation in Pakistan? 1.5 Do you think the National Advisory Forum (NAF) should continue after GEP ends? If yes, how and why? Key Evaluation Question 2. To what extent, and how, did the grants-under-cooperative￾agreement mechanism affect implementation and results? 2.1 What other effective grants models are being used by other donors in Pakistan? Gender Equity Program: Final Evaluation Report 96 Questions Prompts 2.2 Please indicate for each of the following statements whether you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree. Then, please explain why you feel that way. We will do this for each statement, one-by-one. Hand out statement sheet to respondent(s) so they can read each statements and indicate their degree of agreement. Be sure to fill out ID information for each respondent and to record explanations for each statement to which you receive a response. Key Evaluation Question 3. What are the areas of greatest need, priority, or opportunity for new gender programming going forward? (taking into consideration the work of other donors) 3.1 What do you think are the CRITICAL opportunities and constraints at the national, provincial, or local level that can influence future gender programming? 3.2 If we had to choose three key recommendations on which to focus, what would be the key game changers that could be done relatively easily and would really send a strong signal about the importance of women’s empowerment? 3.3 Please describe the role you think men can play in successful women’s empowerment programs? In your opinion, how could we engage with men so that they become active supporters of gender equity and help change the narrative? Gender Equity Program: Final Evaluation Report 97 Questions Prompts 3.4 In your opinion, which 3 of the 13 gender programming areas listed below are the most important for achieving gender equity? Why? (For each of the three most important areas). a. Youth and adolescent sexual and reproductive health rights b. Policy and legislation: enactment, implementation, and reform c. Enhance access to justice d. Enhance women’s participation and visibility in higher education, especially in sciences and ICT e. Prevention and response to Gender-Based Violence (GBV); (Country specific: Violence against Women and Girls) f. Ensuring women’s representation in decision making bodies in all government ministries, departments, committees, and autonomous bodies g. Increased employment of women in government and in private sector h. Enhanced market-driven technical and vocational education i. Ownership of physical assets including property j. Enhance value added and off-farm participation of women in agriculture k. Supporting women’s small, medium, and large enterprises (financial and physical capital) l. Gender responsive budgeting m. Social protection of women Provide handout to respondents to indicate three priority areas. 3.5 For those that selected GBV as one of their three priority areas: to what extent, if at all, so you feel that the GBV focus of GEP successfully contributed to gender equity? For those that did not include GBV as one of their three priority areas: As you may know, GBV was a major focus of GEP. Please describe why you have chosen not to include GBV in your top three priorities. Are there any other comments you would like to share with the evaluation team? “Thank you for your time and efforts to provide us with the information we need for the final evaluation of GEP.” Gender Equity Program: Final Evaluation Report 98 Gender Equity Program (GEP) Final Evaluation Statements – Gender Expert Respondent ID: Please read the following statement and circle the appropriate number Statements Strongly Agree Agree Neither Agree or Disagree Disagree Strongly Disagree # 3 The umbrella sub-grants model is more effective than direct implementation by implementing partners for gender equity programs in Pakistan. 1 2 3 4 5 # 5 A Pakistani organization is a better choice than an international organization for awarding grants to Pakistani organizations. 1 2 3 4 5 “Thank you for your time and efforts to provide us with the information we need for our study.” Gender Equity Program: Final Evaluation Report 99 Gender Equity Program (GEP) Final Evaluation Key Informant Interview (KII) Guide – Government Officials In August 2010, USAID/Pakistan awarded a US$40 million five-year Cooperative Agreement to the Aurat Foundation (AF) to implement its Gender Equity Program (GEP), which has been extended until August 15, 2017. GEP aims to close the gender gap in Pakistan by actively supporting women’s economic, political and social development. It gives grants to government and non-government entities. USAID/Pakistan is conducting an evaluation of GEP through its contractor Management Systems International (MSI) to identify lessons learned, effective implementation modalities and strategic opportunities for follow-on programming. Your insights are important to this study and will be invaluable in designing future gender programs. All your comments will be kept confidential. [If recording please ask interviewee: Do we have your permission to record the interview as it helps in recalling the discussion. Yes: No:] With your permission I/we would like to proceed with the interview. [Oral consent] Questions Prompts Key Evaluation Question 1: What are the key lessons learned (successes, challenges, opportunities, and weaknesses) from implementation of GEP that could inform the design of a follow-on project? (Note: We are seeking here responses that can lead to lessons learnt for future programming. So we seek significant responses.) 1.1 What, if any, contributions has GEP made to social change in terms of gender equity? Has GEP produced any lessons to inform future programming? Explain. 1.2 What, if any, influence did GEP have on legislation/policy formation and/or policy reform/change for gender equity? Any specific examples? 1.3 In your opinion what, if anything, will be the legacy of GEP? (Has GEP been successful? If yes, how? If not, why not? ) Key Evaluation Question 2: To what extent, and how, did the grants-under-cooperative￾agreement mechanism affect implementation and results? Not applicable. No sub-questions asked of government officials for Key Evaluation Question 2 Key Evaluation Question 3: What are the areas of greatest need, priority, or opportunity for new gender programming going forward? (taking into consideration the work of other donors) 3.1 What do you think are the CRITICAL opportunities and constraints at the national, provincial, or local level that can influence future gender programming? Gender Equity Program: Final Evaluation Report 100 Questions Prompts 3.2 If we had to choose three key recommendations on which to focus, what would be the key game changers that could be done relatively easily and would really send a strong signal about the importance of women’s empowerment? 3.3 Please describe the role you think men can play in successful women’s empowerment programs? In your opinion, how could we engage with men so that they become active supporters of gender equity and help change the narrative? 3.4 In your opinion, which 3 of the 13 gender programming areas listed below are the most important for achieving gender equity? Why? (For each of the three most important areas). a. Youth and adolescent sexual and reproductive health rights b. Policy and legislation: enactment, implementation, and reform c. Enhance access to justice d. Enhance women’s participation and visibility in higher education, especially in sciences and ICT e. Prevention and response to Gender-Based Violence (GBV); (Country specific: Violence against Women and Girls) f. Ensuring women’s representation in decision making bodies in all government ministries, departments, committees, and autonomous bodies g. Increased employment of women in government and in private sector h. Enhanced market-driven technical and vocational education i. Ownership of physical assets including property j. Enhance value added and off-farm participation of women in agriculture k. Supporting women’s small, medium, and large enterprises (financial and physical capital) l. Gender responsive budgeting m. Social protection of women Provide handout to each respondent in the group to indicate three priority areas Gender Equity Program: Final Evaluation Report 101 Questions Prompts 3.5 For those that selected GBV as one of their three priority areas: to what extent, if at all, do you feel that the GBV focus of GEP successfully contributed to gender equity? For those that did not include GBV as one of their three priority areas: As you may know, GBV was a major focus of GEP. Please describe why you have chosen not to include GBV in your top three priorities. Are there any other comments you would like to share with the evaluation team? “Thank you for your time and efforts to provide us with the information we need for our study.” Gender Equity Program: Final Evaluation Report 102 Gender Equity Program (GEP) Final Evaluation Key informant interview (KII) Guide – Implementing Partner (TAF & Aurat Foundation) In August 2010, USAID/Pakistan awarded a US$40 million five-year Cooperative Agreement to the Aurat Foundation (AF) to implement its Gender Equity Program (GEP), which has been extended until August 15, 2017. GEP aims to close the gender gap in Pakistan by actively supporting women’s economic, political and social development. It gives grants to government and non-government entities. USAID/Pakistan is conducting an evaluation of (GEP) through its contractor Management Systems International (MSI) to identify lessons learned, effective implementation modalities and strategic opportunities for follow-on programming. Your insights are important to this study and will be invaluable in designing future gender programs. All your comments will be kept confidential. [If recording please ask interviewee: Do we have your permission to record the interview as it helps in recalling the discussion. Yes: No:] With your permission we would like to proceed with the interview. [Oral consent] Questions Prompts Key Evaluation Question 1. What are the key lessons learned (successes, challenges, opportunities, and weaknesses) from implementation of GEP that could inform the design of a follow-on project? [Note: We are seeking here responses that can lead to lessons learnt for future programming. So we seek significant responses.] 1.1 What, if any, influence did GEP have on legislation/policy formation and/or policy reform/change for gender equity? Any specific examples? 1.2 What, if any, contribution has GEP made to social change in terms of gender equity? What lessons, if any, emerged from GEP that could help improve future gender equity programs or inform the direction or focus of future programs? Explain. 1.3 In your opinion did GEP face any CRITICAL constraints? Explain. (For GI hubs replace GEP with hub; for grantees say "with your grant") 1.4 What have you learned, if anything, from any failures and successes in working under GEP? Gender Equity Program: Final Evaluation Report 103 Questions Prompts 1.5 In your opinion, did working with GEP significantly improve your organization’s capacity? In what way? What was particularly successful? What was particularly successful in the way in which your organization’s capacity was improved) 1.6 Were there any external events that impacted gender equity or your ability to execute GEP? How, if at all, did GEP (or your gender programming) respond to these external factors? Floods, change of government, policy changes such as the 18th Amendment, violence, peace and security, terrorism, etc.? 1.7 a. In your opinion, what if any, will be the legacy of GEP? b. Has GEP been successful? If yes, how? If not, why not? 1.8 Please describe the activities of GEP in the areas of prevention and response to gender￾based violence. Hub creation / Hub strengthening, Health response, Legal response, any other? 1.9 To what extent, if at all, has GEP made efforts to ensure the provision of seamless service delivery to victims of GBV? What, if anything, could be changed to make this more effective? (Prompts: structure of hub; diagram it if possible.) 1.10 In your opinion, how can the sustainability of hubs be ensured after GEP ends? 1.11 How, if at all, has the National Advisory Forum (NAF) contributed to GEP? 1.12 What role, if any, do you think, the National Advisory Forum (NAF) has in influencing policy and legislation in Pakistan? 1.13 How, if at all, has the role of the National Advisory Forum (NAF) evolved over the years? 1.14 a. In your opinion, is a mechanism like the National Advisory Forum (NAF) needed to provide vision and oversight for a program like GEP? b. In your opinion, is it the most effective mechanism for the role it performed? 1.15 Do you think the National Advisory Forum (NAF) should continue after GEP ends? If yes, how and why? 1.16 In your opinion, has GEP been able to build an effective network of supported organizations? Gender Equity Program: Final Evaluation Report 104 Questions Prompts 1.17 How does the Pakistan Gender Coalition Network (PGCN) function? Who coordinates and from where? Is it organized into sub-groups, e.g. by interest groups. 1.18 How, if at all, has the Pakistan Gender Coalition Network (PGCN) contributed to GEP? What is the role of PGCN in implementing GEP? 1.19 In your opinion, should the Pakistan Gender Coalition Network (PGCN) continue after GEP ends? If so why and how? Prompt: Specific suggestions or steps already undertaken? Key Evaluation Question 2. To what extent, and how, did the grants-under-cooperative￾agreement mechanism affect implementation and results? 2.1 a. Ask only of TAF What was your experience having a Pakistani NGO as an implementing partner? b. Ask TAF and Aurat Foundation Would you consider replicating the idea of a partnership between the Pakistani organization and an international organization? 2.2 What, if any, challenges have you faced in your partnerships? 2.3 How, if at all, did the umbrella sub-grants model adopted for GEP change Aurat Foundation? 2.4 In your opinion, what, if anything, could have worked better under an umbrella sub-grants model? Looking back, what, if anything, would you have done differently? 2.5 How has your experience been working under the GEP umbrella sub-grants model? In your opinion, what are the strengths and weaknesses of the umbrella sub-grants model in terms of design and implementation? What changes/additions, if any, would you recommend, especially in program design? 2.6 In your view, was the delegation of roles and responsibilities between the three entities in the project effective? (USAID, Aurat Foundation and The Asia Foundation (TAF)). Explain. Gender Equity Program: Final Evaluation Report 105 Questions Prompts 2.7 What was The Asia Foundation’s (TAF’s) role in implementing GEP? How, if at all, did the role evolve over time? 2.8 Please indicate for each of the following statements whether you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree. Then, please explain why you feel that way. We will do this for each statement, one-by-one. Hand out statement sheet to respondent(s) so they can read each statements and indicate their degree of agreement. Be sure to fill out ID information for each respondent and to record explanations for each statement to which you receive a response. Key Evaluation Question 3. What are the areas of greatest need, priority, or opportunity for new gender programming going forward? (taking into consideration the work of other donors) 3.1 What do you think are the CRITICAL opportunities and constraints at the national, provincial, or local level that can influence future gender programming? 3.2 If we had to choose three key recommendations on which to focus, what would be the key game changers that could be done relatively easily and would really send a strong signal about the importance of women’s empowerment? 3.3 Please describe the role you think men can play in successful women’s empowerment programs? In your opinion, how could we engage with men so that they become active supporters of gender equity and help change the narrative? Gender Equity Program: Final Evaluation Report 106 Questions Prompts 3.4 In your opinion, which 3 of the 13 gender programming areas listed below are the most important for achieving gender equity? Why? (For each of the three most important areas). a. Youth and adolescent sexual and reproductive health rights b. Policy and legislation: enactment, implementation, and reform c. Enhance access to justice d. Enhance women’s participation and visibility in higher education especially in sciences and ICT e. Prevention and response to Gender￾Based Violence (GBV); (Country specific: Violence against Women and Girls) f. Ensuring women’s representation in decision making bodies in all government ministries, departments, committees, and autonomous bodies g. Increased employment of women in government and in private sector h. Enhanced market-driven technical and vocational education i. Ownership of physical assets including property j. Enhance value added and off-farm participation of women in agriculture k. Supporting women’s small, medium, and large enterprises (financial and physical capital) l. Gender responsive budgeting m. Social protection of women Provide handout to each respondent in the group to indicate three priority areas 3.5 For those that selected GBV as one of their three priority areas: to what extent, if at all, do you feel that the GBV focus of GEP successfully contributed to gender equity? For those that did not include GBV as one of their three priority areas: As you may know, GBV was a major focus of GEP. Please describe why you have chosen not to include GBV in your top three priorities. Are there any other comments you would like to share with the evaluation team? “Thank you for your time and efforts to provide us with the information we need for the final evaluation of GEP.” Gender Equity Program: Final Evaluation Report 107 Gender Equity Program (GEP) Final Evaluation Statement – Implementation Partners (TAF, Aurat Foundation) Respondent ID: Please read the following statement and check the appropriate box Statements Strongly Agree Agree Neither Agree or Disagree Disagree Strongly Disagree #1 There is a positive and synergistic impact from the umbrella sub￾grants model. 1 2 3 4 5 #2 AF and USAID bureaucratic requirements reduced the efficiency of GEP implementation. 1 2 3 4 5 #3 The umbrella sub-grants model is more effective than direct implementation by implementing partners for gender equity programs in Pakistan. 1 2 3 4 5 #5 A Pakistani organization is a better choice than an international organization for awarding grants to Pakistani organizations. 1 2 3 4 5 #6 A grant under cooperative agreement mechanism is a better choice than a grant under contract mechanism. 1 2 3 4 5 #7 The specific grants received under GEP responded to the needs of the sub-grantees. 1 2 3 4 5 #8 The umbrella sub-grants model is the most effective to promote gender equity. 1 2 3 4 5 #10 The umbrella sub-grants cycles and themes are the best ways to categorize and distribute grant funds. 1 2 3 4 5 #11 Different grant mechanisms (FOG, cost reimbursements) under GEP were good choices for accomplishing sub-grantees objectives. 1 2 3 4 5 Gender Equity Program: Final Evaluation Report 108 Gender Equity Program (GEP) Final Evaluation Key Informant Interview (KII) Guide – GEP Sub-Grantee In August 2010, USAID/Pakistan awarded a US$40 million five-year Cooperative Agreement to the Aurat Foundation (AF) to implement its Gender Equity Program (GEP), which has been extended until August 15, 2017. GEP aims to close the gender gap in Pakistan by actively supporting women’s economic, political and social development. It gives grants to government and non-government entities. USAID/Pakistan is conducting an evaluation of GEP through its contractor Management Systems International (MSI) to identify lessons learned, effective implementation modalities and strategic opportunities for follow-on programming. Your insights are important to this study and will be invaluable in designing future gender programs. All your comments will be kept confidential. [If recording please ask interviewee: Do we have your permission to record the interview as it helps in recalling the discussion. Yes: No:] With your permission we would like to proceed with the interview. [Oral consent] Questions Prompts Key Evaluation Question 1. What are the key lessons learned (successes, challenges, opportunities, and weaknesses) from implementation of GEP that could inform the design of a follow-on project? [Note to facilitator: We are seeking significant responses that can lead to lessons learned for future programming] 1.1 What, if any, contributions has GEP made to social change in terms of gender equity? Has GEP produced any lessons to inform future programming? Explain. 1.2 What, if any, influence did GEP have on legislation/policy formation and/or policy reform/change for gender equity? Any specific examples? 1.3 In your opinion did GEP face any CRITICAL constraints? Explain. (For GI Hubs replace GEP with Hub; for grantees say "with your grant") 1.4 What have you learned, if anything, from any failures and successes in working under GEP? 1.5 Was your organization’s capacity significantly improved in working with GEP? What was particularly successful? Are grantees still working on gender issues in their respective areas? 1.6 Were there any external events that impacted gender equity or your ability to execute GEP? How, if at all, did GEP (or your gender programming) respond to these external factors? Floods, change of government, policy changes such as the 18th Amendment, violence, peace and security, terrorism, etc.? Gender Equity Program: Final Evaluation Report 109 Questions Prompts 1.7 Please describe the activities of GEP in the areas of prevention and response to gender-based violence. Hub creation / Hub strengthening, Health response, Legal response, any other? 1.8 To what extent, if at all, has GEP made efforts to ensure the provision of seamless service delivery to victims of GBV? What, if anything, could be changed to make this more effective? Structure of Hub; diagram it if possible 1.9 In your opinion, has GEP been able to build an effective network of supported organizations? 1.10 How does the Pakistan Gender Coalition Network (PGCN) function? Who coordinates and from where? Is it organized into sub-groups, e.g. by interest groups? 1.11 In your opinion, should the Pakistan Gender Coalition Network (PGCN) continue after GEP ends? If so why and how? Specific suggestions or steps already undertaken. 1.12 If you participated in a Pakistan Gender Coalition Network (PGCN) sub-group was it useful for you in doing your work? Why or why not? 1.13 In your opinion, what, if any, will be the legacy of GEP? Has GEP been successful? If yes, how? If not, why not? Key Evaluation Question 2. To what extent, and how, did the grants-under-cooperative-agreement mechanism affect implementation and results? 2.1 What other effective grants models are being used by other donors in Pakistan? 2.2 How was your working relationship with Aurat Foundation? 2.3 Please indicate for each of the following statements whether you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree. Then, please explain why you feel that way. We will do this for each statement, one-by-one. Hand out statement sheet to respondent(s) so they can read each statements and indicate their degree of agreement. Be sure to fill out ID information for each respondent and to record explanations for each statement to which you receive a response. Key Evaluation Question 3. What are the areas of greatest need, priority, or opportunity for new gender programming going forward? (taking into consideration the work of other donors) 3.1 What do you think are the CRITICAL opportunities and constraints at the national, provincial, or local level that can influence future gender programming? Gender Equity Program: Final Evaluation Report 110 Questions Prompts 3.2 If we had to choose three key recommendations on which to focus, what would be the key game changers that could be done relatively easily and would really send a strong signal about the importance of women’s empowerment? 3.3 Please describe the role you think men can play in successful women’s empowerment programs? In your opinion, how could we engage with men so that they become active supporters of gender equity and help change the narrative? 3.4 In your opinion, which 3 of the 13 gender programming areas listed below are the most important for achieving gender equity? Why? (For each of the three most important areas). a. Youth and adolescent sexual and reproductive health rights b. Policy and legislation: enactment, implementation and reform c. Enhance access to justice d. Enhance women’s participation and visibility in higher education especially in sciences and ICT e. Prevention and response to Gender-Based Violence (GBV); (Country specific: Violence against Women and Girls) f. Ensuring women’s representation in decision making bodies in all government ministries, departments, committees and autonomous bodies g. Increased employment of women in government and in private sector h. Enhanced market-driven technical and vocational education i. Ownership of physical assets including property j. Enhance value added and off-farm participation of women in agriculture k. Supporting women’s small, medium and large enterprises (financial and physical capital) l. Gender responsive budgeting m. Social protection of women Provide handout to each respondent in the group to indicate three priority areas Gender Equity Program: Final Evaluation Report 111 Questions Prompts 3.5 For those that selected GBV as one of their three priority areas: to what extent, if at all, do you feel that the GBV focus of GEP successfully contributed to gender equity? For those that did not include GBV as one of their three priority areas: As you may know, GBV was a major focus of GEP. Please describe why you have chosen not to include GBV in your top three priorities. Are there any other comments you would like to share with the evaluation team? “Thank you for your time and efforts to provide us with the information we need for the final evaluation of GEP.” Gender Equity Program: Final Evaluation Report 112 Gender Equity Program (GEP) Final Evaluation Statements - GEP Sub-Grantee Respondent ID: Please read the following statement and check the appropriate box Statements Strongly Agree Agree Neither Agree or Disagree Disagree Strongly Disagree # 2 There is an efficiency loss because of AF and USAID bureaucratic requirements. 1 2 3 4 5 # 5 A Pakistani organization is a better choice than an international organization for awarding grants to Pakistani organizations. 1 2 3 4 5 # 7 The specific grants received under GEP responded to the needs of the sub-grantees. 1 2 3 4 5 # 9 The umbrella sub-grants model encouraged implementation of innovative gender equity solutions. 1 2 3 4 5 # 10 The umbrella sub-grants cycles and themes are the best ways to categorize and distribute grant funds. 1 2 3 4 5 # 11 Different grant mechanisms (FOG, cost reimbursements) under GEP were good choices for accomplishing sub-grantees objectives. 1 2 3 4 5 “Thank you for your time and efforts to provide us with the information we need for our study.” Gender Equity Program: Final Evaluation Report 113 Gender Equity Program (GEP) Final Evaluation Key Informant Interview (KII) Guide -- USAID / Pakistan In August 2010, USAID/Pakistan awarded a US$40 million five-year Cooperative Agreement to the Aurat Foundation (AF) to implement its Gender Equity Program (GEP), which has been extended until August 15, 2017. GEP aims to close the gender gap in Pakistan by actively supporting women’s economic, political and social development. It gives grants to government and non-government entities. USAID/Pakistan is conducting an evaluation of GEP through its contractor Management Systems International (MSI) to identify lessons learned, effective implementation modalities and strategic opportunities for follow-on programming. Your insights are important to this study and will be invaluable in designing future gender programs. All your comments will be kept confidential. [If recording please ask interviewee: Do we have your permission to record the interview as it helps in recalling the discussion. Yes: No: ] With your permission I/we would like to proceed with the interview. [Oral consent] Questions Prompts Key Evaluation Question 1: What are the key lessons learned (successes, challenges, opportunities, and weaknesses) from implementation of GEP that could inform the design of a follow-on project? [Note to the Interviewer: We are seeking here responses that can lead to lessons learnt for future programming so we seek SIGNIFICANT responses] 1.1 What, if any, influence did GEP have on legislation/policy formation and/or policy reform/change for gender equity? Any specific examples? Any specific examples? 1.2 What, if any, contributions has GEP made to social change in terms of gender equity? Has GEP produced any lessons to inform future programming? Explain. 1.3 In your opinion did GEP face any CRITICAL constraints? Explain. 1.4 What have you learned, if anything, from any failures and successes in the GEP experience? 1.5 Were there any external events that impacted gender equity or your ability to execute GEP? How, if at all, did GEP (or your gender programming) respond to these external factors? Floods, change of government, policy changes such as the 18th Amendment, violence, peace and security, terrorism, etc.? 1.6 How, if at all, has the National Advisory Forum (NAF) contributed to GEP? Gender Equity Program: Final Evaluation Report 114 Questions Prompts 1.7 Do you think the National Advisory Forum (NAF) should continue after GEP ends? If yes, how and why? 1.8 In your opinion, has GEP been able to build an effective network of supported organizations? 1.9 In your opinion, should the Pakistan Gender Coalition Network (PGCN) continue after GEP ends? If so why and how? Specific suggestions or steps already undertaken. 1.10 In your opinion, what if any, will be the legacy of GEP? Has GEP been successful? If yes, how? If not, why not? Key Evaluation Question 2: To what extent, and how, did the grants-under-cooperative￾agreement mechanism affect implementation and results? 2.1 In your opinion, did the umbrella sub-grants model facilitate or hinder USAID’s gender programming objectives? Specific examples. 2.2 How would you compare the umbrella sub-grants model with other models in Pakistan? Looking back, what, if anything, would you have done differently? Specific examples. 2.3 In your opinion, how, if at all, did the umbrella sub-grants model adopted for GEP change Aurat Foundation? 2.4 What was your experience having a Pakistani NGO as an implementing partner? Would you consider replicating the idea of a partnership between the Pakistani organization and an international organization? 2.5 In your view was the delegation of roles and responsibilities between the three entities in the project effective? (USAID, Aurat Foundation and The Asia Foundation (TAF)). Explain. 2.6 How was your working relationship with Aurat Foundation? 2.7 Please indicate for each of the following statements whether you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree. Then, please explain why you feel that way. We will do this for each statement, one-by￾one. Hand out statement sheet to respondent(s) so they can read each statements and indicate their degree of agreement. Be sure to fill out ID information for each respondent and to record explanations for each statement to which you receive a response. Gender Equity Program: Final Evaluation Report 115 Questions Prompts Key Evaluation Question 3: What are the areas of greatest need, priority, or opportunity for new gender programming going forward? (taking into consideration the work of other donors) 3.1 What do you think are the CRITICAL opportunities and constraints at the national, provincial or local level that can influence future gender programming? 3.2 If we had to choose three key recommendations on which to focus, what would be the key game changers that could be done relatively easily and would really send a strong signal about the importance of women’s empowerment? 3.3 Please describe the role you think men can play in successful women’s empowerment programs? In your opinion, how could we engage with men so that they become active supporters of gender equity and help change the narrative? Gender Equity Program: Final Evaluation Report 116 Questions Prompts 3.4 In your opinion, which 3 of the 13 gender programming areas listed below are the most important for achieving gender equity? Why? (For each of the three most important areas). a. Youth and adolescent sexual and reproductive health rights b. Policy and legislation: enactment, implementation and reform c. Enhance access to justice d. Enhance women’s participation and visibility in higher education especially in sciences and ICT e. Prevention and response to Gender-Based Violence (GBV); (Country specific: Violence against Women and Girls) f. Ensuring women’s representation in decision making bodies in all government ministries, departments, committees and autonomous bodies g. Increased employment of women in government and in private sector h. Enhanced market-driven technical and vocational education i. Ownership of physical assets including property j. Enhance value added and off-farm participation of women in agriculture k. Supporting women’s small, medium and large enterprises (financial and physical capital) l. Gender responsive budgeting m. Social protection of women Provide handout to each respondent in the group to indicate three priority areas 3.5 For those that selected GBV as one of their three priority areas: to what extent, if at all, do you feel that the GBV focus of GEP successfully contributed to gender equity? For those that did not include GBV as one of their three priority areas: As you may know, GBV was a major focus of GEP. Please describe why you have chosen not to include GBV in your top three priorities. Are there any other comments you would like to share with the evaluation team? “Thank you for your time and efforts to provide us with the information we need for the final evaluation of GEP.” Gender Equity Program: Final Evaluation Report 117 Gender Equity Program (GEP) Final Evaluation Statement – USAID Respondent ID: Please read the following statement and check the appropriate box Statements Strongly Agree Agree Neither Agree or Disagree Disagree Strongly Disagree #1 There is a positive and synergistic impact from the umbrella sub-grants model. 1 2 3 4 5 #2 There is an efficiency loss because of AF and USAID bureaucratic requirements. 1 2 3 4 5 #3 The umbrella sub-grants model is more effective than direct implementation by implementing partners for gender equity programs in Pakistan. 1 2 3 4 5 #5 A Pakistani organization is a better choice than an international organization for awarding grants to Pakistani organizations. 1 2 3 4 5 #6 A grant under cooperative agreement mechanism is a better choice than a grant under contract mechanism. 1 2 3 4 5 #7 The specific grants received under GEP responded to the needs of the sub-grantees. 1 2 3 4 5 #8 The umbrella sub-grants model is the most effective to promote gender equity. 1 2 3 4 5 #9 The umbrella sub-grants model encouraged implementation of innovative gender equity solutions. 1 2 3 4 5 #10 The umbrella sub-grants cycles and themes are the best ways to categorize and distribute grant funds. 1 2 3 4 5 Gender Equity Program: Final Evaluation Report 118 Statements Strongly Agree Agree Neither Agree or Disagree Disagree Strongly Disagree #11 Different grant mechanisms (FOG, cost reimbursements) under GEP were good choices for accomplishing sub-grantees objectives. 1 2 3 4 5 Gender Equity Program: Final Evaluation Report 119 Annex 6: List of Project Documents Reviewed 1. 15 Grant documents 2. GEP project documents: a. Annual Report 2010- 2015 b. Quarterly Progress Reports (Oct 2010 to April 2016) c. Annual Work Plan 2013 to 2015 d. Result framework e. Pakinfo progress indicators f. Midterm evaluation report Gender Equity Program: Final Evaluation Report 120 Annex 7: Detailed Responses to Hypothesis Statements Hypothesis 1: There is a positive and synergistic impact from the umbrella sub-grants model. Hypothesis 2: AF and USAID bureaucratic requirements reduced the efficiency of GEP implementation. Strongly disagree Disagree Neither Agree nor Disagree Agree Strongly Agree IP (N=24) 0% 0% 4% 58% 38% USAID (N=5) 0% 0% 40% 60% 0% NAF (N=5) 0% 0% 20% 60% 20% Total (N=34) 0% 0% 12% 59% 29% 0% 20% 40% 60% 80% 100% Percent of respondents Strongly disagree Disagree Neither Agree nor Disagree Agree Strongly Agree SG (N=65) 3% 20% 20% 40% 17% IP (N=24) 0% 8% 17% 67% 8% USAID (N=5) 0% 0% 0% 80% 20% Total (N=94) 2% 16% 18% 49% 15% 0% 20% 40% 60% 80% 100% Percent of respondents Gender Equity Program: Final Evaluation Report 121 Hypothesis 3: The umbrella sub-grants model is more effective than direct implementation by implementing partners for gender equity programs in Pakistan. Hypothesis 4: The umbrella sub-grants model enables coordination among Hub grantees. Strongly disagree Disagree Neither Agree nor Disagree Agree Strongly Agree GE (N=5) 0% 20% 20% 20% 40% IP (N=24) 0% 4% 13% 71% 13% USAID (N=5) 20% 20% 40% 20% 0% NAF (N=5) 0% 0% 20% 20% 60% Total (N=39) 3% 8% 18% 51% 21% 0% 20% 40% 60% 80% 100% Percent of respondents Strongly disagree Disagree Neither Agree nor Disagree Agree Strongly Agree IP (N=3) 0% 0% 33% 0% 67% SG (N=11) 0% 0% 0% 55% 45% Total (N=14) 0% 0% 7% 43% 50% 0% 20% 40% 60% 80% 100% Percent of respondents Gender Equity Program: Final Evaluation Report 122 Hypothesis 5: A Pakistani organization is a better choice than an international organization for awarding grants to Pakistani organizations. Hypothesis 6: A grant under cooperative agreement mechanism is a better choice than a grant under contract mechanism. Strongly disagree Disagree Neither Agree nor Disagree Agree Strongly Agree SG (N=77) 3% 17% 12% 32% 36% GE (N=5) 0% 0% 20% 20% 60% IP (N=24) 0% 0% 25% 21% 54% USAID (N=5) 0% 20% 20% 40% 20% NAF (N=5) 0% 0% 20% 40% 40% Total (N=116) 2% 12% 16% 30% 41% 0% 20% 40% 60% 80% 100% Percent of respondents Strongly disagree Disagree Neither Agree nor Disagree Agree Strongly Agree IP (N=24) 0% 4% 4% 54% 38% USAID (N=5) 0% 60% 40% 0% 0% Total (N=29) 0% 14% 10% 45% 31% 0% 20% 40% 60% 80% 100% Percent of respondents Gender Equity Program: Final Evaluation Report 123 Hypothesis 7: The specific grants received under GEP responded to the needs of the sub-grantees. Hypothesis 8: The umbrella sub-grants model is the most effective to promote gender equity. Strongly disagree Disagree Neither Agree nor Disagree Agree Strongly Agree SG (N=77) 3% 5% 17% 58% 17% IP (N=24) 0% 0% 8% 67% 25% USAID (N=5) 0% 0% 40% 60% 0% Total (N=106) 2% 4% 16% 60% 18% 0% 20% 40% 60% 80% 100% Percent of respondents Strongly disagree Disagree Neither Agree nor Disagree Agree Strongly Agree IP (N=24) 0% 13% 13% 50% 25% USAID (N=5) 0% 40% 20% 40% 0% NAF (N=5) 0% 0% 20% 40% 40% Total (N=34) 0% 15% 15% 47% 24% 0% 20% 40% 60% 80% 100% Percent of respondents Gender Equity Program: Final Evaluation Report 124 Hypothesis 9: The umbrella sub-grants model encouraged implementation of innovative gender equity solutions. Hypothesis 10: The umbrella sub-grants cycles and themes are the best ways to categorize and distribute grant funds. Strongly disagree Disagree Neither Agree nor Disagree Agree Strongly Agree SG (N=66) 3% 8% 12% 58% 20% IP (N=4) 0% 0% 25% 75% 0% USAID (N=5) 0% 0% 0% 80% 20% Total (N=75) 3% 7% 12% 60% 19% 0% 20% 40% 60% 80% 100% Percent of respondents Strongly disagree Disagree Neither Agree nor Disagree Agree Strongly Agree SG (N=66) 2% 11% 20% 45% 23% IP (N=24) 0% 17% 8% 50% 25% USAID (N=5) 0% 20% 20% 60% 0% NAF (N=5) 0% 0% 0% 40% 60% Total (N=95) 1% 13% 17% 47% 22% 0% 20% 40% 60% 80% 100% Percent of respondents Gender Equity Program: Final Evaluation Report 125 Hypothesis 11: Different grant mechanisms (FOG, cost reimbursements) under GEP were good choices for accomplishing sub-grantees objectives. Strongly disagree Disagree Neither Agree nor Disagree Agree Strongly Agree SG (N=66) 12% 9% 24% 50% 5% IP (N=22) 0% 5% 36% 41% 18% USAID (N=5) 0% 20% 20% 60% 0% NAF (N=5) 0% 0% 40% 40% 20% Total (N=93) 9% 9% 27% 48% 8% 0% 20% 40% 60% 80% 100% Percent of respondents Gender Equity Program: Final Evaluation Report 126 Annex 8: Current Literature on Women’s Access to Justice; Gender￾Based Violence (GBV); Women, Peace, and Security (WPS); and Child, Early, and Forced Marriage (CEFM) in Pakistan ADB. (2009). Legal empowerment for women and disadvantaged groups: Finding of a project funded by ADB in Bangladesh, Indonesia and Pakistan (Final report). Philippines. Retrieved from http://www.adb.org/publications/legal-empowerment-women-and-disadvantaged-groups AGE, Rutgers WPF Pakistan. (n.d.). The puppet and the puppeteers : An account of consultative process on child marriages. Pakistan. Retrieved from http://www.rutgerswpfpak.org/content/pdfs/IEC/AGE/AGE-CP-Report.pdf Asia Society. (2012). Policing and women’s rights. In Stabilizing Pakistan through police reform (pp. 86- 91). New York. Retrieved from http://asiasociety.org/files/pdf/as_pakistan_police_reform.pdf Aurat Foundation. (2011). Gender based violence in Pakistan : A scoping study. Pakistan. Retrieved from http://af.org.pk/gep/images/deskStudies/GENDER%20BASED%20VIOLENCE%20- %20R%20PARVEEN%20 (2).pdf. Aurat Foundation. (2012a). Customary practice leading to G-BV: A primary data research study Dec 2012 (Research study). Retrieved from http://af.org.pk/gep_2/Research%20Studies%20(Gender%20Based%20Violence)/Customary%20practi ces.pdf Aurat Foundation. (2012b). Domestic violence: A primary data research study. Retrieved from htttp://af.org.pk/gep_2/Research%20Studies%20(Gender%20Based%20Violence)/Domestic%20Violen ce.pdf Aurat Foundation. (2014). Violence against women in Pakistan : A qualitative review of reported incidents (Annual Report on VAW). Pakistan. Retrieved from http://www.af.org.pk/PDF/VAW%20Reports%20AND%20PR/VAW%202014.pdf Brohi, N. (2010). Gender and land reforms in Pakistan. SDPI. Retrieved from https://www.sdpi.org/publications/files/Microsoft%20Word%20-%20A117.pdf Caitlin, R. (2014). Improving womens access to justice during and after conflict: Mapping UN rule of law engagement (Research study). UNDP/UNWomen. Retrieved from http://www.undp.org/content/dam/undp/library/gender/gender%20and%20governance/ImprovingWo mensAccessToJustice-UNDP-UNWomen-Mapping.pdf Child marriages in Pakistan: Submission to the women human rights and gender section of OHCHR | The Institute for Social Justice (ISJ). (n.d.). Retrieved from http://www.isj.org.pk/child-marriages-in￾pakistan/ Conaway, C. P. (2006). The role of women in Stabilization and reconstruction (Stabilization and Reconstruction series No. 3). Washington DC: United States Institute of Peace. Critelli, F. M. (2012). Voices of resistance seeking shelter services in Pakistan. Violence Against Women, 18(4), 437–458. http://doi.org/10.1177/1077801212452104 Gender Equity Program: Final Evaluation Report 127 Day, E. (2016). Regional consultation on the role of key stakeholders in ending impunity for violence against women and girls in Asia. Thailand: UN Women. Retrieved from http://www2.unwomen.org/~/media/field%20office%20eseasia/docs/publications/2016/04/ending%20i mpunity%20031516.pdf?v=1&d=20160429T050044 De Silva, R., & De Alwis. (2014). Women’s voice and agency : The role of legal institutions and women’s movements. World Bank. Retrieved from https://openknowledge.worldbank.org/handle/10986/21029 HRCP. (2016). Women: Rights of the disadvantaged (Annual Report). Lahore. Retrieved from http://hrcp-web.org/hrcpweb/wp-content/uploads/2016/04/Women_12.pdf Hudson, N. F. (2009). Securitizing women’s rights and gender equality. Journal of Human Rights, 8(1), 53–70. http://doi.org/10.1080/14754830802686526 Hussein, M., & Hussein, S. (2012). Internal trafficking of women and girls in Pakistan: A research study (Research study). Pakistan: GEP/Aurat Foundation. Retrieved from http://af.org.pk/gep_2/Research%20Studies%20(Gender%20Based%20Violence)/study%20on%20traffi cking%20final.pdf ICRW, and UNFPA. (2013). Child marriage in South Asia: Policy options for actions (pp. 19–24). Retrieved from http://reliefweb.int/sites/reliefweb.int/files/resources/Child_Marriage.pdf International Crisis Group. (2015). Women, violence and conflict in Pakistan (Asia Report. N°265). Retrieved from http://www.crisisgroup.org/~/media/Files/asia/south-asia/pakistan/265-women￾violence-and-conflict-in-pakistan.pdf Jacoby, H. G., & Mansuri, G. (2010). “Watta Satta”: Bride exchange and women’s welfare in rural Pakistan. The American Economic Review, 100(4), 1804–1825. Lari, M. Z. (2011). A pilot study on “Honor killings” in Pakistan and compliance of law. (Research study). Aurat Foundation. Retrieved from http://www.af.org.pk/pub_files/1366345831.pdf Malé, C., & Wodon, Q. (2016a, March). Basic Profile of child marriage in Pakistan. World Bank. Retrieved from https://openknowledge.worldbank.org/bitstream/handle/10986/24548/Basic0profile00marriage0in0Pa kistan.pdf?sequence=1&isAllowed=y Malé, C., & Wodon, Q. (2016b, March). Basic profile of early children in Pakistan. World Bank. Retrieved from https://openknowledge.worldbank.org/bitstream/handle/10986/24561/Basic0profile00ildbirth0in0Paki stan.pdf?sequence=1&isAllowed=y Nasrullah, M., Zakar, R., Zakar, M. Z., Abbas, S., Safdar, R., Shaukat, M., & Krämer, A. (2014). Knowledge and attitude towards child marriage practice among women married as children : A qualitative study in urban slums of Lahore, Pakistan. BMC Public Health, 14. http://doi.org/10.1186/1471-2458-14- 1148 NCSW. (2015). Standardized indicators on violence against women in Pakistan. Islamabad. Retrieved from http://www.ncsw.gov.pk/resource/previewpublication/36 Gender Equity Program: Final Evaluation Report 128 Rashid, T. (2009). Militarized masculinities, female bodies, and “Security Discourse” in post-9/11 Pakistan. Strategic Analysis, 33(4), 566–578. http://doi.org/10.1080/09700160902907100 Responses to information requests: Recourse and resources available to female victims of violence in Islamabad, Karachi and Lahore. (2015, January). Immigration and Refugee Board of Canada. Retrieved from https://www.justice.gov/sites/default/files/eoir/pages/attachments/2015/04/28/canada_coi_pak105023. e.pdf Rizvi, J. M. (2012). A study of the formal and parallel legal systems prevalent in Pakistan. NCSW. Retrieved from http://www.ncsw.gov.pk/resource/previewpublication/20 Scratching the surface : Democracy, traditions, gender. (2007). Lahore: Heinrich Böll Foundation. Retrieved from www.boell.de/sites/default/files/scratching_the_surface_commentary.pdf Solotaro, J. L., & Pande, R. P. (2014). Violence against women and girls: Lessons from South Asia. Washington DC: World Bank. Retrieved from https://openknowledge.worldbank.org/bitstream/handle/10986/20153/9781464801716.pdf?sequence= 1&isAllowed=y UN Women sourcebook on Women, Peace and Security. (2012). UNWomen. Retrieved from http://www2.unwomen.org/~/media/headquarters/attachments/sections/library/publications/2012/10/ wpssourcebook-01-overview-en.pdf?v=1&d=20141222T221156 UNFPA. (2007). Study Report: Child marriage in Pakistan, a Taboo (Secondary). Retrieved from http://floods2010.pakresponse.info/LinkClick.aspx?fileticket=CtHaS_yaaR0%3D&tabid=228&mid=173 9 UNFPA. (2008). Strategy and framework for action to addressing gender-based Violence (Strategy). Retrieved from https://www.unfpa.org/sites/default/files/pub-pdf/2009_add_gen_vio.pdf UNFPA. (2015a). Girlhood, not motherhood: Preventing adolescent pregnancy. New York. Retrieved from http://www.unfpa.org/sites/default/files/pub-pdf/Girlhood_not_motherhood_final_web.pdf UNFPA. (2015b). Religion, women’s health and rights: Points of contention and paths of opportunities. Retrieved from http://www.unfpa.org/sites/default/files/pub￾pdf/50424_UNFPA_White_Paper_NORAD_Web.pdf UNICEF. (2016). The State of the world’s children 2016 : A fair chance for every child. New York. Retrieved from http://www.unicef.org/publications/files/UNICEF_SOWC_2016.pdf USAID. (2012). US strategy to prevent and respond to gender-based violence globally, USAID, August 2012. Retrieved from http://www.state.gov/documents/organization/196468.pdf Weiss, A. M. (2012). Moving forward with the legal empowerment of women in Pakistan (Special Report No. 305). Washington DC: United States Institute of Peace. Retrieved from https://www.usip.org/sites/default/files/SR305.pdf WHO, and UNDP. (2012). Gender based violence in Pakistan : Response in the perspective of health sector devolution. Pakistan. Retrieved from http://applications.emro.who.int/dsaf/EMRPUB_2012_EN_858.pdf Gender Equity Program: Final Evaluation Report 129 World Bank. (2014). Voice and agency: Empowering women & girls for shared prosperity. Washington DC. Retrieved from http://www.worldbank.org/content/dam/Worldbank/document/Gender/Voice_and_agency_LOWRE S.pdf Zakharova, N. (2012). Women and Peace and Security: Guidelines for National Implementation. UN Women. Retrieved from http://www.unwomen.org/~/media/Headquarters/Media/Publications/en/02BPlanonWomenandPeace andSecurity.pdf Zaman, S. (2014). Forced marriages and inheritance deprivation in Pakistan : A research study exploring substantive and structural gaps in the implementation of prevention of anti-women practices [Criminal Law Amendment] Act, 2011. Aurat Foundation & Trocaire. Retrieved from http://www.af.org.pk/pub_files/1416847483.pdf Zia, A. S. (2010). A policy framework for women’s equal rights- issues, concerns and recommendations for gendered policy. NCSW. Retrieved from http://www.ncsw.gov.pk/resource/previewpublication/7 Zia, M. (2014). Legislative initiatives on women in Pakistan: Need for self-critique and reassessment. Aurat Foundation. Retrieved from http://www.af.org.pk/newsl/1402998220.pdf Zile, H. (2012). Mainstreaming gender into conflict and peace building through women led madras’s in Pakistan, Vol. 2 (No. 7). Gender Equity Program: Final Evaluation Report 130 Annex 9: Donor Activities on Gender (Reflections from the Field Work) Donor Sector German Embassy German Embassy has two components, GIZ and KFW: GIZ works on the soft and technical components such as capacity building, supporting Provincial governments with framing laws etc., while KFW works on financial and infrastructural, Energy, TVET (technical and vocational educational and training) components. Support provided to government departments, such as Social Welfare, WDD, Provincial Commission on the Status of Women, and law enforcement agencies and IPs for protection and prevention of violence against women and other women’s empowerment initiatives. This also includes support for provision of medical, psychosocial and legal services to women in need and awareness raising campaigns at the mass level related to violence against women. Community centers are being upgraded to women protection shelters in KP. Support to Provincial Commissions on the status of women at National and Provincial levels in attending trainings, conducting research, policy advocacy with IP’s and designing of frameworks and technical advice on policy reforms etc. Technical assistance to governments of KP/FATA through the Women’s Empowerment Wing, to implement women-friendly policies and laws such as “Protection of Women against Harassment at the Workplace,” the “Women Empowerment and Women Protection Policy,” and “Acid Control Bill” in KP and FATA. From 2016-2020 devised country strategy that includes laws, policies and frameworks with gender components. In the Energy sector, GIZ and KFW are involved in renewable energy, hydro power plants etc., which allow women to be a part of the economic and infrastructure development as well. Under the umbrella of Sustainable Economic Development, it works in areas of health, textiles, IT, vocational training, etc., all that involve gender mainstreaming. Paralegal Training and Human Rights Advocacy in Punjab World Bank Support and strengthening of women’s commissions at the provincial level, specifically KP. Gender embedded in the infrastructure, resettlement and rehabilitation initiatives. Consultations held with female headed households during resettlement and rehabilitation. “World Bank’s approach to gender issues is different from USAID or DFID, its work is very strongly focused on infrastructure development; land acquisition, resettlements, and livelihoods of resettled people - that’s paramount making sure that they get the right cash compensation. And at the same time they are consulted in a meaningful way and that they are resettled according to their priorities in the best way possible so that they are no worse of and ideally better off than they were in their houses.” Gender Equity Program: Final Evaluation Report 131 Donor Sector Department for International Development – United Kingdom (DFID) Aawaz, (2012-2017) is a five-year, 34 million pound project funded by DFID in 34 districts across Pakistan. “The program views democracy through a gender lens and contributes to women’s political empowerment, resolution of conflicts across the communities that it works with, and better public service delivery in the areas of health and education through informed and active engagement of citizens.” Working on a proper package for gender training, and building capabilities of individuals and organizations through a participatory approach. To date 140,000 women (59,048 in KP and 81,543 in Punjab 59,048) directly empowered through receiving their CNIC cards with 98,000 women registered to vote and 1,342 women (370 for KP and 972 for Punjab) standing as candidates for local government elections. Around 280 women candidates won in these elections. To date 2.477 million people positively impacted in 27 districts owing to prevention of sectarian/interfaith conflicts (1.04m in KP and 1.437m in Punjab) with early response mechanisms established in 4,500 villages and currently working on the establishment of early warning system for conflict prevention. To date 2.44 million people have benefited from improved service delivery through the raising of 14,303 demands (2,497 demands met) by communities for better services in health, education and sanitation. To date 10 studies and 3 position papers produced to influence policy dialogue on minorities, alternative dispute resolution and women’s empowerment to name a few. Also 3 Campaigns launched reaching 2,651,782 people through electronic, print and social media on elimination of VAWG, prevention of anti￾hate speech and participation of women in local government elections. European Union EU has two programs in human rights, one through the Parliament and the other through the National Assemblies. Future programs will be focusing on capacity building around elections. Planning to have one program on human rights and one with the police and judiciary. This also includes support to the National Commission on Human Rights next year in January. Also planning to have a program on Rule of Law. EU also works with CSO and government on programs related to formal education. They do budget support. In Sindh for example they support an education program. They will be developing an Action Plan for Gender for Pakistan at the end of this year. The average dollar value of EU programs is around $500,000. Gender Equity Program: Final Evaluation Report 132 Donor Sector UNDP UNDP does not have a stand-alone Gender program. UNDP & UNWOMEN shared the same executive board. In 2010 UNDP’s gender Unit got merged into policy & poverty and now it is Poverty and SDG. In 2014- 2017 gender equality strategy was approved by the board. UNDP prioritized Gender projects last year and now they have FATA Reform & IDP Rehabilitation project and a huge community empowerment engagement programming working in three agencies of FATA –Khyber, Mohmand and North Waziristan. Through CSO’s, UNDP has a women economic empowerment project. Under the Rule of law project, UNDP has conducted trainings for Police and judiciary, established help desks for women as UNDP has 450 CSO’s on their roster, and UNDP operates in FATA through these CSO’s, and also providing legal aid to women through partners. For sustainable development Goals – SDG’s, UNDP directly funds the government. Through competitive agreements and community engagement, it engages NGO’s / CSO’s for Women’s economic empowerment. Gender Equity Program: Final Evaluation Report 133 Donor Sector Japan International Cooperation Agency (JICA) JICA only works with the government. As a bilateral agency JICA has the mandate and responsibility to implement on behalf of ministry of foreign affairs. JICA offers three kind of assistance: loan assistance, grant assistance and technical assistance. JICA is launching a project on Home Based Workers. It has already started influencing the HBW policy in Sindh, where it is in the final stages of approval. It intends to work on creating better market opportunities for these home based workers and have them recognized in the National databases. “At the moment, HBWs are concentrated more on traditional sector and there is need to get them organized as cooperatives to access more opportunities in terms of better access to finance, market and concentrate on entering in nontraditional sectors power and energy sector. JICA is also keen to mobilize such opportunities; for example, JICA has long term partnership with ministry of water and power and we are working with them to establish training institute. The instructors would be mostly female and engineers. So they can review vulnerabilities of women in disaster sector.” Female instructors will train on Grid system operation & maintenance. Project for Skills Development and Market Diversification of Garment Industry Investment to First Microfinance Bank: 30 % of customers are women; promoting their enhanced economic participation & financial inclusion through project Established architecture course for female students Under design Projects:  Develop energy efficiency guideline s considering gender viewpoints as females have key role in household energy conservation (home appliances etc.).  Strengthen capacities of female staff of Board of Investment (BOI) / Analyze potential scope for engaging females in labor market and reflect result s/findings in policy or plan of BOI.  Automobile Industry Development Advisor; Analyze awareness campaign relevance & scope for both male and females particularly with reference to safety measures  Analyze barriers for females’ use of public transport and reflect them in standards & regulations to be developed by SMTA for improving women’s access to public transport  Flame extension workers will be trained for promoting role of women in livestock management Gender Equity Program: Final Evaluation Report 134 Donor Sector UNICEF More than half a million girls were enrolled in school for the first time in 25 UNICEF-assisted districts from 2005 to 2007. About 400,000 children (nine of out ten) are enrolled in government primary schools in earthquake-affected areas including more than 21,000 – mostly girls￾who had never attended school before. Social Funds for Community Welfare: Social funds are provided to communities through community-based structures for the purpose of identifying, prioritizing and implementing small-scale projects of public interest, with a primary focus on promoting the rights of children and women. Multifunctional Community Centers (MFCCs) in target villages: MFCCs are community-provided and managed public spaces accessible to all segments of the community, focusing on community mobilization, promoting community participation and social interactions and promoting awareness on child rights and women’s participation in the public sphere. (http://www.unicef.org/pakistan/overview.html) Asian Development Bank (ADB) “ADB deals with resettlement in land acquisition projects related to infrastructure. The beneficiaries are mostly males though ADB tries to influence the government to provide compensation checks to both women and men.” Gender Equity Program: Final Evaluation Report 135 Annex 10: Conflict of Interest Statements The conflict of interest disclosures were removed to preserve the confidentiality of evaluation team members. They are available from PERFORM on request. U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523