DISCLAIMER This report is made possible by the support of the America people made possible by the United States Agency for International Development (USAID). The contents are the sole responsibility of the Management Systems International and do not necessarily reflect the views of USAID or the United States Government. SPECIAL DEVELOPMENT OBJECTIVE 4 PERFORMANCE EVALUATION Jordan Monitoring and Evaluation Support Program (MESP) “CREATE EQUALITY” MURAL IN ZARQA JORDAN BY KEVIN LEDO, SUPPORTED BY THE USAID TAKAMOL ACTIVITY III | SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION USAID.GOV CONTENTS ACRONYMS IV EXECUTIVE SUMMARY 1 INTRODUCTION 3 EVALUATION PURPOSE, AUDIENCE, INTENDED USES 6 EVALUATION QUESTIONS 6 EVALUATION METHODOLOGY 6 TIMELINE AND PROCESS OVERVIEW 6 FINDINGS, CONCLUSIONS AND RECOMMENDATIONS 8 EVALUATION QUESTION 1 8 EVALUATION QUESTION 2 11 EVALUATION QUESTION 3 13 EVALUATION QUESTION 3A 13 EVALUATION QUESTION 3B 16 EVALUATION QUESTION 3C 17 SUMMARY OF RECOMMENDATIONS AND SUGGESTED MANAGEMENT STRUCTURE 21 ANNEX A: EVALUATION STATEMENT OF WORK (SOW) 27 ANNEX B: EVALUATION DESIGN WITH INSTRUMENTS 33 ANNEX C: GETTING TO ANSWERS 39 ANNEX D: WORK PLAN 42 ANNEX E: DATA COLLECTION INSTRUMENTS 45 ANNEX F: ROLES & RESPONSIBILITIES 57 ANNEX G: GENDER STRATEGY ROADMAP 58 ANNEX H: THE GENDER MANAGEMENT CONTINUUM 62 USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | IV ACRONYMS AOR Agreement Officer’s Representative CDCS Country Development Cooperation Strategy CEDAW Convention on Elimination of All Forms of Discrimination Against Women DO Development Objective DRG Democracy, Human Rights and Governance EDE Economic Development and Energy EDY Education and Youth EQ Evaluation Question GBV Gender-Based violence GDA Global Development Alliance IP Implementing Partner JNCW Jordanian National Commission for Women KII Key Informant Interview MSI Management Systems International PFH Population and Family Health POC Point of Contact PPR Performance Plan and Report SDO Special Development Objective SIGI Social Institutions and Gender Index SOW Statement of Work USAID United States Agency for International Development USG United States Government WRE Water Resources and Environment 1 | SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION USAID.GOV EXECUTIVE SUMMARY INTRODUCTION Gender equality and women’s empowerment are critical issues in Jordan. The country’s constitution mandates equality for all citizens, yet laws restrict women’s employment options. Jordan lifted its reservation to the Convention on Elimination of All Forms of Discrimination Against Women (CEDAW) on freedom of movement and residence, but left male dominance on family matters. Statistics show that more women than men enrolled in university, but less than 15 percent of women participate in the workforce due to social, economic, political and cultural factors that include unconducive workplaces, poor transportation options and cultural norms dictating that a women’s place is in the home. In 2012, as part of its 2013–2017 Country Development Cooperation Strategy (CDCS), the United States Agency for International Development (USAID) established a special development objective (SDO) to enhance gender equality and female empowerment. The purpose of this SDO is to pursue more effective investments in gender equality and female empowerment to accelerate progress in development. The development hypothesis underlying this SDO is that enhancement of female empowerment and gender equality will follow promotion of changes in discriminatory social norms and practices, strengthening of advocacy and policy reforms for women’s issues and expanded access to female-centered services. SDO4 was designed to achieve a specific set of development objectives, represented through the results framework shown in Figure 1, as well as support other DOs in strengthening gender integration in the context of their programs. FIGURE 1: SDO4 RESULTS FRAMEWORK. EVALUATION PURPOSE The purpose of this evaluation is to provide USAID/Jordan with an analysis and strategic recommendations on the effectiveness of the program strategy and the administrative approach that supports SDO4, “Gender Equality and Female Empowerment,” to mainstream gender across Mission programming. The evaluation results will help inform USAID/Jordan’s programmatic decisions for the future of SDO4 and the Gender Development Objective (DO) Team. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 2 EVALUATION METHODOLOGY This evaluation relied primarily on qualitative data collection from USAID/Jordan, USAID/Washington, USAID/Rwanda, USAID/Tanzania and the USAID/Jordan Takamol activity. USAID/Jordan provided secondary data. Additionally, select findings, conclusions, and recommendations from the Takamol mid-term performance evaluation report informed this evaluation report. EVALUATION QUESTIONS EQ1. To what extent has addressing gender gaps contributed to achieving USAID’s goals for expanding economic opportunities, increasing citizen participation, improving educational outcomes and achieving more sustainable population growth? EQ2. Has USAID been able to build sustainable synergies across USAID activities? Which interventions (either individually or within a thematic area) have been effective to build synergies and have these synergies been maximized? In what ways can they be improved? EQ3. How effective has the Gender DO Team been in integrating gender across the Mission? 1. How did USAID’s approach to managing and implementing gender integration impact effectiveness? 2. Do the team members receive sufficient support from management to be active members and confident Gender experts of the DO team and feel empowered to pursue increasing gender integration? 3. What has the DO team done well in gender integration and what can be improved? KEY CONCLUSIONS AND RECOMMENDATIONS This executive summary shares conclusions and recommendations grouped by key themes that correspond to the three evaluation questions: 1) stand-alone gender DO, 2) management and implementation, 3) strategic focus and 4) sustainability and synergy. Maintaining a Stand-Alone Gender DO. Given Jordan’s political, economic and social context, gender is a critical issue. USAID/Jordan is one of only two USAID missions to have a gender DO; creating this separate SDO elevated the importance and visibility of gender within USAID and with the Government of Jordan. The SDO has faced management and implementation challenges, though similar challenges exist whether gender is a stand-alone DO or a crosscutting theme. The evaluation team recommends that USAID/Jordan maintain a gender DO because of the critical status of gender in Jordan and the visibility the SDO lends to gender issues, and because the management and implementation issues surrounding the SDO can be mitigated. Management and Implementation. The management and implementation of SDO4 are key factors for the success of gender integration across the Mission’s portfolio, as is support from leadership regarding gender as a priority. Accordingly, SDO4 would be more successful with a clearer and more transparent mandate and structure. The evaluation team recommends SDO4 be located in the Program Office, with representation from technical offices as appropriate and at least one full-time dedicated gender expert. Furthermore, the team recommends the establishment of a management team responsible for planning, strategy, design and implementation of gender integration across the USAID/Jordan portfolio, along with an advisory team who will promote gender integration and continue to raise awareness of gender issues at the Mission and with external partners. The Mission Order “Implementation on Gender Equality and Female Empowerment” should be updated and should outline clear roles and 3 | SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION USAID.GOV responsibilities of SDO4 team members. These roles and responsibilities should be officially incorporated into position descriptions/work objectives of team members and their supervisors, to ensure accountability on both ends. Mission leadership should confirm and communicate gender as a priority. As a way to clarify objectives for SDO4, the evaluation team recommends that the SDO4 team incorporate adaptive management approaches, holding a retreat or series of strategy sessions to update the SDO4 strategy, identify implementation priorities and reach consensus on how to work together. The team should identify periodic check-in points to review progress and make adjustments as needed. Strategic Focus. While respondents noted that it has been difficult to discern SDO4’s overarching strategy, the latest version of the USAID/Jordan CDCS includes a fully developed gender strategy, while the last three years of performance plan and report (PPR) documents have aggregated results related to gender integration. The evaluation team recommends that the SDO4 team revisit the gender strategy as articulated in the CDCS, adapting or focusing it as needed and ensuring that team members are aware of the strategy and plan for achievement. Additionally, the gender integration strategy should be routinely communicated to other USAID/Jordan staff. Sustainability and Synergy. Clear definitions and objectives for sustainability and synergy regarding gender integration are needed to help frame SDO4 objectives. The evaluation team recommends that the SDO4 team complement its internal achievements of capacity building and training by focusing externally to create more of a service-orientation approach to relationships with technical offices. Additionally, the SDO4 team should routinely communicate its objectives, roles and responsibilities and corresponding activities to all USAID/Jordan staff. INTRODUCTION Gender equality and women’s empowerment are critical issues in Jordan. The country’s constitution mandates equality for all citizens while other laws restrict women’s employment options. Jordan lifted its reservation to the Convention on Elimination of All Forms of Discrimination Against Women (CEDAW) on freedom of movement and residence, but left male dominance on family matters. Statistics show more women than men are enrolled in university, but less than 15 percent of women participate in the workforce due to social, economic, political and cultural factors that include unconducive workplaces, poor transportation options and cultural norms dictating that a women’s place is in the home. According to a public poll, only 44 percent of men agree that women and men should have equal rights, compared to 82 percent of women; 57 percent of men and 42 percent of women believe men are better leaders.1 Jordan does not perform well on indices measuring gender equality, such as the Global Gender Gap Index and Social Institutions and Gender Index (SIGI). In the 2016 Global Gender Gap Report, which looks at economic participation and opportunities, educational enrollment, health and survival and political empowerment among women and men, Jordan ranked 134 out of 144 countries, performing poorly in several categories such as economic participation (ranked 138), health and survival (134) and political participation (123).2 The 2014 SIGI report 1 Women’s Leadership as a Route to Greater Empowerment: Jordan Case Study 2 http://reports.weforum.org/global-gender-gap-report-2016/economies/#economy=JOR USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 4 highlighted prevailing gender-based violence (GBV) and male guardianship over women as major obstacles to gender equality in Jordan.3 USAID/Jordan’s Strategic Approach. In 2012, as part of its 2013–2017 Country Development Cooperation Strategy (CDCS), USAID established a special development objective (SDO) to enhance gender equality and female empowerment. The purpose of this SDO is to pursue more effective investments in gender equality and female empowerment to accelerate development. The SDO has two prongs. One is focused on achieving the development hypothesis as represented in the results framework, drawing on the activity under SDO4 as well as the activities of other DOs. This development hypothesis is premised on the idea that enhancement of female empowerment and gender equality will follow the promotion of changes in discriminatory social norms and practices are promoted, strengthening of advocacy and policy reforms for women’s issues and expanded access to female-centered services. The second prong is designed to provide support to other DOs. Implementing the Strategy. Internally, USAID established the Gender DO Team, composed of representatives from the technical and support offices within the Mission. The team’s mandate is to help integrate gender across USAID’s programming, coordinate efforts within and among the USAID/Jordan portfolio and provide advice and programming guidance on gender issues. Members are viewed as gender advocates and gender points of contact (POCs) within their offices and they communicate on a variety of issues to and from the Gender Team. To date, the Gender Team has been housed by the Democracy, Rights and Governance (DRG) Office; the Gender Team Leader and Takamol Agreement Officer’s Representative (AOR) are both DRG staff. In an effort to rotate leadership and maintain a robust team and portfolio, the leadership of the Gender Team was transferred in September 2016 to a staff member from the Program Office; the plan is that the leadership of this team will rotate every two years. At the activity level, USAID/Jordan contracted out its Takamol activity to work with communities to raise awareness and promote attitude-shifting dialogue on gender equality while supporting partners in civil society and government to make specific improvements in conditions for women. In addition, Takamol has a crosscutting role in supporting both other DOs and implementing partners (IPs), reflecting the role of the SDO4 team. 3 http://www.genderindex.org/country/jordan USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 5 FIGURE 2: CREATING A SYNERGISTIC APPROACH FOR SDO4: AN OVERVIEW OF STRATEGY AND UNDERLYING MANAGEMENT STRUCTURE. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 6 EVALUATION PURPOSE, AUDIENCE, INTENDED USES The purpose of this evaluation is to provide USAID/Jordan with an analysis and strategic recommendations on the effectiveness of the program strategy and the administrative approach that supports SDO4, “Gender Equality and Female Empowerment,” to mainstream gender across Mission programming. The evaluation results will help inform USAID/Jordan’s programmatic decisions for the future of the SDO and the Gender Team; the evaluation team also expects the evaluation results will provide insights to USAID/Washington and other missions. The full statement of work (SOW) for this evaluation is in Annex A. EVALUATION QUESTIONS 1. To what extent has addressing gender gaps contributed to achieving USAID’s goals for expanding economic opportunities, increasing citizen participation, improving educational outcomes and achieving more sustainable population growth? 2. Has USAID been able to build sustainable synergies across USAID activities? Which interventions (either individually or within a thematic area) have been effective to build synergies and have these synergies been maximized? In what ways can they be improved? 3. How effective has the Gender DO Team been in integrating gender across the Mission? a. How did USAID’s approach to managing and implementing gender integration impact effectiveness? For example, was creating a separate DO the best approach to achieving the desired objectives or are there other more effective approaches? b. Do the team members receive sufficient support from management to be active members and confident Gender experts of the DO team and feel empowered to pursue increasing gender integration? c. What has the DO team done well in gender integration and what can be improved? EVALUATION METHODOLOGY This section describes the overall methodology that was used to answer the evaluation questions. The evaluation’s design, including final data collection instruments, is in Annex B. This report was originally written in February and March 2017 and was updated in June 2017 to include relevant findings from the Takamol mid-term performance evaluation report. TIMELINE AND PROCESS OVERVIEW The evaluation team’s initial review of documents relevant to SDO4 began January 29, 2017. From early to mid-February, the evaluation team developed the evaluation design and work plan, including instruments for key informant interviews with SDO4 team members, Mission leadership, program and technical office staff, and an online survey for SDO4 team members (past and present). The SDO4 evaluation team collected primary data from January 29 to February 17, 2017; the survey of SDO4 team effectiveness was open to current and past members of the SDO4 team from February 13-15, 2017. Analysis of the primary and secondary data was ongoing and a preliminary debriefing summarizing findings, conclusions and recommendations to date and delivered to USAID on February 16, 2017. This version of the final report incorporates the full analysis of data. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 7 DATA COLLECTION METHODS The evaluation team collected, reviewed and analyzed both primary and secondary data for this evaluation and focused primarily on collecting qualitative data through in-depth interviews; limited quantitative data collection through a survey complemented the qualitative data. PRIMARY DATA COLLECTION The evaluation team collected primary data through a series of interviews with USAID/Jordan, USAID/Washington, USAID/Rwanda and USAID/Tanzania. The team developed eight instruments to collect both qualitative and quantitative data. Table 1 shows how the team used each instrument for each respondent type. Annex B contains the instruments. SECONDARY DATA SOURCES Major secondary sources consulted for this evaluation included the following documents. These documents were the source of the activity monitoring data used in this report. For a full list of documents used, see Annex B. 1. USAID/Jordan Country Development Cooperation Strategy 2013–2019 2. Gender PPRs 3. Mission Order: Implementation of Policy on Gender Equality and Female Empowerment 4. SDO4 Team Charter Respondent Type Number of Interviewees Data Collection Method Instrument SDO4 Team Members (past and present)4 13 In-depth interviews Interview Questions for SDO4 Team Mission Leadership 6 Interview Questions for Mission Leadership Technical Office Leads 5 Interview Questions for Technical Offices Technical Office Staff 4 Interview Questions for Technical Offices Program Office Staff 4 Interview Questions for Program Office Takamol Activity Staff 2 Interview Questions for Takamol USAID Mission Staff (Rwanda and Tanzania) 3 Interview Questions for Mission Gender POCs USAID/Washington 2 Interview Questions for Washington Gender POCs Total Interviewees 39 SDO4 Team Members (past and present) 11 Online Survey SDO4 Team Survey Total Respondents (out of 15) 11 4 Several interviewees, particularly with SDO4 team members, fit into multiple respondent types, so the evaluation team used its discretion to determine the most appropriate instrument to use. TABLE 1. SDO4 EVALUATION: DATA COLLECTION METHODS AND INSTRUMENTS BY RESPONDENT TYPE. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 8 DATA ANALYSIS PROCEDURES This evaluation relied on content analysis of in-depth interviews, survey data and relevant documents. When applicable, this report provides findings, conclusions or recommendations by respondent group, such as Mission leadership or SDO4 Team. COORDINATION WITH TAKAMOL EVALUATION TEAM The evaluation of SDO4 overlapped with a midterm performance evaluation of the Takamol activity. Both evaluation teams worked together to identify areas of complementarity and specific areas where one evaluation would strengthen the other. The SDO4 evaluation team expects that the Takamol evaluation findings related to one of their evaluation questions will enrich SDO4 evaluation findings regarding information specific to Takamol’s potential creation of synergies. The evaluation question from Takamol that has the greatest relevance for this evaluation is: Takamol EQ2: Has Takamol built synergies with other activities across the Mission portfolio? Why or why not? In what ways can they be improved? In May 2017, the SDO4 evaluation team received a draft of the Takamol mid-term performance evaluation report. The SDO4 team reviewed the report, and in June 2017 integrated relevant findings into this version of the evaluation report. LIMITATIONS SDO4 evaluation question 1 focuses on the extent to which addressing gender gaps contributes to the achievement of USAID’s other goals (or development objectives). While the evaluation can provide impressionistic answers to this question based on interviewees’ perceptions, the evaluation team cannot ascertain attribution or even correlation. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS EVALUATION QUESTION 1 To what extent has addressing gender gaps contributed to achieving USAID’s goals for expanding economic opportunities, increasing citizen participation, improving educational outcomes and achieving more sustainable populations growth? FINDINGS Interviews indicated a strong consensus that addressing gender gaps contributes significantly to achieving development outcomes and that gender is a critical aspect of USAID’s work across sectors, especially given Jordan’s social, political and economic context. However, the extent to which addressing those gaps has contributed to the achievement USAID objectives is not clear. Few respondents were able to identify tangible examples of how gender integration in USAID activities and interventions are contributing to meeting objectives, although several noted examples of how their teams are including gender considerations into their work. The text box that follows provides examples drawn from interviews, while Table 2 highlights achievements mentioned in the FY 2016 Gender PPR. However, the examples cited in interviews are activities (or what the technical team is doing) and the achievements summarized in the PPR are outputs rather than outcomes. Finally, interviews revealed that USAID’s overarching gender strategy was not clear to most Mission staff. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 9 GENDER-RELATED WORK NOTED BY TECHNICAL OFFICES The Democracy, Human Rights and Governance (DRG) Team trained women politicians; successful women candidates exceeded the quota by five seats in the most recent election. The Economic Development and Energy (EDE) Team is helping create a flex-time/part-time bylaw that is intended to facilitate women’s participation in the workforce. The Education and Youth (EDY) Team plans to focus on improving boys’ education to address low performance and high dropout rates, relative to girls. The Population and Family Health (PFH) Team is targeting men more in its family planning efforts, since they are often the decision-makers in the household and need to be sensitized/made aware of family planning options; health systems strengthening efforts also incorporate ways to address gender issues. The Water Resources and Environment (WRE) Team has trained its implementing partners on how gender is relevant for the water sector. FY 2016 PPR Gender-Related Achievements Policy and Legal Frameworks  Supported Ministry of Labor revise the Jordanian Labor Code for greater flexibility for women’s alternative work arrangements  Helped Jordanian National Commission for Women (JNCW) to develop policy briefs  Supported drafting of 45 policies and laws to promote gender equality (e.g., businesses that will allow women to move into the formal sector)  Worked on gender-based policies related to Jordan's Equal Futures Partnership Action Plan; revisions to civil services bylaws will ensure equal opportunities in seeking leadership /supervisory positions Capacity Building and Training  Hosted 350 community meetings and trainings to challenge perceptions, counter gender-based violence and highlight role models  Organized training/networking events for women in the energy & engineering sectors  Provided teacher training on gender-sensitive approaches to improve schoolchildren’s reading and math skills  Engaged parliamentary and civil society organizations in training and advocacy to encourage the use of gender-lens in legislation, policy and strategy formation  Trained 300 women leaders considering running for parliamentary elections; 191 ran campaigns and 11 won seats in the September 2016 elections TABLE 2: SUMMARY OF FY 2016 PPR GENDER-RELATED ACHIEVEMENTS. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 10 FY 2016 PPR Gender-Related Achievements Communications and Outreach  Used social dialogue and policy tools to promote knowledge and awareness on the role of women in elections, and women’s low participation in the workforce  Partnered with the JNCW and local media-related civil society organizations to enhance engagement to shift toward issue-based voting  Conducted social media outreach on women's access to finance and placing women's economic participation on the political agenda (120,000 views) Technical Assistance  Supported 25 female lawyers to offer information on laws (Personal Status Law, Labor Law, Domestic Violence Law and the Anti-Human Trafficking Law)  Supported the Women's Caucus and town hall meetings  Revised gender indicators in the budget law and increased the Government of Jordan’s capacity to collect/analyze data to ensure equitable distribution of government resources  Supported more than 300 women-owned businesses to enhance the quality and productivity, integrating them into supply chains, helping them develop financial management skills to comply with legal and regulatory changes  Strengthened microfinance institutions by introducing new products and services that will benefit more than 2,000 women in 2017  Supported 27 women-owned/ led small and medium enterprises, enabling them to receive loan guarantees totaling USD $3.5 million CONCLUSIONS Interviews revealed a relatively strong consensus that, in principle, addressing gender gaps contributes to achieving USAID’s goals for expanding economic opportunities, increasing citizen participation, improving educational outcomes and achieving more sustainable population growth. However, few interviewees provided tangible examples of outcomes (or higher-level results). While the PPR identifies a range of significant outputs that are ultimately intended to achieve higher-level outcomes, the extent to which those outcomes will be achieved is not yet clear. This evaluation is occurring beyond the halfway point of the strategy (2013-2019). Considering the life cycle of strategies and activities, most outcomes take more time to achieve and occur in the latter phases of implementation. This question assumes a model where the integration of gender enhances cumulative development outcomes at the higher level. In practice, the evaluation team found a much more complex model. That is, gender is substantively integrated at every level of the results framework, both as a means to an end (to enhance development objectives) and as an end itself, as reflected in higher-level results. For example, DO1 is “Broad-based, Inclusive Economic Development Accelerated.” Inclusiveness means that both genders must benefit. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 11 The fact that USAID’s overarching gender strategy is not clear for most staff means that both the development hypothesis and the higher-level objectives (and associated indicators) are not a guiding focus. In a sense, this contributes to an understanding of gender to a series of activities, which may or may not lead to larger objectives and outcome measures. RECOMMENDATIONS The SDO4 team should review the gender strategy in the CDCS, and incorporate that strategy as a focal point for the team—similar to how a technical office uses the results framework as a guiding reference for analysis, project design, monitoring and evaluation, and implementation. Second, the team should consider taking a strategic approach toward gender programming by identifying the highest priorities within each technical sector. Finally, it is important for the SDO4 team to regularly and proactively communicate the gender integration strategy and key priorities to other technical teams. Annex C excerpts and summarizes USAID/Jordan’s gender strategy as described in the CDCS and summarizes the results reported in the most recent PPR. This is intended to assist the SDO4 Team by organizing the overarching gender strategy and results reporting against that strategy in one document. For future consideration, the evaluation team recommends that the SDO4 Team consider revising an impact question to simply focus on what outcomes have been achieved, either as a result of integration or in terms of achieving gender objectives in their own right. This was a common point that emerged across interviews from respondents themselves, and is important to understand, particularly as the strategy enters into the latter phases of maturity. EVALUATION QUESTION 2 Has USAID been able to build sustainable synergies across USAID activities? Which interventions have been effective to build synergies and have these synergies been maximized? In what ways can they be improved? FINDINGS Overall, most respondents were not able to define sustainability or synergy in the context of the gender program. In particular, most members of the SDO4 team viewed sustainability through an internal lens, meaning they believe building internal capacity is a good way to build sustainability, in contrast to viewing it through a program lens. One respondent framed sustainability as systems, processes, policies and behaviors that support gender integration and that exist independently, without additional SDO4 support. This respondent defined synergy as the ability of the SDO4 team to view activities across the entire USAID/Jordan portfolio and identify opportunities to work across programs to maximize impact. Respondents recognized that potential exists for creating greater synergy, but said they are not undertaking systematic or regular efforts to work with other teams. In particular, the SDO4 team thought that having representation from technical offices was a good way to promote synergy. Respondents identified a couple of instances where synergies were leveraged. One was the WRE Office and SDO4 team putting together a two-day water and gender training for IPs, WRE technical team members and Ministry of Water staff. The other example, mentioned by two respondents from different technical offices, was a Global Development Alliance (GDA) that included DRG, EDE and the Program Office. The GDA resulted in young women and men being trained to work in the hospitality sector. The Takamol mid-term performance evaluation found that Takamol facilitated several synergies with implementers and activities in the technical areas of Democracy, Rights, and Governance (IP National Democratic Institute for Public Affairs; Rule of Law and Public Accountability Strengthening activity) and Economic Development and Energy (Building Economic Sustainability through Tourism; USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 12 Jordan Competitiveness Program; and Local Enterprise Support activities). Enabling factors for these synergies included interest and commitment from IPs and common objectives of the implementers. The Takamol evaluation noted that Takamol and IPs expected USAID would take a leading role in coordinating synergies and provide direction about where gender fits into activities; both Takamol and IPs felt that the level of USAID coordination did not meet these expectations. From USAID’s perspective, the evaluation team found that USAID lacked internal clarity around the definition, vision and strategy to achieve synergy, including the extent to which Takamol should be integrated into partner coordination meetings. This lack of clarity limited greater opportunities for synergy. IPs were also not aware of Takamol’s scope, approaches, and interventions such as training for IPs. Finally, several respondents noted that the question of who would receive “credit” for achieving results (for example, promoting gender mainstreaming and integration) would emerge as a problem among implementers. CONCLUSIONS At the most basic level, the first step toward achieving an objective is to clearly define what it means in a particular context. Both sustainability and synergy are broad concepts that have different meanings depending on the program context. In other words, it is difficult to achieve what is not clearly defined and understood by the team implementing the strategy. Sustainability. Based on the clearest definition provided (see above), sustainability has not been achieved. However, this is not unexpected, given the magnitude of the development problem in the Jordanian context and the fact that SDO4 has been in existence only since 2013. Synergy. The evaluation team defines synergy as the ability of SDO4 and technical offices to identify and create opportunities to integrate gender into other sector programs to achieve development results more effectively. Interestingly, technical offices identified more opportunities to implement a multi-sectoral approach than SDO4 did. This may reflect the fact that SDO4 tended to be more inward-looking during the first few years. Issues such as gender, whether treated as a DO or a crosscutting theme, confront operational challenges within USAID Missions. USAID systems and processes do not often encourage working across technical offices or DO teams (e.g., budgeting, reporting, implementation, etc.). Including team members from technical offices on the gender team is necessary but not sufficient to create synergy. It is also critical to establish clear responsibilities for working across offices, ensure senior management support and clarify how to operationalize synergy. Additionally, as the Takamol mid￾term performance evaluation emphasizes that USAID has a crucial coordination role to play, offering guidance to IPs about where gender fits into their respective activities and helping identify or foster synergies between IPs. Partner coordination meetings provide opportunities for coordination. RECOMMENDATIONS Sustainability and synergy are two distinct concepts that can mean different things in different programming contexts. The evaluation team recommends that the SDO4 team clearly define its objectives related to sustainability in a way that is appropriate to the gender program. By incorporating a stronger focus on strategy (the SDO4 strategy as well as reviewing the DO strategies through a gender lens), SDO4 will be better positioned to see the entire portfolio through a gender lens and identify priorities from a gender point of view. Particularly in relation to synergy, roles and responsibilities for SDO4 team members should be clearly defined and include the responsibility to function as gender champions within their technical offices in a more proactive way. Expectations regarding individual and team roles for creating synergy should be clearly stated to team members and their supervisors. The team should also reach out to technical offices, develop a service orientation and build relationships with the technical offices. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 13 Ensure that SDO4 team members are empowered to reach across offices as part of the SDO4 team. Provide clear expectations of what this should look like and hold the gender and technical team leads accountable for working across sectors. The following Takamol mid-term evaluation recommendations related to synergy are also relevant for the SDO4 evaluation: USAID should provide greater facilitation between Takamol and IPs by inviting them to sector￾specific coordination meetings, and ensure that Takamol’s mandate is well understood by IPs. “Takamol and USAID should pursue relationships with USAID IPs that have complementary areas of implementation, with the objective of maximizing both Takamol’s and the IP’s successes. The synergy developed around common goals and interests can result in dynamic results if there is open communication, clear roles and responsibilities are outlined, and Takamol can provide added value.” In addition, it is important to allow for or facilitate “shared credit” for success so that this does not prevent collaboration. While the Mission cannot “double count”, there is no reason that two implementers cannot take credit for results that they have both engaged in, as long as the numbers are only counted once by USAID. This creates a positive incentive to do more in this area, in an environment where gender is a priority. Takamol could also report the numbers but clearly indicate that they are being reported by another IP as well, and the narrative could discuss each’s unique role in bringing about a particular result. The Mission could categorize these as “shared indicators” and ensure that the number is only reported once. 5 EVALUATION QUESTION 3 How effective has the Gender DO Team been in integrating gender across the Mission? Findings, conclusions, and recommendations for EQ3 are organized under each sub-question below. EVALUATION QUESTION 3A How did USAID’s approach to managing and implementing gender integration impact effectiveness? 5USAID (prepared by MSI), Takamol Mid-Term Performance Evaluation, Draft Report, May 2017 p. 36 USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 14 FINDINGS Respondents voiced strong consensus that creating a separate DO, or SDO, elevated the importance and visibility of gender, both within USAID and with the Government of Jordan. This point was widely acknowledged, even among those who prefer to address gender as a crosscutting issue. Opinions were decidedly mixed as to whether the Mission should have a separate DO for gender in contrast to addressing gender as a crosscutting issue. However, a majority of SDO4 members preferred maintaining a separate DO, while others outside of the SDO4 team supported the SDO but were concerned that the Mission is not achieving the intended results across the portfolio. While all individuals in leadership positions consider gender a critical issue for achieving development results, a significant number prefer to address it as a crosscutting issue or felt that addressing it as a crosscutting issue may be just as effective as addressing it as a separate DO. Respondents also noted the lack of a clear, overarching strategy and they questioned whether there are significant development outcomes. Additionally, several respondents noted that having both an SDO4 Team Leader and the Takamol AOR reside within the same technical office (in this case, DRG) influenced the direction of the team and activities regarding which technical topics to focus on; thus, the tasks of the SDO4 team took on a predominantly DRG focus. On the other hand, SDO4 team members identified internal capacity building and training as a key success. Others recognized that SDO4 is an experiment of sorts, and suggested the need for more time to judge success or failures. CONCLUSIONS As USAID/Jordan is one of only two USAID missions to have a gender DO, it should be recognized for both its commitment to gender and for developing a new approach to addressing gender in the portfolio. The elevation of gender in the form of a SDO has played an important role in raising the significance and visibility of gender in a context where addressing gender gaps is integral to achieving development objectives. At the same time, while USAID encourages integrated programing in principle (particularly in strategic planning and in the form of collaboration, learning and adapting), many systems and processes (budgeting, reporting, and implementation) do not create an environment that encourages or requires USAID staff to work across technical sectors. This creates management challenges for any crosscutting development issue, whether it is a DO or addressed as a crosscutting issue. The most significant issues facing this DO are related to management and implementation, which reflect a set of challenges that can be addressed. It is important to distinguish the management and implementation of SDO4 from that of technical offices. RECOMMENDATIONS DO or Crosscutting Issue? The evaluation team recommends that USAID/Jordan maintain a separate gender DO through the life of the CDCS. The rationale for this recommendation is that: 1) the extent to which gender is a constraint for development in Jordan is compelling, 2) the SDO Management and Implementation Challenges A majority of Mission staff identified the management and implementation of the SDO as a key challenge. Those challenges include the following:  Lack of clear roles and responsibilities (and no official gender responsibilities in the staff position descriptions/work objectives)  Lack of clarity regarding the management and reporting structure  Uncertainty about the extent to which gender is a priority for USAID/Jordan  Capacity of the SDO4 team (lack of one or more dedicated gender experts, competing demands, varied experience and expertise by SDO4 team members)  The time that is devoted by SDO4 team members to the team USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 15 raises the visibility and importance of gender, 3) the management and implementation problems identified are common to any crosscutting theme, whether handled as a stand-alone objective or as a crosscutting issue and 4) the management issues represent solvable problems. Management and Structure of the SDO4 Team. The following summarizes recommendations related to management and structure of the SDO4 team. Table 5 on pages 21-22 summarizes these recommendations and identifies key structural questions that emerged and were considered by the evaluation team, as well as the rationale behind the recommendation. The evaluation team also recommends that the SDO4 Team Leader be permanently housed in the Program Office, because it is best situated to work across all of the technical offices — something that is essential for the success of this crosscutting DO. In addition to its naturally crosscutting view of the program, the Program Office has significantly more staff compared to technical offices. Locating the SDO4 Team Leader within the Program Office rather than in a particular technical office can mitigate the risk of focusing on one technical area too heavily. However, it is also critical to ensure sufficient technical expertise to ensure success. A DO that focuses on increasing economic growth requires staff with appropriate technical expertise and the same is true for SDO4. The team needs at least one dedicated full-time team member and, ideally, a full-time administrative support position to effectively implement the DO, including coordination of gender initiatives, the provision of support on gender to the other DOs, the organization of training and supporting related outreach and communication. In addition, the team should see itself as a central coordinating point to access further gender expertise when needed, through Takamol or other mechanisms, or by reaching back to resources in USAID/Washington. In addition, the evaluation team recommends dividing the SDO4 team into two parts: the management team and the advisory team. The management team represents a core group who will oversee the SDO on a consistent basis and will be accountable for its success. This group would consist of individuals who meet the following criteria:  Have the deepest technical expertise related to gender;  Have the time and skills to share in the workload;  Manage gender activities;  Play a leadership role;  Have the capacity to both influence and work in partnership with other DO teams;  Are highly committed to gender; and  Are skilled in building relationships with technical offices across the Mission. Membership on the management team may also reflect priority areas. For example, if one priority is to improve women’s participation in the formal workforce, ideally at least one member of the EDE Office would be part of the management team. This team should also have explicit gender responsibilities included in its work objectives or position descriptions. In addition, the time that each team member will contribute should be negotiated with each office director or supervisor. The advisory team includes staff who play a consultative role and function as gender champions in their offices. They participate on the advisory team to learn about gender issues, communicate issues to the management team and communicate information back to their home office. They may be periodically (but not regularly) be tasked with work. The SDO4 team should integrate adaptive management approaches for assessing progress and making changes as necessary. Recommendations include: USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 16  Hold a series of strategy sessions (optimally as part of a retreat, but if that is not possible, then through a staggered series of planned sessions) following the completion of the evaluations to process findings, clarify the strategy, identify priorities and achieve consensus on how the team will work. This should result in an initiative to address management and operational challenges that have been raised in the evaluation.  Once the team’s internal strategy is clarified, hold sessions with technical offices to provide an overview of how SDO4 will work moving forward, how key challenges will be addressed, and to emphasize what services/ benefits the SDO4 team can provide.  Commit to holding a separate portfolio review for SDO4 and support technical teams in incorporating gender into their portfolio reviews.  Identify periodic check-in points to review progress against priorities. At a minimum, this can take the form of a team session at least twice a year to discuss successes, challenges and actions for change. This can also integrate accountability for implementing recommendations that result from the evaluations. These sessions ideally would take place prior to the portfolio review and then key outcomes reported. The evaluation team recommends that SDO4 review, update and obtain formal approval for the Mission Order (drawing from the draft SDO4 Team Charter). While this was not implemented in the Mission for the other DO teams (which now function as technical offices), it remains critical for this SDO, which does not operate as a technical office. It is essential to gain clarity and consensus around roles and responsibilities with senior leadership and others across the Mission. The finalized and approved Mission Order should be subsequently shared with USAID/Jordan more broadly to foster an understanding of the role of SDO4 team members and how others are best suited to support the team and their gender integration efforts. EVALUATION QUESTION 3B Do the team members receive sufficient support from management to be active members and confident gender experts of the DO team and feel empowered to pursue increasing gender integration? FINDINGS A majority of SDO4 team members believe that management supports them on gender integration. However, the extent to which gender is a priority to senior management is unclear to Mission staff, including members of the SDO4; some noted that it was a high priority while others were unsure. SDO4 team members acknowledged that explicit support from Mission leadership regarding gender integration would help them feel more empowered. Additionally, some team members requested clarification of their role on the SDO4 team and overall objectives of the team. One respondent called for the integration of gender responsibilities in position descriptions/work objectives to elucidate roles and responsibilities but also create accountability and recognition for the work the SDO4 team members undertake in addition to their other official responsibilities. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 17 In particular, several SDO4 team members noted that their skills are growing, but acknowledged that they are not gender experts. SDO4 team members both recognize and respect gender expertise in its own right and the importance of ensuring that adequate skills are mobilized to implement the SDO effectively. CONCLUSIONS All SDO4 team members serve in other offices, have primary expertise in other areas and serve part-time on the gender team. They have successfully focused on deepening their own capacity through training. However, the gender SDO, like any other DO, requires some dedicated staff with expertise to implement the strategy and support other DOs effectively. Team members must also have opportunities to regularly use the skills gained from trainings and be accountable for their duties as SDO4 team members. Leadership support at all levels is a critical ingredient for supporting SDO4 team members and to empower the team. At the most senior levels, it is important to communicate gender as a Mission￾wide priority. This will support the team by creating incentives, opportunities and space to work with other technical offices. At the technical office level, teams must regularly communicate internally about how gender integration affects their larger goals and share achievements and challenges towards gender integration. This cannot happen only during annual portfolio reviews, or gender integration will remain a “check the box” exercise. Clarity around roles and responsibilities is also a critical factor for success. Without it, team members are unsure of how they can interact and whether they can advocate for gender with technical offices. The ability to look outward, build relationships and identify opportunities to support technical offices is critical for the success of this crosscutting DO. RECOMMENDATIONS The evaluation team recommends that senior management (including the Mission Director and deputy directors) reinvigorate (or communicate) the Mission’s commitment to SDO4. This can be done through conveying gender as a priority in senior management meetings and communicating efforts to improve management approaches for implementing SDO4. In addition, the evaluation team recommends that the Deputy Mission Director responsible for managing the SDO4 team have a regular meeting (e.g., monthly) with the SDO4 team to ensure an effective communication channel is available to senior management and vice versa. Portfolio reviews should incorporate gender as a crosscutting element and also focus on gender as a stand-alone SDO to review progress of the SDO against the larger strategy. EVALUATION QUESTION 3C What has the DO team done well in gender integration and what can be improved? Areas for SDO4 Team improvement (Cited by Respondents During Interviews) External to SDO4 Team  Increased support from leadership  Broader buy-in of gender integration by Mission staff and IPs Internal to SDO4 Team  Focused SDO4 Team strategy  Clear purpose, roles, and responsibilities of SDO4 team  High-quality gender analysis  Increased visibility of SDO4 Team  Improved communications and information sharing  Service orientation towards technical teams  Strengthened internal gender integration capacity of SDO4 team members USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 18 FINDINGS The SDO4 team noted during in-depth interviews that internal capacity building and trainings were highly supported and cited these as key successes for the SDO4 team. In addition to standard USAID online gender training (101,102 and 103), the SDO4 team facilitated training that received positive feedback and worked with the WRE Office to deliver a two-day gender and water training. Technical offices were not aware of what SDO4 is working on and were generally less confident in the expertise available to them through the SDO4 team. They said SDO4 should be more proactive and service oriented to provide more support to the technical teams. The evaluation team also conducted a survey of the SDO4 team members, which asked them to assess the effectiveness of the team across seven dimensions based on the PERFORM model (purpose, empowerment, relationships and communications, flexibility, optimal performance, recognition and appreciation, and morale).6 Respondents rated each area on a scale of 1 (strongly disagree) to 5 (strongly agree).The overall results of the survey were very positive, with a lowest average score of 3.18 (see Table 3). Findings from the survey mirror the information resulting from in-depth interviews, with the most positive or effective elements of the SDO4 team being opportunities for learning and respect among team members (both in the empowerment category). Conversely, the areas that were relatively the weakest are support from the organization and senior leadership (recognition and appreciation), enthusiasm for working on the team (morale) and clear policies, rules and processes to achieve team objectives (empowerment). Area Average Score (Scale of 1-5) Respect for other team members 4.27 Learning opportunities 4.27 Conflict management of personalities 4.18 Willingness to contribute to the team 4.09 Commitment to getting the job done on time 4.09 Constructive feedback 4.0 Trust 3.9 6 The survey based on the PERFORM Model is in Annex B. TABLE 3: AVERAGE SURVEY SCORES. Team Effectiveness: Factors for Success (Cited by Respondents)  Senior leadership support and engagement  Clarity of roles and responsibilities of SDO4 team  Effective communication and outreach/ a proactive approach  Team leadership and ability to influence other technical teams  Sufficient capacity to implement the SDO, including expertise and time/resources  Accountability (incorporation of gender responsibilities in position descriptions/work objectives) USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 19 Area Average Score (Scale of 1-5) Commitment to high standards and results 3.9 Purpose 3.81 Freedom of expression 3.81 Conflict management of ideas 3.81 Recognized accomplishments 3.81 Confidence in team 3.63 Effective problem-solving and decision-making 3.54 Clear policies and rules 3.45 Shared responsibilities 3.45 Enthusiasm for working on team 3.36 Support from organization and senior leadership 3.18 CONCLUSIONS The SDO4 team has been focused more internally, for example, building its internal capacities for gender integration as a starting point, but has not yet fully focused outward in terms of building relationships across programs and identifying priorities to build synergy with other technical teams. Figure 3 presents a model to understand three distinct and progressive stages for integration of gender in a mission. The evaluation team concludes that USAID/Jordan is somewhere in stage 2, having met foundational requirements of gender integration. Annex D includes the Gender Management Continuum and rationale for the criteria. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 20 FIGURE 3: THE GENDER MANAGEMENT CONTINUUM. While the SDO4 team focused on exchanging information, preparing for specific events like portfolio reviews and training, there was not adequate focus on managing against the SDO4 strategy, either in the form of the SDO4 results framework or across other DOs. SDO4 and other technical offices alike displayed a lack of understanding of the overarching strategy. However, the CDCS includes a fairly well articulated strategy. The Mission is much further along in tackling gender issues and there is recognition that this takes continued effort. Initially, during periods of strategy development in 2012 and 2013, a lot of effort focused on gender as a SDO, but some of that initial effort and passion dissipated as team members were pulled back into daily work and Mission staff changed. Newer Mission staff were not familiar with the thinking behind SDO4. RECOMMENDATIONS The evaluation team recommends that the Mission continue to build and expand on the foundation created. Communicate the role of gender in the portfolio and message the specific priorities for gender programming. The SDO4 team should recognize that, precisely as a crosscutting DO, efforts to look outward by reaching out, advocating, communicating, providing services, and demonstrating value-added, are critical. The SDO4 team needs to be more proactive in playing this role on an ongoing basis. In addition, having adequate and qualified staff are critical for that to happen. SDO4 should focus on achieving development results (as represented in the results framework for SDO4 and across the other DOs), as is recommended for all DOs. This can be done by reviewing USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 21 the strategy and identifying priorities or by reviewing performance against the larger strategy at least twice a year. SUMMARY OF RECOMMENDATIONS AND SUGGESTED MANAGEMENT STRUCTURE Table 4 on the following page summarizes the recommendations outlined in this section, while Table 5 provides more detailed comments regarding the recommended management structure for SDO4. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 22 Audience Recommendation Implementation Status SDO4 Team 1. Hold a series of strategy sessions following the completion of the evaluations to process evaluation findings, clarify the strategy, roles and responsibilities of team members, identify priorities and achieve consensus on how the team will work. 2. Review the gender strategy in the CDCS. Use it as a guiding reference for gender analysis, project design, monitoring and evaluation and implementation. 3. Review, update and obtain formal approval for the Mission Order and share with USAID/Jordan more broadly to foster an understanding of the role of SDO4 team. 4. Clearly define SDO4 objectives related to sustainability, in a way that is appropriate to the gender program. 5. Clarify and communicate expectations regarding individual and team roles for creating synergy to team members and their supervisors. 6. Regularly and proactively communicate the gender integration strategy and key priorities to technical offices. 7. Once the team’s internal strategy is clarified, hold sessions with technical offices to provide an overview of how SDO4 will work moving forward and how key challenges will be addressed, and to emphasize what services/ benefits the SDO4 team can provide. 8. Develop a rapport with technical offices and conduct more proactive outreach or develop a service orientation. 9. Commit to holding a separate portfolio review for SDO4 and support technical teams in incorporating gender into their portfolio reviews. 10. Identify periodic check-in points to review progress against priorities. At a minimum, this can take the form of a team session at least twice a year to discuss successes, challenges, and actions for change (optimally just prior to portfolio review). TABLE 4: SUMMARY OF RECOMMENDATIONS BY AUDIENCE. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 23 Audience Recommendation Implementation Status Mission Leadership 1. Maintain a separate gender SDO. 2. Permanently house SDO4 leadership in the Program Office. 3. Hire or allocate one dedicated full-time SDO4 team member with gender expertise. 4. Hire or dedicate a full-time administrative support position to effectively implement SDO4. 5. Create two sub-teams for SDO4: management team and advisory team. 6. Highlight gender as a priority in senior management meetings and communicate efforts to improve management approaches for implementing SDO4. 7. Conduct regular (e.g., monthly) meetings between Deputy Mission Director responsible for managing the SDO4 team and the SDO4 management team. Technical Office Leads 1. Technical office leads conduct periodic check-ins (weekly or monthly) with their SDO4 team representative to encourage two-way communication regarding gender updates. The Recommended Management Structure for Gender SDO4 Core Overarching Recommendations: 1. The team leader position should be permanently located in the Program Office because it is best-situated to support/ work across Mission programming, which is critical for gender as a crosscutting DO and it builds continuity over time. 2. Establish a management team within the larger SDO4 team. 3. Hire a full time gender expert in the Program Office to serve on the SDO4 team. 4. Optimally, provide one full-time administrative assistant for SDO4. 5. Function as a central repository for accessing gender expertise. 6. Incorporate adaptive management approaches, where possible to revisit and fine-tune the structure and approaches (see pages 13 and 14 for detailed recommendations). 7. Clarify roles and responsibilities in the Mission Order and Team Charter, as appropriate. 8. Focus on achieving development results, including the results framework for SDO4 and across other DOs. TABLE 5.1: RECOMMENDED MANAGEMENT STRUCTURE FOR SDO4. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 24 SDO4 Issues and Recommendations Issue: Should team leadership rotate as originally envisioned? Recommendation: The evaluation team recommends that the team leader for SDO4 be located in the Program Office on a permanent basis in contrast to the original model where leadership would rotate across technical offices. Why? This is a complex issue and each model has its own advantages and disadvantages. The evaluation team carefully considered the feedback received from USAID staff on key issues, an identification of key factors for the effective functioning of any team, and an analysis of the advantages and disadvantages of each model. On balance, we believe locating team leadership in the Program Office is likely to provide the team with the greatest opportunity for success. Advantages and disadvantages for rotating leadership are summarized below: Advantages of Rotating Leadership: 1. One technical office takes greater ownership and is strongly engaged in gender. 2. By rotating this leadership role, each of the technical offices will have an opportunity to play this role. Disadvantages of Rotating Leadership: 1. Other technical offices may be less engaged with gender when not fulfilling a leadership role. 2. Opportunities for gender programming may be missed across programs. Even though the team has the representation of all technical offices, discussions, interaction, and ideas occur more naturally within the hosting office. This point emerged during KIIs, where technical offices outside of DRG did not have a strong sense of what the SDO4 team does. In addition, staff noted a concern that the hosting technical office would likely “capture” more of the SDO4 team’s focus. 3. This idea is driven by a principle of rotating leadership rather than considering what office and/or individual may be best-suited over the long term to manage the program. 4. Just as the leader is at the peak of his/her knowledge, insight and learning, and develops a rapport with the team, leadership changes. For the same reason this would be a disadvantage for other DOs (managed by office directors), it is a disadvantage for SDO4. Issue: Where should the AOR for SDO4 activities be located? Recommendation: The evaluation team recommends that USAID review the two options proposed (below) and determine the best choice. Why? The AOR for Takamol is currently located in the DRG office, as was the past Team Leader for SDO4. Moving forward, the Team Leader will be located in the Program Office. The key issue for any Team Leader is to define the relationship with the AOR because Takamol represents the main gender activity that drives SDO4. A key ingredient for the success of SDO4 is to ensure that the Team Leader continues to play an important role in influencing how Takamol contributes to the overall strategy. Having said this, there may be different options available to the Mission to do this, as outlined below, each with its own advantages and disadvantages. Option 1: Maintain the AOR in the DRG office, however, ensure that the alternative AOR is the SDO4 Team Leader. This model ensures that the team leader has a structural role in managing Takamol, albeit as a back-up AOR. It is based on the idea that Takamol has a more natural home TABLE 5.2 SDO4 ISSUES AND RECOMMENDATIONS USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 25 SDO4 Issues and Recommendations in the DRG office, because there is an alignment between the types of issues addressed by Takamol and issues that are normally addressed in the context of DRG programming (participation, engagement, public awareness, human rights, laws and policies). It is also important to note that DRG programming is often crosscutting as well. Option 2: Move the AOR to the Program Office. In this case, the AOR would be a member of the SDO4 “management team” (defined below) and located in the Program Office. This model brings the Team Leader and the AOR into more direct and clear structural alignment. Issue: How can the SDO4 team function more effectively? Recommendation: The evaluation team recommends that the SDO4 team establish a smaller management team within the larger SDO4 team. Why? This would help the team to function more effectively, with the management team focusing on the day-to-day management and implementation of gender-related work and the larger team serving as gender champions and providing periodic support to the management team when needed. Who? The criteria for who should be on each team is identified below, followed by an illustrative list of management team members for consideration (based on current membership). The evaluation team proposes that individuals on the management team:  Are the strongest gender experts available in the Mission  Have the time and skills to share in the workload and complete tasks on a regular basis  Manage gender activities relevant to the strategic priorities of SDO4  Are able to play a leadership role in the Mission  Have the capacity to both influence and work in partnership with other DO teams  Are highly committed to gender  Are skilled in building and maintaining relationships with Technical Offices across the Mission The evaluation team proposes that the individuals in the larger SDO4 team:  Are willing to function as gender champions  Function as a proactive voice on gender in their home office and share information about relevant issues back to the SDO4 team  Seek to identify opportunities to address gender issues  May not have strong skills in gender but have a commitment to learning more  Are willing to support the management team by performing periodic tasks as needed Illustrative Candidates for the Management Team: Team Leader USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 26 SDO4 Issues and Recommendations Deputy Team Leader AOR for Takamol Additional Representatives from each Technical Office and Program Office (as guided by criteria; in addition more than 1 team member may participate from one office depending on the complementary skills/ roles they may bring to the team) 1 Gender Expert (when hired) 1 Administrative Support Specialist Issue: How can the Mission ensure a channel of communication to senior management? Recommendation: The evaluation team recommends weekly or monthly meetings between SDO4 and the responsible deputy Mission director. Why? This would ensure a strong channel of communication to senior management. In addition, the evaluation team recommends that the SDO4 team leader organize periodic check-in points with senior management (meaning the responsible deputy Mission Director) to discuss what is working and not working and make appropriate adjustments. For other technical teams, who are responsible for managing a DO or an IR, the office director reports to the Deputy Mission Mirector (DMD). This provides an important channel of communication to understand senior management priorities and for staff to message key issues to senior management. In this case, the evaluation team recommends that the team leader report to the Program Office Director. However, at the same time, it is important for the team leader to be given sufficient authority and latitude to manage the team, while keeping the director informed of major issues. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 27 ANNEX A: EVALUATION STATEMENT OF WORK (SOW) USAID – Special Development Objective #4 Performance Evaluation Statement of Work (SOW) INTRODUCTION USAID/Jordan requires an external performance evaluation of the role and effectiveness of the Gender Special Development Objective team (SDO4). This special development objective was created to enhance gender equality and female empowerment across USAID programming. BACKGROUND AND ACTIVITY DESCRIPTION In analyzing gender equality and female empowerment in Jordan, the dichotomies are stark: multi￾degreed, well-traveled women of West Amman live side-by-side with women whose male relatives rarely allow them to leave home unaccompanied; the Constitution7 mandates equality for all citizens while other laws restrict women’s employment options; Jordan lifted its reservation to the Convention on Elimination of All Forms of Discrimination Against Women on freedom of movement and residence but left male dominance on family matters; statistics show more women than men enrolled in university but under 15 percent participate in the workforce due to factors including unconducive workplaces, poor transportation options, and pressure to conform to the norm of the “good woman” who prioritizes home and family. Newer trends are also disturbing: younger men, although afforded a broader world view via the internet, replacing their fathers as even more conservative “guardians” of their sisters; one of Jordan’s two shelters for gender-based violence victims reporting 913 beneficiaries in 2013, up from 299 in 2007; and educated women battling depression, unable to put their intellectual capital to work for lack of “suitable” opportunities. In 2012, USAID established as part of its 2013-2017 Country Development Cooperation Strategy (CDCS) an SDO to enhance gender equality and female empowerment. The purpose of this SDO is to pursue more effective investments in gender equality and female empowerment to accelerate progress in development. The premise of this SDO is that female empowerment and gender equality will be enhanced as changes in discriminatory social norms and practices are promoted, advocacy and policy reforms for women’s issues are enhanced, and access to female-centered services are expanded. The SDO would support and complement the other DOs through stand-alone results. At the same time, strategies to achieve female empowerment and gender equality across sectors will be integrated into the other DOs. Through responsiveness to the needs and interests identified by Jordanian women, the Mission will reach out to areas outside of Amman to respond to local needs. Furthermore, the Mission will support efforts to incorporate identified constraints to gender equality and female empowerment into policy discussions and decision-making processes at the national level, thus making the Government of Jordan and civil society entities more accountable and responsive to women across Jordan. To support the SDO, USAID/Jordan contracted out the “Takamol” activity to work with communities to raise awareness and promote attitude-shifting dialogue on gender equality while supporting partners in civil society and government to make specific improvements in conditions for women. Internally, USAID established the Gender DO Team composed of representatives from the technical and support offices within the Mission. The Team’s mandate is to help integrate gender across USAID’s programming, coordinate efforts within and among the USAID/Jordan portfolio, and 7 The World Bank’s 2013 Country Gender Assessment: Economic Participation, Agency and Access to Justice in Jordan; the 2012 Woman and Man in Jordan: A Statistical Portrait, produced by the Jordanian Department of Statistics and KVINFO; and USAID 2012 Gender Assessment provide comprehensive compilations of data and analysis of gender trends in the law, economy, social life and political participation. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 28 provide advice and programming guidance on gender issues. Each member is viewed as a gender advocate and Gender POC within their offices and communicates on a variety of issues to and from the Gender Team. To date, the Gender Team has been housed by the Democracy, Rights and Governance (DRG) Office; the Gender Team Leader and Takamol AOR are both DRG staff. In an effort to rotate leadership and maintain a robust team and portfolio, the leadership of the Gender Team was transferred in September 2016 to a staff member from the Program Office; the plan is that the leadership of this team will rotate every two years. PURPOSE OF THE EVALUATION The purpose of this evaluation is to provide USAID with an analysis and strategic recommendations on the effectiveness of the program strategy and the administrative approach that supports SDO4 “Gender Equality and Female Empowerment” in order to mainstream gender across USAID programming. The evaluation results will help inform USAID programmatic decisions for the future of the SDO and the Gender Team. OBJECTIVES AND THEORY OF CHANGE As articulated above, the premise of the SDO is that as men and women become more accepting of concepts of equality and practice these beliefs and as laws further reinforce these concepts, economic status of women will be elevated which will in turn have positive effects on family health, participation of women in other aspects of household decision-making, more involvement of men in domestic and caring roles, and women’s greater participation in community and civic life. Intermediate results and performance indicators were developed for this SDO. They were kept generic and broad in order to enable a variety of Mission programming to contribute to and feed into them. An activity, Takamol, was designed to both support the Mission’s efforts in terms of capacity building of Mission staff and IPs as well as fill in the gaps not addressed through USAID/Jordan’s multifaceted portfolio – namely the social dialogue and institutional capacity building components. The Gender Team was to provide overall guidance on the gender integration. An AOR was assigned to the Takamol activity to manage the day-to-day implementation of the activity and serve as the interlocutor with the Gender Team. The Gender SDO Results Framework and indicators are presented below. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 29 GENDER SDO RESULTS FRAMEWORK EVALUATION QUESTIONS In addressing the stated purpose, the evaluation will explicitly answer the questions stated below. EFFECTIVENESS AND SUSTAINABILITY: 1. To what extent has addressing gender gaps contributed to achieving USAID’s goals for expanding economic opportunities, increasing citizen participation, improving educational outcomes and achieving more sustainable population growth? 2. Has USAID been able to build sustainable synergies across USAID activities? Which interventions (either individually or within a thematic area) have been effective to build synergies and have these synergies been maximized? In what ways can they be improved? 3. How effective has the Gender DO Team been in integrating gender across the Mission? a. How did USAID’s approach to managing and implementing gender integration impact effectiveness? For example, was creating a separate DO the best approach to achieving the desired objectives or are there other more effective approaches. b. Do the team members receive sufficient support from mid-level management to be active members and confident Gender experts of the DO team and feel empowered to pursue increasing gender integration? c. What has the DO team done well in gender integration and what can be improved? AVAILABLE BACKGROUND MATERIALS  USAID/Jordan 2013-2017 CDCS  Mission Order on Gender Integration  Gender Project Appraisal Document (PAD)  Gender Sections from the Annual Operational Plan (OP) and Performance Plan and Report (PPR) USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 30 EVALUATION DESIGN AND METHODOLOGY The external evaluation will use quantitative and/or qualitative approaches to assess how effective SDO4 was in meeting its objectives and to provide recommendations on how the Mission can effectively integrate and mainstream gender into its programming and achieve its development objectives over the remainder of the CDCS life (which is being extended to November 2019). The evaluation will utilize a participatory approach to answer valuable qualitative questions. It is expected that the evaluation will focus primarily on USAID and will draw on other stakeholders as appropriate, including USAID IPs, and potentially other key informants such as members of the Jordanian public sector entities, civil society organizations, academia and others, to help answer the evaluation questions. Simultaneously with this evaluation, USAID is undertaking a performance evaluation of the Takamol activity to assess its effectiveness in meeting its goals and provide recommendations to help guide implementation of the activity over its remaining two years. Since Takamol is the main activity under this SDO, it is expected that findings and recommendations from the Takamol evaluation will be closely coordinated and feed into the overall SDO assessment, and vice versa. EVALUATION TEAM COMPOSITION In order to meet the requirements of team composition, ensure data quality, and contribute to building capacity of local evaluation specialists, the following is suggested for team composition: 1. Team Leader (With relevant experience related to strategy, operations and management processes in USAID Missions including within and between technical offices) 2. Gender Specialist (With relevant gender and programming experience) The evaluation team for this evaluation may also need to work closely with the Takamol evaluation team; for example, data collection tools and methods could be developed to inform both the Takamol evaluation as well as the SDO assessment. At a minimum, it is expected that this evaluation team will coordinate closely with the Takamol evaluation team. The evaluation team will also be supported by the MESP Evaluation Manager, Senior M&E Specialist and Evaluation Assistant. PERFORMANCE PERIOD The evaluation will be conducted from January 2017 through April 2017 with data collection conducted in January and February 2017, and final report submitted by the end of April 2017. Logistics for the assessment and evaluation will be provided by MESP. DELIVERABLES AND TIMELINE Deliverable Timeline MESP finalize SOW, begin desk review of relevant DO4 documents and develop work plan January 2017 MESP develop evaluation methodology and tools, finalize work plan and submit evaluation design report February 2017 Field Work February 2017 Debriefing presentation for USAID and stakeholders on evaluation findings, initial conclusions and recommendations April 2017 USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 31 Deliverable Timeline USAID and Evaluation team collaboration on developing the recommendations  USAID Participation in the team FCR Session  Meeting/s between the evaluation team and USAID to develop actionable recommendations April 2017 MESP submit draft report April 2017 MESP submit final report April 2017 NOTES:  The evaluation report will not exceed 30 pages and must adhere to USAID’s Evaluation Policy; it must include a table of contents, list of acronyms, and executive summary as well as a copy of the SOW and data collection instruments;  The report will address each of the key questions identified in the relevant sections of the SOW and any other factors the team considers to have a bearing on the objectives of the evaluation;  The key evaluation questions must be answered, and recommendations must be stated in an actionable way with defined responsibility for the action;  Sources of information will be properly identified and listed in an annex;  The assessment and evaluation reports will be published on USAID’s Development Experience Clearinghouse at edec.usaid.gov.  Upon request from USAID or closure of MESP, both electronic and hard copy data files will be transferred to USAID. In the meantime, electronic files are on the MESP file and hard copies are warehoused at MESP. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 33 ANNEX B: EVALUATION DESIGN WITH INSTRUMENTS BACKGROUND AND PURPOSE In 2012, as part of its 2013-2017 Country Development Cooperation Strategy (CDCS), USAID established a Special Development Objective (SDO) to enhance gender equality and female empowerment. The purpose of this SDO is to pursue more effective investments in gender equality and female empowerment to accelerate progress in development. The premise of this SDO is that female empowerment and gender equality will be enhanced as changes in discriminatory social norms and practices are promoted, advocacy and policy reforms for women’s issues are enhanced, and access to female-centered services are expanded. The SDO would support and complement the other DOs through stand-alone results. At the same time, strategies to achieve female empowerment and gender equality across sectors will be integrated into the other DOs. To support the SDO, USAID/Jordan contracted out the “Takamol” activity to work with communities to raise awareness and promote attitude-shifting dialogue on gender equality while supporting partners in civil society and government to make specific improvements in conditions for women. Internally, USAID established the Gender DO Team composed of representatives from the technical and support offices within the Mission. The Team’s mandate is to help integrate gender across USAID’s programming, coordinate efforts within and among the USAID/Jordan portfolio, and provide advice and programming guidance on gender issues. Each member is viewed as a gender advocate and Gender POC within their offices and communicates on a variety of issues to and from the Gender Team. To date, the Gender Team has been housed by the Democracy, Rights and Governance (DRG) Office; the Gender Team Leader and Takamol AOR are both DRG staff. In an effort to rotate leadership and maintain a robust team and portfolio, the leadership of the Gender Team was transferred in September 2016 to a staff member from the Program Office; the plan is that the leadership of this team will rotate every two years. The purpose of this evaluation is to provide USAID with an analysis and strategic recommendations on the effectiveness of the program strategy and the administrative approach that supports SDO4 “Gender Equality and Female Empowerment” in order to mainstream gender across USAID programming. The evaluation results will help inform USAID programmatic decisions for the future of the SDO and the Gender Team. EVALUATION QUESTIONS 1. To what extent has addressing gender gaps contributed to achieving USAID’s goals for expanding economic opportunities, increasing citizen participation, improving educational outcomes and achieving more sustainable population growth? 2. Has USAID been able to build sustainable synergies across USAID activities? Which interventions (either individually or within a thematic area) have been effective to build synergies and have these synergies been maximized? In what ways can they be improved? 3. How effective has the Gender DO Team been in integrating gender across the Mission? a. How did USAID’s approach to managing and implementing gender integration impact effectiveness? For example, was creating a separate DO the best approach to achieving the desired objectives or are there other more effective approaches. b. Do the team members receive sufficient support from mid-level management to be active members and confident Gender experts of the DO team and feel empowered to pursue increasing gender integration? c. What has the DO team done well in gender integration and what can be improved? USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 34 EVALUATION DESIGN UNDERSTANDING OF THE EVALUATION QUESTIONS The three evaluation questions focus on three key themes, including: the achievement of program outcomes, creating sustainable synergies, and management effectiveness. Below we provide an outline of how the evaluation team will capture and operationalize these concepts to answer the three evaluation questions. MEASURING EFFECTIVENESS Given the scope and time available for this evaluation, the Evaluation Team will operationalize efforts to address gender gaps and create more gender equitable outcomes by examining effectiveness at the program level rather than at an activity-level. This will allow the Team to link efforts to the overarching USAID goals referenced in Question 1. The Evaluation Team will consider to what extent the Technical Offices are addressing key gender issues as identified in the DO 4 PPRs (FY14- 16) and 2012 Gender Assessment. Effectiveness will also be examined through the purpose of the DO 4 Team and the structure of the DO itself. The DO 4 Team asserts a twofold purpose in its draft Charter: to oversee any design and implement projects directly under the DO (currently the Takamol project); and to work throughout the mission to integrate gender equality by working with different technical offices, assisting with gender analysis, conducting training, assisting projects, collecting and reporting on indicators. In addition to researching the relationships the DO 4 Team has with other offices and teams within the mission, the Evaluation Team will consider the support the DO 4 Team receives from the mission, particularly regarding gender training, funding, and communications and outreach. The structure of DO 4 and its Team will also be looked at, with regards to its current organization, any changes made since its inception, and the factors that enable or inhibit gender integration as associated with its existing structure. As feasible, it will also be important to gain a better understanding of how and to what extent gender was integrated into programming prior to the CDCS (2013-2017). Roles and responsibilities of mission offices as outlined in the Mission Order: Implementation of Policy on Gender Equality and Female Empowerment and the extent to which they are being fulfilled will also be used as a measure of effectiveness. The evaluation team will operationalize the key concept of synergy by assessing the ability of the DO4 team to view activities across the entire portfolio and identify opportunities to work across programs to maximize impact. The concept of sustainability will be assessed based on understanding whether systems, processes, policies and behaviors that support gender integration have emerged as a result of DO4 work, and are able to exist independently, that is even without any additional DO4 support. APPROACH TO ANSWERING THE EVALUATION QUESTIONS This section details the evaluation team’s approach to answering the evaluation questions. Combined with document review, the evaluation team will utilize qualitative key informant interviews (KIIs)) as their primary data collection approach to capture the opinions and perspectives of the different stakeholders, ranging from various offices within USAID/Jordan to implementing partners of current activities. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 35 Question 1. To what extent has addressing gender gaps contributed to achieving USAID’s goals for expanding economic opportunities, increasing citizen participation, improving educational outcomes and achieving more sustainable population growth? The team will talk to mission staff in KIIs to ascertain how gender is integrated and assess how addressing gender gaps has contributed to achieving USAID/Jordan’s objectives. The team will combine both secondary and primary data and include the point of view of different actors across technical offices (specifically Democracy, Rights & Governance, Economic Development & Energy, Education & Youth, Health as their goals align with those referenced in Question 1) and DO teams, and mission leadership. Questions will focus on identifying the prevailing gender gaps in the labor market, civic participation, education, and health sectors and learning how intentional efforts to mitigate these gaps have led to more gender equitable outcomes. The Evaluation Team will also review program level reports such as the CDCS (to understand key objectives) in addition to the PMP and PPR. We will also examine DO 4: Gender Equality and Female Empowerment Enhanced, 2012 Gender Assessment, and other documents to inform our contextual understanding of key gender issues in Jordan. Question 2. Has USAID been able to build sustainable synergies across USAID activities? Which interventions (either individually or within a thematic area) have been effective to build synergies and have these synergies been maximized? In what ways can they be improved? By interviewing DO 4 Team members, DO Team Leads and Technical Office Leads, and implementing partners in KIIs, the Evaluation Team seeks to learn whether the DO 4 Team has been able to share and replicate successful interventions in other activities. In particular, this question highlights the need for coordination with the Takamol evaluation team. Their SOW includes a similar evaluation question: Has Takamol built synergies with other activities across the Mission portfolio? Why or why not? In what ways can they be improved? The SDO4 evaluation will focus on whether the DO 4 team has been able to build sustainable synergy across the other DO’s. In contrast, the Takamol evaluation team will focus on how the Takamol activity built synergy with other activities, and will also identify enablers and barriers to synergy. Question 3. How effective has the Gender DO Team been in integrating gender across the Mission? a. How did USAID’s approach to managing and implementing gender integration impact effectiveness? For example, was creating a separate DO the best approach to achieving the desired objectives or are there other more effective approaches. b. Do the team members receive sufficient support from management to be active members and confident Gender experts of the DO team and feel empowered to pursue increasing gender integration? c. What has the DO team done well in gender integration and what can be improved? The team will talk to mission staff in key informant interviews to learn how well the Gender DO Team is being supported by the mission, how they interact with other mission staff, and the status of gender integration across the mission. The team will also interview a representative from USAID/Tanzania (the only other Mission with a stand-alone gender DO), USAID/Egypt (addresses gender as a crosscutting issue), and the Regional Bureau’s gender advisor. Youth may also be examined within the context of USAID/Jordan’s program to compare the issues related to a crosscutting program in contrast to a stand-alone DO. The Evaluation Team will also conduct a document review, drawing from the Mission Order: Implementation of Policy on Gender Equality and Female Empowerment, draft DO Team 4 Charter, and other sources. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 36 The team will utilize all data streams (primary and secondary data) to identify lessons learned and opportunities for improvement. Additionally, the SDO4 Evaluation Team will seek to share findings with and draw information from the Takamol Evaluation Team, which has a related evaluation question: What external factors and challenges influenced Takamol’s performance, and how? DATA COLLECTION METHODS This evaluation will rely on secondary and primary data collection. SECONDARY DATA COLLECTION The team will review and incorporate into the analysis relevant secondary data sources, following are some key pieces of secondary data that the team has considered: 1. USAID/Jordan CDCS 2. USAID/Jordan DO 4 Gender Results Framework) 3. USAID/Jordan’s Performance Management Plan (PMP) 4. Jordan Mission Order: Implementation of Policy on Gender Equality and Female Empowerment 5. Assessment of the Implementation of USAID’s Gender Equality and Female Empowerment Policy: Annex A, The USAID Gender Equality and Female Empowerment Policy: A Report on Implementation (June 2016, EnCompass) 6. Assessment of the Implementation of USAID’s Gender Equality and Female Empowerment Policy: Annex B, Report on Gender Integration in CDCSs 2011-2015 (May 2016, Insight Systems Corporation in collaboration with PPL) 7. Assessment of the Implementation of USAID’s Gender Equality and Female Empowerment Policy: Annex C, Gender Integration in USAID Appraisal Documents (June 2016, PPL) 8. Assessment of the Implementation of USAID’s Gender Equality and Female Empowerment Policy (Final Report, November 2016, PPL) 9. DO Team 4 Charter 10. Special DO 4 Performance Indicator Reference Sheet 11. Lessons Learned: Gender Policy and DRG Strategy Implementation Assessments (PPL webinar slides) 12. Gender Equality and Female Empowerment Enhanced FY2014 PPR 13. Gender Equality and Female Empowerment Enhanced FY2015 PPR 14. Gender Equality and Female Empowerment Enhanced FY2016 PPR 15. 2012 Gender Assessment USAID/Jordan 16. Takamol Mid-Term Performance Evaluation Report PRIMARY DATA COLLECTION For primary data collection, the evaluation team will primarily rely on qualitative research. Qualitative Research: Under qualitative research, the evaluation team plans on doing key informant interviews (KII). We will explore the use of group discussions where appropriate and as time permits. Quantitative Research: Under quantitative research, the evaluation team plans to conduct a survey with current and former SDO4 team members to ascertain their perceived strengths and weaknesses of the SDO4 team. Below is a summary of the data collection plan: Key informant in-depth interviews will be conducted with the following key informants. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 37 MISSION LEADERSHIP  Mission Director  Deputy Mission Director DO 4 TEAM  DO 4 Team Lead (past and current),  Mission’s Gender POC (current)  DO 4 Team members  Including representatives from these five DO Team 4 working groups  Monitoring & Evaluation  Research & Analysis  Grants  Events & Outreach  Program, Policy Implementation, and Integration PROGRAM OFFICE  Program Office Staff TECHNICAL OFFICES  Technical Office Leads/POCs  Democracy, Rights and Governance  Economic Development & Energy  Education and Youth  Population and Family Health  Water Resources and Environment  Other Technical Office staff as time permits ACTIVITIES/IMPLEMENTING PARTNERS We expect the Takamol evaluation to conduct most of the interviews with USAID IPs, however, we may interview specific IPs based on our discussions with key stakeholders.  Takamol (IREX) OTHER USAID MISSIONS  USAID/Tanzania  Other Missions as recommended by USAID USAID WASHINGTON  Office of Gender Equality and Women’s Empowerment  The Middle East and Africa Regional Bureau Gender Advisors USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 38 The survey will be conducted with current and former members of the SDO4 team. DATA COLLECTION INSTRUMENTS The data collection instruments (Annex III) have been developed to address the research questions and tailored to the type of respondent and the data collection approach. DATA ANALYSIS METHODS Data collected via document review, survey, and KIIs (interview notes) will be analyzed and synthesized in a matrix format, allowing the Evaluation Team to conduct a content analysis and visually identify key trends and themes. The team will also assess findings by respondent types (for example Program Office, Mission Leadership, Technical Teams etc.) to understand and compare the perspectives of the different stakeholders. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 39 ANNEX C: GETTING TO ANSWERS Program or Project: DO 4 Evaluation Team Members: Michelle Adams Matson, Aleksandra Markovich, Evaluation Questions Type of Answer/ Evidence Needed (Check one or more, as appropriate) Methods for Data Collection, Sampling or Selection Approach, (if one is needed) Data Analysis Methods Data Source(s) Method Effectiveness To what extent has addressing gender gaps contributed to achieving USAID’s goals for expanding economic opportunities, increasing citizen participation, improving educational outcomes and achieving more sustainable population growth? Yes/No  USAID Staff  Documents  In depth interviews  Document Review  Content analysis of KIIs  Content analysis of the documents * Description Comparison * Explanation Has USAID been able to build sustainable synergies across USAID activities? * Yes/No  USAID Staff  Documents  In depth interviews USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 40 Evaluation Questions Type of Answer/ Evidence Needed (Check one or more, as appropriate) Methods for Data Collection, Sampling or Selection Approach, (if one is needed) Data Analysis Methods Data Source(s) Method Which interventions (either individually or within a thematic area) have been effective to build synergies and have these synergies been maximized? In what ways can they be improved?  Document Review  Content analysis of KIIs  Content analysis of document review * Description Comparison * Explanation 3. How effective has the Gender DO Team been in integrating gender across the Mission? a. How did USAID’s approach to managing and implementing gender integration impact effectiveness? For example, was Yes/No  USAID Staff  Documents  In depth interviews  Document Review  Survey  Content analysis of KIIs  Content analysis of document review * Description Comparison * Explanation USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 41 Evaluation Questions Type of Answer/ Evidence Needed (Check one or more, as appropriate) Methods for Data Collection, Sampling or Selection Approach, (if one is needed) Data Analysis Methods Data Source(s) Method creating a separate DO the best approach to achieving the desired objectives or are there other more effective approaches. b. Do the team members receive sufficient support from mid-level management to be active members and confident Gender experts of the DO team and feel empowered to pursue increasing gender integration? c. What has the DO team done well in gender integration and what can be improved?  Content analysis of survey USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 42 Evaluation Questions Type of Answer/ Evidence Needed (Check one or more, as appropriate) Methods for Data Collection, Sampling or Selection Approach, (if one is needed) Data Analysis Methods Data Source(s) Method ANNEX D: WORK PLAN DESCRIPTION DELIVERABLE RESPONSIBLE DATES Planning Desk review of project documents  Desk review  List of evaluation reference documents Team Leader Evaluation Team January 25, 2017 – February 10, 2017 Work plan submitted to USAID for approval  Work plan/schedule Team Leader Evaluation Team February 5, 2017 USAID approves work plan TBD Design of the evaluation: methodology and tools  Research design report  Draft instruments in English Team leader Evaluation Team February 5, 2017 USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 43 DESCRIPTION DELIVERABLE RESPONSIBLE DATES USAID approves evaluation research design report TBD Implementation Phase Data Collection Phase 1 Data Collection through key informant interviews (KII)  KII notes and summary  Data Review Matrix Evaluation Team February 5 -13, 2017 USAID Mid-Point Check-In  Meeting Team Leader Evaluation Team February 9, 2017 Analysis, De-briefing and Reporting Phase Initial Data Analysis  KII notes and summary  Focus group notes Team Leader Evaluation Team February 11-15, 2017 Debriefing of initial findings with USAID DO 4 Team  Presentation Team Leader Evaluation Team and MESP February 16, 2017 USAID Consultation: Initial findings and actionable recommendations  Presentation Team leader Evaluation Team and MESP February 16, 2017 USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 44 DESCRIPTION DELIVERABLE RESPONSIBLE DATES Final data analysis and drafting of the report  FCR Table  Draft report Team Leader & Evaluation Team February 19-March 22, 2017 Draft report is submitted  Report Evaluation Team March 23, 2017 USAID comments on draft evaluation report  Draft report with comments USAID April 10, 2017 Response to the USAID comments and update report  Updated report and response to comment table Team Leader, Evaluation Team and MESP April 13-27, 2017 Final evaluation report incorporating USAID comments  Final evaluation report MESP, Team Leader and Evaluation Team April 28, 2017 USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 45 ANNEX E: DATA COLLECTION INSTRUMENTS SDO4 EVALUATION –QUESTIONS FOR DO 4 (GENDER) TEAM INTRODUCTION Thank you for speaking to us today and helping us gather information for the evaluation of Special Development Objective 4. The purpose of the evaluation is to provide USAID with an analysis and strategic recommendations on the effectiveness of the program strategy and the administrative approach that supports SDO4 “Gender Equality and Female Empowerment” in order to mainstream gender across USAID programming. The evaluation results will help inform USAID programmatic decisions for the future of the SDO and the Gender Team. The findings from these interviews will be analyzed in aggregate and nothing you share with us will be attributed to you. In case we would like to share a quote from this conversation, we will obtain your permission first. Do you have any questions at this stage? We have a number of questions to ask and want to begin by asking you to tell us about your current role in the mission. How long have you worked in the mission? Do you have gender responsibilities in your position description? ACHIEVEMENT 1. In your area of responsibility, what are the major gender issues? 2. To what extent do you believe that addressing gender has helped USAID achieve its objectives a. Could you tell give us some examples? SUSTAINABLE SYNERGIES 1. What are DO 4 goals related to sustainability? Could you please elaborate on the approach/es used to achieve these goals? 2. What are DO 4 goals in relation to creating synergies across the program? Could you please elaborate on the approaches used to achieve these goals? MANAGEMENT EFFECTIVENESS 1. How effective has the Gender DO Team been in integrating gender across the mission? What are some examples? a. In your opinion what are the main challenges when it comes to integrating gender across the mission? 2. What is the role of the DO 4 Team in working with other technical offices? Has this role remained the same over time or has it changed? IF CHANGED, ASK: Why? USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 46 3. What is your role on the DO 4 Team? 4. Is USAID’s approach to managing and implementing gender integration effective? 5. Is a separate DO the best approach? 6. Do you feel you receive adequate information and support from management to be effective? 7. Do you feel you have the adequate skills to be effective? 8. Have you attended gender trainings and events? 9. How well does the Gender DO team operate as a team? Why do you think this is? 10. How often do you meet? 11. What do you typically discuss? 12. What has the DO team done well? What can be improved? Why do you say that? 13. How are you able to influence decision-making in your teams (home office) and other DOs? 14. Based on your experience, what are some key lessons learned or changes that need to be made in order for DO4 team to be more effective? OTHER 1. Are there any other issues we have not mentioned that you would like to discuss? 2. Is there anyone else you recommend we interview? SDO4 EVALUATION – QUESTIONS FOR TECHNICAL OFFICES INTRODUCTION Thank you for speaking to us today and helping us gather information for the evaluation of Special Development Objective 4. The purpose of the evaluation is to provide USAID with an analysis and strategic recommendations on the effectiveness of the program strategy and the administrative approach that supports SDO4 “Gender Equality and Female Empowerment” in order to mainstream gender across USAID programming. The evaluation results will help inform USAID programmatic decisions for the future of the SDO and the Gender Team. The findings from these interviews will be analyzed in aggregate and nothing you share with us will be attributed to you. In case we would like to share a quote from this conversation, we will obtain your permission first. Do you have any questions at this stage? We have a number of questions to ask and want to begin by asking you to tell us about your current role in the mission. How long have you worked in the mission? Do you have gender responsibilities in your position description? USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 47 Have you attended gender trainings? If so, what are they? ACHIEVEMENT 1. What are some of the prevailing gender issues (including gaps) in your technical area? 2. How and to what extent has your technical office made intentional efforts to identify and address gender in your programming? What are some examples of successes? 3. To what extent has addressing gender contributed to or enabled the achievement of your objectives? Could you give some examples? SUSTAINABLE SYNERGIES 1. When it comes to integrating gender into your work, are there are any examples of synergies between your team and other technical teams, and/or between your team and the DO4 team? If so, what are they? Optional if not covered in previous responses 2. Do you feel gender integration is important and/or relevant to your daily role/work? Why do you say that, please elaborate? MANAGEMENT EFFECTIVENESS 1. Is creating a separate DO for gender the best approach to achieving objectives? a. Are there better approaches? 2. To what extent have you engaged the DO 4 Team for support? a. What support did you receive? b. What support would you like to receive? c. What has the DO 4 Team done well? What can be improved? OTHER 1. Are there any other issues we have not mentioned that you would like to discuss? 2. Is there anyone else you recommend we interview? SDO4 EVALUATION – QUESTIONS FOR MISSION LEADERSHIP INTRODUCTION Thank you for speaking to us today and helping us gather information for the evaluation of Special Development Objective 4. The purpose of the evaluation is to provide USAID with an analysis and strategic recommendations on the effectiveness of the program strategy and the administrative approach that supports SDO4 “Gender Equality and Female Empowerment” in order to mainstream gender across USAID programming. The evaluation results will help inform USAID programmatic decisions for the future of the SDO and the Gender Team. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 48 The findings from these interviews will be analyzed in aggregate and nothing you share with us will be attributed to you. In case we would like to share a quote from this conversation, we will obtain your permission first. Do you have any questions at this stage? We have a number of questions to ask and want to begin by asking you to tell us about your current role in the mission. How long have you worked in the mission? Do you have gender responsibilities in your position description? Have you attended gender trainings? ACHIEVEMENT 1. What are some of the prevailing gender issues across the mission? 2. What is your overall appraisal of USAID efforts to address gender gaps? a. In your opinion, what has worked? What has not worked? Why do you say that? 3. How and to what extent has the mission made intentional efforts to identify and address gender? What are some examples of successes? 4. To what extent has addressing gender contributed to or enabled the achievement of mission objectives? SUSTAINABLE SYNERGIES 1. Are there are any examples of synergies across the portfolio? If so, what are they? Optional if not covered in previous responses 2. What are the goals related to sustainability? MANAGEMENT EFFECTIVENESS 1. Is USAID’s approach to managing and implementing gender integration effective? a. Is a separate DO the best approach? 2. What has the DO team done well? Why do you say that? a. What can be improved? OTHER 1. Are there any other issues we have not mentioned that you would like to discuss? 2. Is there anyone else you recommend we interview? USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 49 SDO4 EVALUATION – QUESTIONS FOR PROGRAM OFFICE INTRODUCTION Thank you for speaking to us today and helping us gather information for the evaluation of Special Development Objective 4. The purpose of the evaluation is to provide USAID with an analysis and strategic recommendations on the effectiveness of the program strategy and the administrative approach that supports SDO4 “Gender Equality and Female Empowerment” in order to mainstream gender across USAID programming. The evaluation results will help inform USAID programmatic decisions for the future of the SDO and the Gender Team. The findings from these interviews will be analyzed in aggregate and nothing you share with us will be attributed to you. In case we would like to share a quote from this conversation, we will obtain your permission first. Do you have any questions at this stage? We have a number of questions to ask and want to begin by asking you to tell us about your current role in the mission. How long have you worked in the mission? Do you have gender responsibilities in your position description? Have you attended gender trainings? ACHIEVEMENT 1. What are some of the prevailing gender issues across the mission? 2. What is your overall appraisal of USAID efforts to address gender gaps? a. In your opinion, what has worked? What has not worked? Why do you say that? 3. How and to what extent has your team made intentional efforts to identify and address gender in your work? What are some examples of successes? 4. To what extent has addressing gender contributed to or enabled the achievement of mission objectives? SUSTAINABLE SYNERGIES 1. When it comes to integrating gender into your work, are there are any examples of synergies across the portfolio? If so, what are they? Optional if not covered in previous responses 2. Do you feel gender integration is important and/or relevant to your daily role/work? Why do you say that, please elaborate? MANAGEMENT EFFECTIVENESS 1. In your opinion what are the main challenges when it comes to integrating gender across the mission? 2. Is USAID’s approach to managing and implementing gender integration effective? USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 50 a. Is a separate DO the best approach? 3. What is the role of the DO 4 Team in working with the Program Office? Has this role remained the same over time or has it changed? IF CHANGED, ASK: Why? 4. To what extent have you engaged the DO 4 Team for support? a. What support did you receive? 5. What has the DO team done well? Why do you say that? a. What can be improved? OTHER 1. Are there any other issues we have not mentioned that you would like to discuss? 2. Is there anyone else you recommend we interview? SDO4 EVALUATION – QUESTIONS FOR TAKAMOL INTRODUCTION Thank you for speaking to us today and helping us gather information for the evaluation of Special Development Objective 4. The purpose of the evaluation is to provide USAID with an analysis and strategic recommendations on the effectiveness of the program strategy and the administrative approach that supports SDO4 “Gender Equality and Female Empowerment” in order to mainstream gender across USAID programming. The evaluation results will help inform USAID programmatic decisions for the future of the SDO and the Gender Team. The findings from these interviews will be analyzed in aggregate and nothing you share with us will be attributed to you. In case we would like to share a quote from this conversation, we will obtain your permission first. Do you have any questions at this stage? What is your role in supporting USAID’s DO and technical offices in other sectors? ACHIEVEMENT 1. What are some of the prevailing gender issues across the mission? 2. What is your overall appraisal of USAID efforts to address gender gaps? a. In your opinion, what has worked? What has not worked? Why do you say that? 3. To what extent has addressing gender contributed to or enabled the achievement of mission objectives in other sectors? SUSTAINABLE SYNERGIES 1. Is it your role to try to strengthen synergies across USAID’s portfolio? How and what are some examples? USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 51 2. What are the goals related to sustainability in relation to supporting the mission’s strategy? MANAGEMENT EFFECTIVENESS 1. What is your role in relation to SDO4? 2. To what extent have you worked with the SDO4 Team? 3. Have you provided support to the SDO4 Team? USAID/Jordan Technical Offices? Other IPs? a. What support did you provide? 4. In your opinion what are the main challenges when it comes to integrating gender across the mission? 5. Is USAID’s approach to managing and implementing gender integration effective? Is a separate DO the best approach? OTHER 1. Are there any other issues we have not mentioned that you would like to discuss? 2. Who else do you think we should interview? SDO4 EVALUATION –QUESTIONS FOR USAID MISSION GENDER POCS INTRODUCTION Thank you for speaking to us today and helping us gather information for the evaluation of Special Development Objective 4. The purpose of the evaluation is to provide USAID with an analysis and strategic recommendations on the effectiveness of the program strategy and the administrative approach that supports SDO4 “Gender Equality and Female Empowerment” in order to mainstream gender across USAID programming. The evaluation results will help inform USAID programmatic decisions for the future of the SDO and the Gender Team. The findings from these interviews will be analyzed in aggregate and nothing you share with us will be attributed to you. In case we would like to share a quote from this conversation, we will obtain your permission first. Do you have any questions at this stage? Tell us about your current role in the mission. How long have you worked in the mission? Do you have gender responsibilities in your position description? ACHIEVEMENT 1. What is your overall appraisal of efforts to address gender gaps in your Mission? a. In your opinion, what has worked? What has not worked? Why do you say that? 2. How and to what extent has the mission made intentional efforts to identify and address gender? What are some examples of successes? USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 52 3. To what extent has addressing gender contributed to or enabled the achievement of mission objectives? MANAGEMENT EFFECTIVENESS 1. How does your mission manage and implement gender across the portfolio? What is the management structure? 2. Is your Mission’s approach to managing and implementing gender integration effective? a. Is a separate DO the best approach? Is addressing gender as a crosscutting issue the best approach? 3. Do you think the Mission/Mission staff afford gender the level of attention it deserves? Why or why not? 4. What do you think the factors of success are? 5. What has the mission done well? What can be improved? Why do you say that? OTHER 1. Are there any other issues we have not mentioned that you would like to discuss? 2. Is there anyone else you recommend we interview? SDO4 EVALUATION –QUESTIONS FOR USAID WASHINGTON GENDER POCS INTRODUCTION Thank you for speaking to us today and helping us gather information for the evaluation of Special Development Objective 4. The purpose of the evaluation is to provide USAID with an analysis and strategic recommendations on the effectiveness of the program strategy and the administrative approach that supports SDO4 “Gender Equality and Female Empowerment” in order to mainstream gender across USAID programming. The evaluation results will help inform USAID programmatic decisions for the future of the SDO and the Gender Team. The findings from these interviews will be analyzed in aggregate and nothing you share with us will be attributed to you. In case we would like to share a quote from this conversation, we will obtain your permission first. Do you have any questions at this stage? Tell us about your current role. How long have you worked at USAID? With the USAID/Jordan in particular? What is your level of involvement with the Jordan Mission and SDO team in particular? Do you have gender responsibilities in your position description? USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 53 ACHIEVEMENT 1. What is your overall appraisal of efforts to address gender gaps in the Middle East region? a. In your opinion, what has worked? What has not worked? Why do you say that? 2. To the best of your knowledge, has USAID/Jordan made intentional efforts to identify and address gender? What are some examples of successes? 3. To the best of your knowledge, to what extent has addressing gender contributed to or enabled the achievement of USAID/Jordan’s objectives? MANAGEMENT EFFECTIVENESS 1. As you know, Jordan has a special DO for gender. What are other missions in the region doing in terms of gender integration, particularly in terms of management structure? 2. Do you think a separate DO for gender is an effective approach to managing and implementing gender integration? a. Is addressing gender as a crosscutting issue a better approach? 3. What are the factors of success for successful gender integration in your mind? SURVEY FOR SDO4 TEAM PURPOSE 1. Team members do not understand or agree to the team’s purpose, goals and objectives. Team members understand very well and fully agree to the team’s purpose, goals and objectives. 1 - 2 - 3 - 4 - 5 EMPOWERMENT 2. Team members do not have confidence that the team can overcome obstacles and achieve its objectives. Team members have a high level of confidence that the team can overcome obstacles and achieve its objectives. 1 - 2 - 3 - 4 - 5 USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 54 3. Team members do not respect one another’s skills and commitment. Team members respect very highly one another’s skills and commitment. 1 - 2 - 3 - 4 - 5 4. The team does not have the policies, rules and processes needed to accomplish the team’s objectives. The team has the policies, rules and processes needed to accomplish the 1 - 2 - 3 - 4 - 5 team’s objectives. 5. The team does not provide its members with opportunities to grow and learn new skills. The team provides its members with many opportunities to grow and learn new skills. 1 - 2 - 3 - 4 - 5 RELATIONSHIPS AND COMMUNICATION 6. Team members do not express their opinions and feelings openly and directly. Team members express their opinions and feelings very openly and freely. 1 - 2 - 3 - 4 - 5 7. Team members do not provide one another with helpful reactions. Team members provide one another with helpful reactions and advice. 1 - 2 - 3 - 4 - 5 8. There is apparently very little trust among team members. There is apparently a high level of trust among team members. 1 - 2 - 3 - 4 - 5 USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 55 9. Conflict over ideas among team members is not being managed effectively, and it is hindering the team’s progress. Conflict over ideas among team members is being managed effectively, and it is contributing to the team’s progress. 1 - 2 - 3 - 4 - 5 10. Conflict involving personalities among team members is high and it is not being managed effectively. Conflict involving personalities among team members is low, but when it occurs, it is being managed effectively. 1 - 2 - 3 - 4 - 5 FLEXIBILITY 11. Team members are unable or unwilling to perform different roles and tasks as needed. Team members are able and willing to perform different roles and tasks as needed. 1 - 2 - 3 - 4 - 5 12. The responsibility for team development and leadership is not shared by team members. The responsibility for team development and leadership is shared widely among team members. 1 - 2 - 3 - 4 - 5 OPTIMAL PERFORMANCE 13. There is very low commitment to high standards and quality results among team members. There is very high commitment to high standards and quality results among team members. 1 - 2 - 3 - 4 - 5 USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 56 14. Team members are not committed to getting the job done on time. Team members are very committed to getting the job done on time. 1 - 2 - 3 - 4 - 5 15. The team does not have effective decision-making and problem-solving processes. The team has effective decision￾making and problem-solving processes. 1 - 2 - 3 - 4 - 5 RECOGNITION AND APPRECIATION 16. Team members rarely recognize or celebrate individual and team accomplishments. Team members frequently recognize or celebrate individual and team accomplishments. 1 - 2 - 3 - 4 - 5 17. The team receives a low level of support and appreciation from the larger organization and senior management. The team receives a high level of support and appreciation from the larger organization and senior management. 1 - 2 - 3 - 4 - 5 MORALE 18. Team members are not enthusiastic about working on the team. Team members are very enthusiastic about working on the team. 1 - 2 - 3 - 4 - 5 USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 57 ANNEX F: ROLES & RESPONSIBILITIES MICHELLE ADAMS MATSON – TEAM LEADER  Will be responsible for team organization  Lead the technical aspects with USAID  Lead the preparation of the work plan and evaluation design  Lead the design and writing of the field data instrument  Identify information gaps and requirements  Conduct interviews and focus groups with key-informants  Data analysis and integrating the FCRs  Deliver the findings, conclusions, and recommendations to USAID  Support report writing ALEKSANDRA MARKOVICH - GENDER SPECIALIST  Contribute to and support preparation of work plan and evaluation design  Contribute to and support design and writing of the field data instrument  Conduct interviews and focus groups with key-informants  Support data analysis and integrating the FCRs  Summarize findings  Deliver the findings, conclusions, and recommendations to USAID  Lead report writing USAID EVALUATION SPECIALIST  Contribute to and support the development of the evaluation design  Support the design of data collection instruments  Conduct interviews and focus groups with key–informants  Support data analysis USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 58 ANNEX G: GENDER STRATEGY ROADMAP CDCS Framework Key Issues to Explore Strategy Goal, Development Hypothesis and Objectives Pursing more effective investments in gender and female empowerment will accelerate progress in development. DO1: Broad-based, inclusive Economic Development IR 1.1 Private Sector Competitiveness increased. Women-owned businesses do not have access to the kind of collateral that they would need to get a bank loan or information on networking and value chain opportunities. However, where USAID discusses what they will do about lack of access to finance, then women are not mentioned. Indicators are weak, and only disaggregate data. Overall, how the gender problem will be addressed is not discussed. There is no discussion of gender issues in terms of the enabling environment. The strategy does not explain how access to finance will be addressed. IR 1.2: Workforce Development and Opportunities for Vulnerable Groups Increased, Especially for the Poor, Women and Youth. This IR may be one of the most important priorities for the crosscutting gender DO. The IR addresses women in the wording of the IR and does a good job of articulating the problem. However, it does not convey an overall strategy around how the activities will add up to larger results (or a strategic approach). This affects the gender strategy when viewed through a gender lens, but may also be an issue for workforce development in general. The indicators are not sex disaggregated. IR 1.3 Management of Energy Resources Improved. No discussion of gender. IR 1.4: Fiscal Stability and Public Financial Management Improved Issues like containment of the public sector wage bill, phasing out of fuel and food subsidies may have implications for women that are worth understanding. For example, 49% of employed women work in public sector jobs. Is there a way to mitigate those issues for women? In addition, USAID provides 50% of the annual operating year budget to the GOJ. Conditions precedent should be reviewed to ensure that gender is incorporated where it makes sense and in alignment with US foreign policy interests. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 59 CDCS Framework Key Issues to Explore DO 2: Democratic Accountability Strengthened IR 2.1: Governance Processes and Institutions Improved. Overall, the strategy focuses on supporting Jordanian-led reforms to broaden and strengthen political participation. SDO4 should apply this strategic approach as well. The strategy around civil society is less clear, there is discussion that the strategy will focus on civic initiatives, political processes, rule of law and community engagement. For gender as for civil society programs more general, the question is how to choose interventions that have broader impact. IR 2.2. Rule of Law Strengthened This section does a good job integrating gender and incorporating studies. One weakness is that indicators are not gender-sensitive, given the importance of gender issues in this section. 2.3 Civil Society Engagement and Effectiveness Increased The key issue for gender in the context of civil society programming is to have an overall strategy for achieving outcomes/results. Are there strategic approaches that form opportunities to support or strengthen other programs? This is a particular opportunity for synergy. Indicators are not gender￾sensitive. IR 2.4 Community Cohesion Enhanced. Strengthen the capacity of local leaders (including women and youth) to resolve disputes and handle grievances. This is a key area to understand the role of women in addressing conflict, resolving disputes, and handling grievances. While some indicators will presumably be disaggregated (number of vulnerable people with increased access to productive assets is critical to view through a gender lens), the gender indicator is not linked to the programming described, and should be reconsidered. In addition, public perception indicators can easily be affected by externalities that have no link to USAID programming. Here, the program should define what the gender strategy is, and then select a set of indicators that reflect that strategy (or set of activities if only at that level). DO3: Essential Services to the Public Improved IR 3.1 Use of Integrated Family Planning and Reproductive Health Services Increased While gender is inherently integrated into FP/RH programs, it may be useful to focus on a) contextual, cultural factors that affect these programs and b) how gender may be integrated in a meaningful way into health systems strengthening. The team has already done the latter, but that is an area where continued focus would be useful from a gender point of view. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 60 CDCS Framework Key Issues to Explore IR 3.2 Learning Outcomes for all Students Improved The issues around boys' education are a key priority from a gender point of view. In addition, it may be important to understand the larger political/social/economic context that affects this programming. Explore the relationship between trends in boys’ education, gender-based violence and increasing conservatism in society. IR 3.3 Accountable, Sustainable Management of Water and Natural Resources Increased Gender is not addressed in this IR. If working at the community level, it may be important to understand issues related to how men and women have access to and use water in different ways (e.g., for the agricultural sector). SDO4: Gender Equality and Female Empowerment Enhanced Overview. The development hypothesis: Female empowerment and gender equality will be enhanced as changes in discriminatory social norms and practices are promoted, advocacy and policy reforms for women’s issue are enhanced, and access to female￾centered services are expanded. SDO4 supports the overarching strategic goal of promoting equality, will lead to improved prosperity, as higher levels of female participation in the labor force will increase economic growth at the macro level and household security at the micro level. As the economic status of women is elevated, there will be positive effects on family health, participation of women in other aspects of household decision￾making and women’s participation and gender balance in community and civic life. Jordan ranks 134 out of 140 on the Gender Gap Index. IR 4.1 Changes in Discriminatory Social Norms and Practices Promoted/Encouraged FY 2016 PPR Achievements: What Was Achieved: 1.USAID held 350 community meetings and training across a number of activities in addition to online campaigns to challenge perceptions, counter gender-based violence, and highlight role models. 2. Training /networking events for women in the energy / engineering sectors, USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 61 CDCS Framework Key Issues to Explore IR 4.2 Advocacy and Policy Reforms Enhanced FY 2016 PPR Achievements: What Was Achieved? 1. Engaged parliamentary and CSOs in training and advocacy to encourage the use of gender-lens in legislation, policy, and strategy formation. 2. USAID supported drafting of 45 policies and laws to promote gender equality (e.g., businesses that will allow women to move into the formal sector.) 3. Revisions to the Labor code will support alternative work arrangements for women. 4. Worked on gender￾based policies, related to Jordan's Equal Futures. IR 4.3 Access to Women- and Girl Centered Services Expanded FY2016 PPR Achievements: USAID used female community health workers to reach out to Syrian households and discuss family planning with extended family members. Through this outreach, USAID reached almost 220,000 women. It is important to note that the CDCS envisioned that this would be achieved through the other DO’s (e.g. access to finance for women will be done through DO1, improved family planning services will be done under DO3, etc. It will be important to monitor and continue this reporting in the future. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 62 ANNEX H: THE GENDER MANAGEMENT CONTINUUM USAID/Jordan is in the process of conducting an evaluation to review the effectiveness of the program strategy and the management approach that supports SDO4 “Gender Equality and Female Empowerment Enhanced.” As a part of that evaluation, the team developed the gender management continuum in order to provide context in terms of what has been accomplished and a potential vision around a set of objectives for the future. In addition to the model outlined below, we have summarized the rationale behind the model, including how the criteria were derived on page 2. Stage 3- Advanced (High Commitment and Adaptive Processes) 1. Relevant tools and resources are readily available and easily accessible to USAID and IP staff. 2. Staff identify and address gender issues early in strategy and program design; integrate analysis and other considerations at various levels of the program cycle; and incorporate gender into solicitations and project/activity management. 3. Staff, at all levels, are highly committed to addressing gender gaps and collaborate effectively to achieve development results. Stage 1- Founding (Basic Requirements Met) 1. Gender Point of Contact is designated. 2. Gender analysis is completed for the CDCS and for project designs. 3. Staff receive gender training and are learning about how gender can strengthen programming. 4. Key gender issues are reported in the Operational Plan. Stage 2- Expanding (Deepening Gender Integration) 1. Gender advisors at the mission have deep expertise or alternatively, the gender POC and/or team have access to needed gender skills. 2. Gender analysis is carefully processed and incorporated into strategic planning and project design. 3. Staff are generally aware of the importance of gender and can explain priorities. 4. Staff can articulate a clear strategy and priorities around gender. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 63 The Gender Management Continuum Rationale Behind the Criteria Stage 1 Rationale/ Explanation Stage 2 Rationale/Explanation Stage 3 Rationale/Explanation How Were the Criteria Derived? The five criteria reflect the requirements outlined in USAID’s Gender Equality and Female Empowerment Policy. How Were the Criteria Derived? These criteria represent the key issues that emerged in KIIs as a part of USAID/Jordan’s evaluation of SDO4. How Were the Criteria Derived? Stage 3 builds on the criteria outlined in stage 2 to create a vision around a more advanced level of achievement. Access to Gender Resources/Expertise 1. Gender Point of Contact is designated. This is a policy requirement. 1. Gender advisors at the mission have deep expertise or alternatively, the gender POC and/or team have access to needed gender skills. Appropriate gender expertise is available to Mission staff. Expertise can be available in a variety of ways, either through USAID staff (at the Mission or in USAID/W) or through other mechanisms. 1. Relevant tools and resources are readily available and easily accessible to USAID and IP staff. All stage 2 requirements are met and, in addition, tools and resources are readily available and easily accessible to all USAID and IP staff. These tools and resources may be developed and/or tailored for the Mission’s context and programming requirements. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 64 The Gender Management Continuum Rationale Behind the Criteria Stage 1 Rationale/ Explanation Stage 2 Rationale/Explanation Stage 3 Rationale/Explanation Gender Analysis 2. Gender analysis is completed for the CDCS and for project designs. This is a policy requirement. 2. Gender analysis is carefully processed and incorporated into strategic planning and project design. While gender analysis is done in stage 1 (meeting the basic requirement), attaining the next level means that analysis is considered in a thoughtful, intentional, and collaborative manner and integrated into strategy or project design. 2. Staff identify and address gender issues early in strategy and program design. In addition, analysis is integrated at various levels of the program cycle; and incorporated systematically into solicitations and project/activity management. USAID studies indicate that gender integration is strongest in earlier stages of the program cycle (for example, during preparation of the CDCS) and weakens as Missions move toward the preparation of solicitations. This criterion not only reflects early identification of gender issues, but also integrates gender at every stage of the program cycle, including solicitations and in project/activity management. Staff Skills, Awareness & Commitment 3. Staff receive gender training and are learning about how gender can strengthen programming. This is a policy requirement. 3. Staff are generally aware of the importance of gender. For USAID/Jordan, a clear majority of staff interviewed were aware of the importance of gender and noted that having a stand-alone SDO focused on gender was one factor that contributed to this awareness. 3. Staff, at all levels, are highly committed to addressing gender gaps and collaborate effectively to achieve development results. In this stage, staff move from being generally aware to being highly committed to addressing gender gaps as a way to achieve development impact. USAID.GOV SPECIAL DEVELOPMENT OBJECTIVE 4: PERFORMANCE EVALUATION | 65 The Gender Management Continuum Rationale Behind the Criteria Stage 1 Rationale/ Explanation Stage 2 Rationale/Explanation Stage 3 Rationale/Explanation Strategy and Priorities 4. Key gender issues are reported in the Operational Plan. This is a policy requirement. 4. Staff can articulate a clear strategy and priorities around gender. For USAID/Jordan, technical staff were generally able to identify specific gender issues within their program, but the overarching strategy and priorities across programming were less clear. 4. Strategies and priorities are reexamined and updated on a regular basis as needed. In addition, there is process to adapt as needed (adaptive management). Management Structure 5. A Mission Order on gender is drafted. This is a policy requirement. 4. An operational/ management structure is clearly articulated, understood and implemented throughout the Mission. Some Missions may have a stand-alone DO, while others address gender as a crosscutting issue. While either approach is acceptable, it is critical for roles, responsibilities, and management structures to be clear at various levels both internally for those who work on gender and other technical staff across the Mission. 5. The management structure is effective and adapts to change as needed. The management structure is not only clear, but is capable of adapting to change as circumstances require. This may include periodic reviews and adjustments to ensure that gender is addressed in an optimal way. U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 2000