DISCLAIMER: This report is made possible by the support of the American people through the United States Agency for International Development (USAID). The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ABT ASSOCIATES INC 2021 MID-TERM EVALUATION OF THE USAID/NEPAL SAJHEDARI-SUPPORT TO FEDERALISM (STF) ACTIVITY— FINAL REPORT December 2021 MID-TERM EVALUATION OF USAID/NEPAL SAJHEDARI SUPPORT TO FEDERALISM (STF) ACTIVITY Mid-Term Evaluation Report December 2021 Contract Number: AID-486-I-14-0001 Task Order: 72048619F00001 This publication was prepared independently by Social Impact, Inc. at the request of the United States Agency for International Development. Evaluation Team Nahakul K.C., Team Leader Balkrishna Silwal, GESI Specialist Upama Achyara, Field Researcher Bandana Sinha, Field Researcher Katherine Vittum, Subject Matter Advisor i | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV ABSTRACT Sajhedari-Support to Federalism (STF), a six-year United States Agency for International Development (USAID)/Nepal funded Activity, implemented by Abt Associates and its consortium, aims to empower local governments, civil society, and the Nepali people in ten municipalities within two provinces to institutionalize effective, accountable, transparent, responsive, and inclusive local governance systems and practices. This mid-term performance evaluation is meant to inform STF’s ongoing implementation as well as similar programming. The evaluation’s qualitative methodology included document review, key informant interviews, and focus group discussions to identify findings addressing USAID/Nepal’s evaluation questions. STF’s primarily output-level progress across all three objectives and gender equality and social inclusion (GESI) integration indicates more is required to ensure higher-level outcomes are achieved. As federalism was new when STF started, STF focused its first two years on capacity building and basic outputs. Having implemented several key lessons, STF steadily improved synergies with the Nepali government and similar donor-funded activities. Despite being an important resource for financial management, GESI, and policy development, STF could engage more within its consortium and possibly at the provincial level. Lack of clear consensus about STF’s design suggests maintaining the current model is best, while simultaneously working to extend STF’s reach by adopting the proposed model municipality approach by ensuring that successful policies, processes, tools, and approaches are shared across Nepal’s municipalities. Qualitative stakeholder feedback indicates that all groups appreciate and value STF’s many efforts and indicate STF is providing value in terms of efficiency, effectiveness, economy, and equity. The report provides nine recommendations for STF and USAID/Nepal’s consideration. USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | ii TABLE OF CONTENTS LIST OF TABLES III LIST OF FIGURES III ACRONYMS IV EXECUTIVE SUMMARY V OVERVIEW V EVALUATION PURPOSE AND METHODS V RECOMMENDATIONS IX EVALUATION BACKGROUND AND PURPOSE 1 COUNTRY CONTEXT 1 EVALUATION QUESTIONS 4 EVALUATION METHODOLOGY AND LIMITATIONS 5 DATA COLLECTION METHODS 5 SAMPLING 7 DATA ANALYSIS 7 EVALUATION LIMITATIONS AND BIASES 8 FINDINGS AND CONCLUSIONS 8 EQ 1 8 EQ 2 13 EQ 3 17 EQ 4 22 EQ 5 25 RECOMMENDATIONS 28 ANNEX I: EVALUATION DESIGN MATRIX 30 ANNEX II: DATA COLLECTION TOOLS 33 TOOL 1- INTERVIEWS WITH USAID/NEPAL 33 TOOL 2- INTERVIEWS WITH STF STAFF 39 TOOL 3- INTERVIEWS WITH MUNICIPALITIES AND LOCAL GOVERNMENT/ MOFAGA, PLGSP, MINISTRY OF FINANCE, MOWCSW, NNFRC, AND PROVINCIAL GOVERNMENTS/ NATIONAL ASSOCIATIONS (MUAN, NARMIN, ASSOCIATION OF DISTRICT COORDINATION COMMITTEES OF NEPAL). 43 TOOL 4- INTERVIEWS WITH OTHER RELEVANT DONORS & DONOR FUNDED PROJECTS 49 TOOL 5- FGDS QUESTIONS FOR SEPARATE GROUPS OF WOMEN, MEN, AND MIXED GROUPS OF MARGINALIZED PEOPLE 53 ANNEX III: DOCUMENTS REVIEWED 59 ANNEX IV: ILLUSTRATIVE LIST OF RESPONDENTS 60 ANNEX V: TEAM MEMBERS 62 ANNEX VI: CONFLICT OF INTERESTS DISCLOSURES 64 ANNEX VII: SCOPE OF WORK 70 iii | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV LIST OF TABLES Table 1: KII/SGIs Achieved....................................................................................................................................... 6 Table 2: FGDs Achieved ........................................................................................................................................... 6 Table 3: Sampled Municipality Characteristics..................................................................................................... 7 LIST OF FIGURES Figure 1: Objective 1 and Sub-Results................................................................................................................... 3 Figure 2: Objective 2 and Sub-Results................................................................................................................... 3 Figure 3: Objective 3 and Sub-Results................................................................................................................... 4 USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | iv ACRONYMS CAO Chief Administrative Officer CDSG Capacity Development Support to Governance CECI Centre for International Studies and Cooperation COP Chief of Party COVID-19 Coronavirus Disease 2019 CS MAP Civil Society: Mutual Accountability Project CSO Civil Society Organization DCOP Deputy Chief of Party D/GBV Domestic/Gender Based Violence DRG Democracy, Human Rights and Governance ET Evaluation Team EQ Evaluation Question FGD Focus Group Discussion GESI Gender Equality and Social Inclusion GIZ Deutsche Gesellschaft für Internationale Zusammenarbeit GON Government of Nepal GRB Gender Responsive Budget GRM Grievance Redress Mechanism IP Implementing Partner IT Information Technology KII Key Informant Interview LISA Local Government Institutional Capacity Self-Assessments MoFAGA Ministry of Federal Affairs and General Administration MUAN Municipality Association of Nepal NARMIN National Association of Rural Municipalities in Nepal PCGG Provincial Center for Good Governance PFM Public Financial Management PLGSP Provincial Local Government Support Program PSC Project Steering Committee RIAP Revenue Improvement Action Plan SI Social Impact SGI Small Group Interview SOW Scope of Work STF Sajhedari-Support to Federalism SUTRA Subnational Treasury Regulatory Application TAF The Asia Foundation TDO Tole Development Organization TOR Terms of Reference UNDP United Nations Development Program USAID United States Agency for International Development USG U.S. Government VFM Value for Money v | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV EXECUTIVE SUMMARY OVERVIEW Sajhedari-Support to Federalism (STF), a six-year United States Agency for International Development (USAID)/Nepal funded Activity, is implemented by Abt. Associates and its consortium (Urban Institute, TEAM Consult Pvt Ltd., and the Centre for International Studies and Cooperation Nepal). The Activity’s base period, May 13, 2019, to May 12, 2022, has an awarded value of $7,981,738, while the option period extending the contract through May 12, 2025 provides an additional awarded value of $7,076,930. STF aims to empower local governments, civil society, and the Nepali people in ten municipalities within Lumbini and Sudurpaschim provinces to institutionalize effective, accountable, transparent, responsive, and inclusive local governance systems and practices. EVALUATION PURPOSE AND METHODS The primary purpose of this mid-term performance evaluation is to inform USAID/Nepal about ongoing Activity implementation and the feasibility of designing and delivering similar programs. The evaluation’s primary audiences include USAID/Nepal and Abt. Associates. This evaluation used a qualitative approach to answering the evaluation questions. The evaluation team (ET) conducted a desk review; 30 key informant interviews (KIIs) and 13 small group interviews (SGIs) with a total of 85 individuals (19 women and 66 men) from key stakeholder groups including USAID, STF and its consortium, Government of Nepal (GON) officials at all three levels (local, provincial, and national), other Donor/Implementing Partners (IPs), and direct beneficiary civil society organizations (CSOs); and ten FGDs with 82 participants (46 women and 36 men) from both direct and indirect beneficiary groups. The ET conducted data collection in Kathmandu, one municipality in Lumbini Province (Tulsipur Sub-Metropolitan), and four municipalities in Sudurpaschim Province (Dhangadhi Sub￾Metropolitan, Bheemdatt Municipality, Dipayal Silgadhi Municipality, and Bitthadchir Rural Municipality). EVALUATION QUESTION 1: HOW HAS STF MADE PROGRESS ON ITS THREE OBJECTIVES? FINDINGS Objective 1: The GON creates and implements sound policies and legislation that are informed by evidence, and empower, finance, and regulate provincial and municipal governments. Qualitative data from KII/SGIs and FGDs confirmed monitoring data trends around STF achieving research and policy-related outputs and shed light on successes related to building stakeholder capacity for policy formulation. However, several respondents noted that even though studies and policies have been developed they have not always been particularly useful or operationalized to citizens’ ultimate benefit. These respondents also highlighted that the policy focus has predominantly been on the local and national levels rather than on the provincial level. Objective 2: The targeted province and selected municipal governments therein are more responsive, inclusive, accountable, transparent, and effective in decision making and delivering priority services to fulfilling core legal responsibilities. Qualitative data reinforced the monitoring data in terms of increasing government capacity to conduct better financial management, social mobilization, and deliver core services. Respondents commented that though local government capacity may have been designed to facilitate marginalized groups’ access to decision-making processes, in reality municipal planning and decision-making processes do not adequately fulfill their gender equality and social inclusion (GESI) USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | vi mandates and that it will take both time and sustained efforts by key stakeholders to effectively address this need. Objective 3: Citizens and civil society organizations in targeted geographic areas, including the historically marginalized, actively participate in local decision-making and oversight. Despite positive indicator performance, qualitative data collected presented a more nuanced picture of achievement under this objective. Respondents reported that STF trained CSOs and citizens to engage in local decision making, assisted local governments to conduct public hearings as a platform for citizens to raise their concerns, facilitated local residents’ inclusion in participatory planning processes, as well as other interventions. Though participation mechanisms have been established, respondents felt that civil society engagement has not been as robust as anticipated, and that the established participation mechanisms need to be better mobilized. Gender and Social Inclusion. The desk review and KII/SGIs confirmed positive monitoring data trends around GESI integration. In Year 1, STF developed a formal GESI strategy and resulting GESI assessment to identify existing GESI efforts and gaps at all governmental levels to integrate GESI across the Activity. However, data suggested the interventions have not yet resulted in higher level GESI outcomes materializing. Effectiveness Factors. To explain STF’s overall slower startup and progress towards its objectives in Year 1 and eventual Year 2 achievements, respondents reported the following factors hindering progress: design delays; staff turnover; Year 1 priority focus on capacity building, policy support, and local stakeholder engagement, all of which take time to do well; and the Coronavirus Disease 2019 (COVID-19) pandemic and political instability. STF also benefited from several critical enabling factors: a focused approach within Nepal’s new federalism system, integrated interventions, and IT support. CONCLUSIONS STF made progress on all three objectives and its cross-cutting GESI integration goal, though mainly at the output level. More is required to ensure higher-level outcomes of good, inclusive governance and citizen participation are achieved. Delays in realizing results over time were in part due to internal and external factors that impeded timely implementation. Simultaneously, it may not have been realistic to expect higher level outcomes within only a few years. EVALUATION QUESTION 2: HOW EFFICIENT HAS STF BEEN IN ACHIEVING RESULTS CONSIDERING THE LEVEL OF FUNDING/EXPENDITURE TO DATE? FINDINGS KII/SGI respondents reported that the Activity is implementing several strategies that demonstrate the four E’s (efficiency, effectiveness, economy, and equity) associated with value for money (VFM). STF consortium and GON respondents noted that STF has a cost sharing approach with local government partners and CSOs to 1) improve economy by reducing the intervention implementation costs to USAID, 2) improve efficiency through alignment with GON frameworks, and 3) improve effectiveness by contributing to local ownership. STF is avoiding duplication of efforts by coordinating with other Donor/IPs, allowing activities to cover more ground and facilitating wider coverage in the GON’s shifts toward federalism, which is contributing to efficiency and effectiveness. Respondents across most stakeholder groups appreciated STF’s efficiency and effectiveness regarding its IT support, especially in light of COVID-19. STF consortium and GON respondents noted STF’s value-add through efficiencies and effectiveness related to research and policy support facilitating the GON’s operationalization of federalism. Several KII/SGI respondents noted that STF’s embedded Municipal Coordinators allow the Activity to achieve both efficiencies and effectiveness through vii | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV increased responsiveness to local government partners. GON KII/SGIs and beneficiary FGDs suggested that STF is contributing to increased equity by working with local government partners to develop and implement policies enhancing historically marginalized groups’ participation, though more still needs to be done. Respondents provided specific suggestions for strengthening the Activity’s VFM: 1) Working better as a consortium to enhance all four E’s; 2) Continuing cost sharing with the GON; 3) Continuing collaboration with other Donor/IPs to ensure programming complementarity; 4) Training local government officials about STF supported policy development to ensure their implementation; 5) Selecting less well supported municipalities that have less donor involvement; 6) Streamlining operational processes (e.g., how technical experts are sourced); 7) Considering better monitoring strategies to measure good governance outcomes; and 8) Continuing to apply the Local Government Institutional Capacity Self-Assessments (LISA) tool to align vision and strategy with government needs, allowing STF to offer a better value proposition. CONCLUSIONS VFM has not been a STF focus to date. Nonetheless, stakeholder feedback indicates that STF is providing value along all four of the proposed framework’s dimensions: efficiency, effectiveness, economy, and equity. However, the extent to which this is occurring is difficult to determine in the absence of quantifiable data and an established VFM framework. EVALUATION QUESTION 3: HOW HAS STF ENGAGED WITH RELEVANT STAKEHOLDERS, INCLUDING GON ENTITIES (E.G., MOFAGA, PROVINCIAL GOVERNMENT, LOCAL GOVERNMENTS), OTHER DONOR PROJECTS AND OTHER STAKEHOLDERS (E.G., PLGSP, THE DEUTSCHE GESELLSCHAFT FÜR INTERNATIONALE ZUSAMMENARBEIT [GIZ] PROJECT, [USAID] DEMOCRACY, HUMAN RIGHTS, AND GOVERNANCE [DRG] IMPLEMENTING PARTNERS) IN THE FIELD SUPPORTING FEDERALISM? FINDINGS SGI/KII data show that though coordination and synergy between STF and other Donor/IPs was quite challenging at first, it improved over time. One means by which the Activity maintains synergy with other Donor/IPs and the GON is by participating in various coordination and partnership mechanisms including the Project Steering Committee (PSC), Provincial Center for Good Governance (PCGG), National Association of Rural Municipalities in Nepal (NARMIN), and Provincial Local Government Support Program (PLGSP). Due to improved synergy, GON respondents reported feeling greater ownership for Activity interventions, which makes them want to share STF interventions. Stakeholders also noted the quality of STF’s support stemming from its proactive coordination and collaboration with the GON and other Donor/IPs, particularly in the areas of policy development, financial management, GESI, and software/digital support. The GON also appreciated STF’s flexible approach relative to other donor-funded projects, in terms of the areas it supported such as policy making, planning, and technical assistance, which led to a more integrated approach. Another value-add STF contributed is working in Sudurpaschim where few other donors are working. Despite STF’s improved collaboration with the GON and other Donor/IPs, there remain opportunities for even greater synergies, including better collaboration within the STF consortium and providing provincial-level support. Despite noted improvement since the new Chief of Party’s (COP’s) arrival, consortium members noted on-going challenges with communication and coordination among consortium partners. Similarly, USAID would like to see greater synergy among USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | viii its IPs. Some respondents felt STF should engage more with the provincial-level government, though some also noted that other Donor/IPs are already working in this space. CONCLUSIONS STF has steadily improved synergies with the GON and similar donor-funded activities. In doing so, the Activity has become an important resource for other activities in several critical areas including for financial management, GESI, and policy development broadly. Moreover, its Y3 Work Plan also indicates its ongoing commitment to continue these efforts. However, there are two areas where STF could engage more fully: 1) within its own consortium to ensure clear coordination and collaboration among its partners and across objectives; 2) potentially at the provincial level. However, this would need to be coordinated with other Donor/IPs who are already working at the provincial level. EVALUATION QUESTION 4: WHAT CAN STF LEARN FROM SIMILAR PROJECTS CURRENTLY BEING IMPLEMENTED (E.G., PLGSP, GIZ) IN THE FIELD SUPPORTING FEDERALISM IN THE COUNTRY AND HOW CAN STF INCORPORATE THOSE LEARNINGS IN ITS IMPLEMENTATION? FINDINGS KII/SGI data indicate that STF is already working with and learning from other federalism projects. Learning from other donors has reinforced the importance of government ownership and working closely with GON partners. Although STF has already provided municipalities with significant digital support to facilitate their adaptation to COVID-19, a national GON respondent noted the situation necessitates even more support. Several respondents also commented on STF’s tendency to publish reports in English, which they noted have only limited usefulness. KII/SGI data indicate that STF has reflected upon and learned from its own implementation challenges and adapted practices. Respondents noted that the Activity was reactive at first, but eventually learned to be more proactive and focused in its efforts. Another critical lesson for STF was that interventions need to incorporate applied learning techniques. Respondents differed in their opinions concerning if and how STF should broaden its scope, by working in more than ten municipalities or at different government levels akin to other federalism projects. Some GON respondents were content with the ten-municipality model and the amount of support currently provided while others questioned the efficacy of the ten-municipality model. Some GON respondents commented that STF selected municipalities were relatively more accessible in which to work. Other GON respondents appreciated STF’s focus on local governments. Others felt STF should invest more at the provincial level. Though USAID noted that STF’s scalability is not yet known, GON respondents suggested the possibility of expansion by using the current ten municipalities as models for other municipalities. CONCLUSIONS STF has already implemented several key lessons from other Donor/IPs, the GON, and its own experience. Key lessons included the importance of collaborating with other donors and USAID IPs, working closely with the GON, having a proactive and focused approach, and incorporating applied learning techniques in its interventions. Additional lessons STF can incorporate include focusing even more on providing digital service delivery support such as virtual training opportunities for local government and civic engagement as well as writing reports in Nepali rather than English. Given the lack of a clear consensus about the best model for STF, it may be best to maintain the current ten￾municipality model. Simultaneously, there is the possibility for STF to work with the GON and other Donor/IPs to find ways to extend the proposed model municipality approach and ensure that successful policies, processes, tools, and approaches are shared among all Nepal’s municipalities. ix | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV EVALUATION QUESTION 5: IN WHAT WAYS HAS STF ALREADY ADAPTED TO THE NEW COVID-19 CONTEXT? FINDINGS COVID-19 has had a tremendous impact on STF and its progress. Data indicated that Objectives 2 and 3 were especially impacted. STF’s Year 3 Workplan and KII/SGI respondents described how the pandemic impeded GON’s ability to create a more inclusive and responsive local government, as well as how COVID-19 mobility restrictions impeded citizens’ ability to engage with government. Respondents reported fewer insights regarding how COVID-19 impacted Objective 1, but some noted the pandemic hindered intervention implementation at the municipal level. KII/SGI respondents noted STF established strategies to mitigate COVID-19 imposed challenges. KII/SGI respondents and some FGD participants noted STF’s strides in informing the public about the severity of COVID-19 through a help desk and other interventions, and effectively used technology at the municipal level to mitigate challenges regarding a lack of in-person contact. One of STF’s theory of change assumptions is that “STF achievements will be possible if no natural disasters or health-related pandemics occur during the life of the Activity.” Due to COVID-19, this assumption no longer holds, reflecting the need to shift implementation modalities. Additionally, another assumption that has been affected is “All tiers of government (federal, provincial, and local) are receptive and collaborate with STF in the targeted regions, including dedicating budget and resources.” This assumption did not hold throughout the activity as GON’s priorities and resource allocation shifted towards handling COVID-19. CONCLUSIONS COVID-19 created obstacles to citizen engagement and negatively impacted progress towards its Objectives, especially Objectives 2 and 3. Nonetheless, STF adapted to the pandemic in two main ways: 1) using remote work strategies that allowed for continued implementation, and 2) providing communities with the necessary information to deal with COVID-19. While a change in the overall theory of change may not be necessary, assumptions within it should be re-evaluated. Specifically, municipalities need more support to ensure better service delivery given the likelihood that COVID￾19 will continue to influence the Activity. STF is currently reevaluating this assumption based on the understanding that COVID-19 challenges and restrictions will continue to affect the Activity. RECOMMENDATIONS In the future USAID/Nepal should consider the following: 1. USAID/Nepal should exercise the STF option period with possible increased focus at the provincial level, in coordination with other Donor/IPs already working at that level and possibly developing the current ten municipalities as models. 2. USAID should maintain STF’s current geographic scope, while working with the Activity and relevant GON officials to develop processes for sharing proven policies, tools, etc. with non￾targeted municipalities. 3. If USAID/Nepal is interested in quantitatively capturing any Activity’s VFM, including STF, consider requiring the financial reporting necessary to facilitate such an analysis in the Activity’s contract. As STF moves forward to complete its base period and into the option period, should USAID exercise it, the Activity should consider the following: 4. Continue efforts already started to improve implementation, especially: better integrate work across STF objectives, enhance synergies with other similar efforts and GON initiatives, and focus on policy operationalization rather than just policy development. 5. Develop and apply technology more, specifically related to e-Governance, which can ensure transparency and accountability among citizens at the municipal level. USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | x 6. Work more closely with municipalities to develop systems that can better mainstream marginalized communities into planning processes and project implementation. 7. Ensure that all key publications, including policy documents, are translated into Nepali so that GON officials and stakeholders at all levels have access to and can use them. 8. Continue to focus on capacity building around policy implementation and support to new systems/processes operationalization. 9. STF should closely collaborate with other key actors, such as GIZ, who are active at the provincial level to determine where and how STF can add value. 1 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV EVALUATION BACKGROUND AND PURPOSE Sajhedari-Support to Federalism (STF) is a six-year Activity funded by United States Agency for International Development (USAID)/Nepal. It is implemented by Abt Associates and its consortium (Urban Institute, TEAM Consult Pvt Ltd., and the Centre for International Studies and Cooperation [CECI] Nepal). The Activity has a base period that started May 13, 2019, and ends May 12, 2022, with an awarded value of $7,981,738, and an option period that extends the contract through May 12, 2025, with an additional awarded value of $7,076,930. STF aims to empower local governments, civil society, and the Nepali people in ten municipalities within two provinces (Lumbini and Sudurpaschim) to institutionalize effective, accountable, transparent, responsive, and inclusive local governance systems and practices.1 STF contributes to capitalizing on USAID/Nepal’s Sajhedari Bikaas (Partnership for Development) Activity through “providing technical assistance to the Government of Nepal (GON) in transitioning from a unitary state to a federation of provinces with more effective, accountable, responsive, and inclusive local governance.”2 The primary purpose of this mid-term performance evaluation is to inform USAID/Nepal about ongoing Activity implementation and the feasibility of designing and delivering similar programs. Primary audiences for the evaluation include USAID/Nepal and Abt Associates. Secondary audiences include the Ministry of Federal Affairs and General Administration (MoFAGA) and other GON officials at the provincial and local level, and particularly relevant municipal officials in Sudurpaschim and Lumbini Provinces. COUNTRY CONTEXT After its ten-year civil war ended in 2006, Nepal transitioned from a monarchy to a republic based on multi-party democracy. This transition began with the adoption of a new constitution in 2015 in which a three-tier governance system was established along with major territorial reforms. These reforms created 753 local government units in 2017 including rural and urban municipalities, as well as sub-metropolitan and metropolitan cities.3 In the same year, Nepal held elections at all levels establishing seven provinces, which were intended to address identity issues related to social inclusion.4 The creation of the new federal structure was meant to make public service delivery more effective and efficient, allow citizens’ access to governance processes, improve elected leaders’ accountability toward citizens, and ensure governance based on a clear vision and division of powers and responsibilities among the GON’s different tiers.5 However, due to complexities surrounding the GON’s limited capacity, lack of clear communication, and poor coordination among the three governance tiers, these goals have not been achieved. In 2020, political turmoil—including Prime Minister K.P. Sharma Oli’s dissolution of parliament, the Supreme Court’s reinstatement of parliament, and weeks of protests—further hindered Nepal’s progress toward federalism as the GON’s focus was diverted from related interventions.6 1 USAID. (2020). Nepal Fact Sheet: Sajhedari Support to Federalism. USAID. 2 STF Contract Base 3 USAID. (2021). Sajhedari-Support to Federalism: Baseline Perception Survey. USAID. 4 USAID. (2021). Sajhedari-Support to Federalism: Baseline Perception Survey. USAID. 5 USAID. (2021). Sajhedari-Support to Federalism: Baseline Perception Survey. USAID. 6 https://www.nytimes.com/2021/02/23/world/asia/nepal-parliament-dissolved.htm 7 Base contract modification. USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 2 Nepalese society has historically discriminated against and excluded people based on gender, caste, ethnicity, geography, disability, and other dimensions.7 This systematic discrimination has continued, largely unchanged, since the ten-year civil war. Marginalized groups face discrimination due to inequitable laws, policies, cultural norms, and practices. Current federal laws and social norms prevent marginalized groups from achieving citizenship, government jobs, and proper representation within the government.8 Although women have had increased opportunities to participate in government since the war’s end, cultural and institutional barriers continue to limit women leaders’ ability to address gender issues and contribute to all aspects of Nepal’s governance.9 Additionally, current cultural norms have silenced the voices of the lesbian, gay, bisexual, transgender, queer, and intersex population, increased violence against individuals with disabilities, and limited the religious freedom of lower castes such as Kami, Sarki, and Halkhor.10 The Coronavirus Disease 2019 (COVID-19) pandemic exacerbated these inequalities, making it difficult for marginalized groups to participate in local government decision-making, gain equal access to assets and resources, and receive equal rights. These challenges have hindered the growth of federalism in Nepal.11 USAID/Nepal created and adapted the STF Activity in response to these governance and federalism issues as well as those related to social inclusion generally, and women’s issues and female leadership specifically. The Activity is meant to assist with the transition from a unitary state to an effective, inclusive democratic republic. ACTIVITY THEORY OF CHANGE STF’s approach to addressing the development problem posed by Nepal’s shift from a monarchy to republic-based governance system is articulated in its theory of change, which states: “IF federalism policy and legislation is improved, local governments are more effective, inclusive, and accountable, and citizens are constructively engaged in local decision making and oversight, THEN governance in Nepal will be more effective and inclusive at all levels (federal, provincial, and municipal), resulting in improved public decision making and access to and quality of public services for a more prosperous and peaceful Nepal.” The Activity’s ability to achieve these results is predicated on the following assumptions: 1. Support for federalism remains a USAID/Nepal priority; 2. All tiers of government (federal, provincial, and local) are receptive and collaborate with STF in the targeted regions, including dedicating budget and resources; 3. No natural disasters (category 3 level) or health-related pandemics occur during the life of the Activity; 4. Civic space remains open at all levels of government; and 5. The political situation in the country remains stable allowing mobility for Activity implementation.12 7 Base contract modification. 8 USAID. (2020). Nepal Gender Equality and Social Inclusion Analysis. USAID. 9 Contract modification. 10 USAID. (2020). Nepal Gender Equality and Social Inclusion Analysis. USAID. 11 USAID. (2020). Nepal Gender Equality and Social Inclusion Analysis. USAID. 12 Performance Evaluation for USAID/Nepal Sajhedari-Support to Federalism Project (STF) Statement of Work (SOW). 3 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV ACTIVITY OBJECTIVES STF has three contributing objectives to work toward its overarching goal to provide support to federalism through technical support and assistance to the Nepalese government. These objectives, associated sub-results, and examples of relevant interventions are described below: Figure 1: Objective 1 and Sub-Results Objective 1 seeks to strengthen institutional capacity for policy development and legislation to regulate provincial and municipal governments. STF initiated interventions and strategies that assist with policy development and legislation as well as create a solid space in which policy processes are more open, participatory, and inclusive.13 Additionally, the Activity established partnerships with different stakeholders such as MoFAGA, the Provincial Local Government Support Programme (PLGSP), and the Provincial Center for Good Governance (PCGG).14 Figure 2: Objective 2 and Sub-Results Objective 2 seeks to build better government capacity at both the provincial and municipal level to improve decision-making, public service delivery, and institutionalized capacity building. To accomplish this goal, STF conducted profiling and planning for municipalities and provinces, ensuring that governments are delivering services effectively and fulfilling their core legal responsibilities. STF completed interventions such as creating and publishing municipal profiles in Bheemdatt, Dhangadhi, and Dangisharan; assisting with drafting policies and procedures; and supporting Judicial Committees 13 USAID. (2020). Sajhedari-Support to Federalism: Year One Annual Performance Report May 2019–September 2020. USAID 14 USAID. (2020). Sajhedari-Support to Federalism: Year One Annual Performance Report May 2019–September 2020. USAID. Objective 1: The GON creates and implements sound policies and legislation that are informed by evidence, and empower, finance, and regulate provincial and municipal governments. Sub-Result 1.1: Policy and legislation are informed by evidence. Sub-Result 1.2: Policy processes are more transparent, participatory, and inclusive. Objective 2: The targeted province and selected municipal governments therein are more responsive, inclusive, accountable, transparent, and effective in decision making and delivering priority services to fulfilling core legal responsibilities. Sub-Result 2.1: Municipal and provincial governments are more responsive, inclusive, accountable, transparent, and effective in decision making. Sub-Result 2.2: Municipal and provincial governments are more responsive, inclusive, accountable, transparent, and effective in delivering priority services to fulfill core legal responsibilities. Sub-Result 2.3: Learning institutions are enhanced and sustainable. USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 4 by providing trainings on conducting hearings. Additionally, STF provided digital tools to local governments to improve implementation and planning.15 Figure 3: Objective 3 and Sub-Results Objective 3 seeks to increase citizen engagement through both government and civil society organizations (CSOs). STF completed various interventions to address this objective such as conducting a CSO mapping, facilitating CSO engagement with local governments, and convening CSO forums in Bitthadchir, Sikhar, Bheemdatt, Punarbas, Gauriganga, and Tulsipur.16 Additionally, STF assisted with awareness raising and capacity-building initiatives for CSOs through diverse good citizen initiatives as well as organizing good governance labs to introduce social accountability tools and to build civil society capacity for monitoring and public oversight.17 GENDER EQUALITY AND SOCIAL INCLUSION In addition to the three objectives described, STF incorporates a cross-cutting theme of gender equality and social inclusion (GESI). STF adopted a GESI mainstreaming and integration approach that targets women, marginalized groups, and youth. To ensure that GESI mainstreaming is achieved, STF adopted multiple strategies such as supporting roll out of GESI policies and plans, helping local governments expand the reach and effectiveness of their GESI initiatives, developing local level support for GESI mainstreaming, and building local government officials’ GESI knowledge and skills.18 Critical achievements in advancing this cross-cutting theme included the 2020 GESI Assessment, which set the stage for tailored GESI initiatives in Activity Year 3, such as (1) conducting local government gender responsive budgeting (GRB) audits and handing these to PCGG/MoFAGA for replication of the GRB e-module in other municipalities and (2) mainstreaming GESI across provincial and municipal planning and service delivery, including COVID-19 responsive efforts.19 EVALUATION QUESTIONS This evaluation addressed the following evaluation questions (EQ), drafted by USAID/Nepal in the initial Scope of Work (SOW) and finalized in collaboration with USAID/Nepal by the evaluation team (ET): 1. How has STF made progress on its three objectives? In answering this question, the ET must assess how progress varies among the three objectives and identify any factors that may be contributing to or hindering progress. 15 USAID. (2020). Sajhedari-Support to Federalism: Year One Annual Performance Report May 2019–September 2020. USAID. 16 STF Year 2 Quarterly Report 3. 17 STF Year 2 Quarterly Report 3. 18 Sajhedari STF Year 2 FY21 Work Plan. 19 Sajhedari Year 3 Work Plan, p. 27–28. Objective 3: Citizens and civil society organizations in targeted geographic areas, including the historically marginalized, actively participate in local decision making and oversight. Sub Result 3.1: Enhanced citizen and civil society engagement in local decision making. Sub-Result 3.2: Improved civil society monitoring and oversight. 5 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV 2. How efficient has STF been in achieving results considering the level of funding/expenditures to data? 3. How has STF engaged with relevant stakeholders, including GON entities (e.g., MoFAGA, provincial government, local governments), other donor projects and other stakeholders (e.g., PLGSP, the Deutsche Gesellschaft Für Internationale Zusammenarbeit [GIZ] project, [USAID] Democracy, Human Rights, and Governance [DRG] implementing partners) in the field supporting federalism? 4. What can STF learn from similar projects currently being implemented (e.g., PLGSP, GIZ) in the field supporting federalism in the country, and how can STF incorporate those learnings in its implementation? 5. In what ways has STF already adapted to the new COVID-19 context? EVALUATION METHODOLOGY AND LIMITATIONS The evaluation covered Activity Years 1, 2, and part of Year 3 (which began in May 2021). To address the EQs, the ET used a qualitative evaluation methodology consisting of a document review, key informant interviews (KIIs), small group interviews (SGIs), and focus group discussions (FGDs). The team adopted a utilization-focused approach, based on the principle that an evaluation should be judged on its usefulness to intended audiences—in this case, to inform USAID/Nepal’s decision about STF’s ongoing implementation. As a result, the ET engaged relevant stakeholders, especially USAID/Nepal and STF consortium members, from the evaluation’s beginning to ensure findings can be utilized to improve performance. The ET collected data between September 16 and November 11, 2021, including a brief pause to account for Nepal’s major festival period for Dashain and Dipawali. Given ongoing COVID-19 safety concerns, most data collection occurred remotely, except for FGDs, which were conducted in￾person with hard-to-reach marginalized populations. Prior to the start of data collection, USAID and Social Impact (SI)’s Institutional Review Board reviewed and approved all data collection instruments, and all ET members engaged in data collection completed a certificate course on essential elements of ethics for engaging with human subjects. The Evaluation Design Matrix can be found in Annex I. All data collection instruments, and informed consent scripts can be found in Annex II. DATA COLLECTION METHODS DOCUMENT REVIEW The ET conducted a comprehensive desk review of key documents to provide background on the processes and decisions that contributed to the design of STF, as well as up-to-date information on implementation progress and results. Document review additionally informed the evaluation design, such as the identification of key stakeholders, selection of data collection methods, and sampling of data collection sites. A list of the major documents reviewed is included in Annex III. KEY INFORMANT INTERVIEWS AND SMALL GROUP INTERVIEWS The ET conducted KIIs and SGIs to collect data relevant to all EQs. Interviews explored themes related to the Activity design and metrics, learning from past and present efforts, the scope and nature of challenges to achieving STF’s objectives, perspectives on the effectiveness of approaches used, organizational characteristics, and project management processes. The team also explored aspects related to marginalized populations and differential impacts on men and women. USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 6 The ET completed 30 KIIs and 13 SGIs with a total of 85 individuals (19 women, 66 men) from key stakeholder groups including USAID; STF and its consortium (STF consortium); GON officials at the local, provincial, and national levels (GON); other Donor/IPs; and CSOs (direct beneficiaries [beneficiaries]). Interviews lasted approximately 45 to 60 minutes each. Two ET members—a lead interviewer and notetaker—conducted each interview remotely using the Microsoft Teams virtual platform, Zoom, or telephone as required. Aside from some KIIs and SGIs at the national level which were conducted in English, the ET completed most interviews in Nepali. The ET obtained respondents’ verbal informed consent at the start of each interview regarding participation in the interview and audio reordering, ensuring discussion content was accurately captured. Table 1 indicates the number of KII/SGIs conducted per respondent group. Table 1: KII/SGIs Achieved KII/SGI Respondent Groups # of Planned KIIs/SGIs # of Achieved KIIs/SGIs # of Individuals USAID 3 3 5 STF Consortium (Abt Associates, Urban Institute, TEAM Consult Pvt. Ltd., CECI Nepal) 9 12 12 GON (MoFAGA, PLGSP, National Association of Rural Municipalities in Nepal [NARMIN], GON Officials [local, provincial, and national]) 22 17 35 Other Donors/IPs (DAI, TAF, GIZ, Tayar, CS MAP) 6 6 9 Direct Beneficiaries (CSOs) 5 5 24 Total KIIs/SGIs 45 43 85 FOCUS GROUP DISCUSSIONS The ET conducted FGDs with Activity recipients (direct beneficiaries) and municipality-level stakeholders from various marginalized groups (indirect beneficiaries) to assess their views on STF and collect data relevant to all EQs. Indirect beneficiary FGDs included stakeholder groups such as youth, socially marginalized individuals (e.g., Dalits), returnee migrants, as well as direct beneficiaries from Tole Development Organizations (TDOs) with which STF worked. The ET completed a total of ten FGDs with 82 participants (46 women, 36 men). A breakdown of FGDs by participant categories is displayed in Table 2. Table 2: FGDs Achieved Beneficiary Status FGD Participant Categories # of Planned FGDs # of Achieved FGDs # of Individuals # of Men # of Women Indirect Beneficiaries Socially Marginalized 4 4 32 17 15 Returnee Migrants 1 1 6 0 6 Direct Beneficiaries Youth Organizations and TDOs 4 4 36 22 14 Unorganized Occupational Groups 1 1 8 7 1 7 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV FGDs included between six and 12 participants and lasted approximately 60 to 90 minutes each. Two ET members—a facilitator and notetaker—conducted each FGD in person. The ET conducted all FGDs in Nepali and obtained FGD participants’ verbal informed consent at the start of each FGD to participate in the interview and have the discussion recorded. The ET conducted four sex￾segregated FGDs, one with solely men and three with solely women to ensure gender-specific issues for participation in policy and planning and their influential capacity were appropriately captured and represented. All other FGDs were mixed. The ET selected participants for each FGD from communities where STF interventions are taking place. The ET obtained this information from STF’s sub-awardee staff in the selected five municipalities. SAMPLING The ET applied a multistage purposive sampling approach to select national-level entities, municipalities, and relevant stakeholders for KII/SGIs and FGDs based on the following criteria: (1) identified as key informants by USAID/Nepal and/or STF; (2) Activity engagement across a range of roles and locations (five municipalities from the two Activity provinces: Sudurpaschim and Lumbini); and (3) inclusion of female interviewees and marginalized communities to the fullest extent possible, given the priority of marginalized communities in STF Objective 3 and the cross-cutting theme of GESI integration. The ET conducted data collection in Kathmandu and five municipalities: one municipality in Lumbini Province (Tulsipur Sub-Metropolitan) and four municipalities in Sudurpaschim Province (Dhangadhi Sub-Metropolitan, Bheemdatt Municipality, Dipayal Silgadhi Municipality, and Bitthadchir Rural Municipality). The municipalities were purposively selected to include a mix of municipality type as well as ecological zone types, including himal, mountain, and terai municipalities from different districts, as well as those classified as border areas (See Table 3 for a description of sampled municipality characteristics). Table 3: Sampled Municipality Characteristics Municipality Districts Province Ecological Zone Tulsipur Sub-Metropolitan Dang Lumbini Terai Dhangadhi Kailali Sudurpaschim Terai/Border Area Bheemdatt Kanchanpur Sudurpaschim Terai/Border Area Bitthadchir Bajhang Sudurpaschim Himal Dipayal -Silgadhi Doti Sudurpaschim Mountain DATA ANALYSIS The team coded all notes from KIIs, SGIs, and FGDs using Dedoose under themes pre-identified in the Evaluation Design Matrix as well as those that emerged through the data collection. The ET’s analysis triangulated data within and across data collection methods, stakeholder groups, as well as by gender and geography to ensure the reliability and validity of findings. The ET constructed an outline of findings, conclusions, and recommendations by EQ to capture findings across all data collected to arrive at recommendations that context-appropriate and actionable. Total 10 10 82 46 36 USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 8 All findings were reviewed through a GESI lens to identify trends affecting particular marginalized populations and their bearing on recommendations to improve inclusiveness within federalism. EVALUATION LIMITATIONS AND BIASES • Generalizability: The ET collected primary data in only five municipalities within the two provinces. Furthermore, primary data was entirely qualitative and was collected from small, purposive samples of respondents and geographic locations. Emphasis was on collecting and analyzing in-depth perspectives grounded in participants’ experiences on the topics evaluated. As such, the design was adequate to meet the purpose of the evaluation, but the conclusions from the evaluation cannot necessarily be generalized to all STF implementation sites. • Potential Biases: There is the possibility of selection bias for non-random samples. Respondent answers may also be subject to recall bias related to past experiences, as well as response/desirability biases whereby respondents may alter their answers due to explicit or implicit expectations. To mitigate these risks, the sample included a variety of participant types to reflect, as accurately as possible, the range of stakeholder views. For FGDs, the selection sites and participants were based on discussions with STF and local CSOs. For mixed male/female FGDs, male participants dominated the discussion in some instances (including leaders of youth and the TDO in Tulsipur) while women were more vocal than men elsewhere (e.g., unorganized occupational groups in Dhangadhi). • Value for Money (VFM) Analysis: For EQ 2, the ET initially proposed to identify how much money has been spent by objective (and to the extent possible by year) against stakeholder perceptions of the relative progress achieved to date for each objective and their recommendations for improving VFM. Unfortunately, that proved impossible because there was no financial data to support such an analysis. As such, the ET, in agreement with USAID/Nepal, focused on strictly qualitative methods to answer this question. Drawing on learning and good practices for VFM within international development, the ET used KIIs and SGIs to explore topics such as: whether STF uses a VFM framework and if it includes stakeholder perceptions of how well resources are being used; when results are expected to be achieved; the quality of results being achieved under each objective; and how well the different Activity objectives fit together. The ET also explored whether STF is experimenting with different ways of delivering results more cost-effectively and recommendations for improving VFM. The team triangulated all the information to draw conclusions about STF’s VFM from a qualitative perspective. • Remote Data Collection: Due to COVID-19 movement restrictions, data collection for KII/SGIs was remote and required respondents to have reliable email, internet, and/or phone access. There is a possibility that remote nature of these communications may have affected the ability and willingness of respondents to be forthcoming in their responses. FINDINGS AND CONCLUSIONS In the following sections, we present findings and conclusions according to each EQ. EQ 1 How has STF made progress on its three objectives? 9 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV FINDINGS OBJECTIVE 1: THE GON CREATES AND IMPLEMENTS SOUND POLICIES AND LEGISLATION THAT ARE INFORMED BY EVIDENCE, AND EMPOWER, FINANCE, AND REGULATE PROVINCIAL AND MUNICIPAL GOVERNMENTS Monitoring data showed that STF advancement toward Objective 1 was sluggish in Year 1 but accelerated in Year 2, with slightly more progress being made toward ensuring policies and research are evidence-based. Of STF’s four indicators for Objective 1, in Year 1 the Activity was only able to meet its targets for one indicator regarding the number of policies in various stages of development as a result of U.S. Government (USG) assistance, aligned with Sub-Result 1.2, Policy processes are more open, participatory, and inclusive. The Activity got close to meeting the target for one indicator measuring Sub-Result 1.1, Policy and legislation are informed by evidence, and its second indicator under Sub-Result 1.2, Policy and legislation informed by evidence. In Year 2, the Activity exceeded its target for two of its four indicators (one for each sub-result) and made additional progress toward its second indicator under Sub-Result 1.1. The only indicator that STF did not make progress toward both years was the number of legal instruments drafted, proposed, or adopted with USG assistance designed to promote gender equality or nondiscrimination against women or girls at the federal and provincial level (aligned with Sub-Result 1.2). Qualitative data from KII/SGIs and FGDs confirmed the monitoring data trends around STF achieving research and policy-related outputs and shed light on successes related to building stakeholder capacity for policy formulation. GON, USAID, STF, and other Donor/IP respondents cited numerous examples of STF completing studies and other research, as well as successfully getting municipalities to draft and put into place policies or legislation, including GESI policies to enhance the participation of historically marginalized groups. An STF consortium respondent asserted that in Years 1 and 2, and now into Year 3, the Activity has recognized that it cannot meet all policy needs and has tried to focus its support on policies that are aligned with gaps identified through Local Government Institutional Capacity Self-Assessments (LISAs). In terms of capacity building, STF consortium respondents mentioned concrete examples such as STF assisting with the development of a manual for local government on local government assembly operations and the local-level law-making process, initiating sectoral committees’ terms of reference (TORs) to strengthen the planning mechanism of local governments, and providing training to municipalities on the seven-step planning process, which eventually helped local governments create policies and legislation. As a GON respondent put it, “STF’s program has helped to generate more skilled [human power], and different policies [have] been established with their help.” This respondent noted strategic planning, revenue improvement, and participatory planning skills as specific examples of increased human power. However, several respondents noted that even though studies and policies have been developed they have not always been particularly useful or operationalized such that citizens are ultimately benefiting. Respondents pointed out that the policy focus had been more on the local and national levels rather than the provincial level. Respondents, particularly from USAID and FGD participants, expressed the need to move beyond research and policy development and ensure that these outputs are being effectively implemented in an inclusive manner. According to a youth who participated in an FGD, “policies are sound and well-formed; they are not well-implemented. The ward office is not adequately oriented toward inclusive development; youth are ignored; and no projects for capacity building or youth leadership exist.” OBJECTIVE 2: THE TARGETED PROVINCE AND SELECTED MUNICIPAL GOVERNMENTS THEREIN ARE MORE RESPONSIVE, INCLUSIVE, ACCOUNTABLE, TRANSPARENT, AND USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 10 EFFECTIVE IN DECISION MAKING AND DELIVERING PRIORITY SERVICES TO FULFILLING CORE LEGAL RESPONSIBILITIES Monitoring data showed that STF advancement toward Objective 2 was also slow in Year 1, but picked up in Year 2, with the strongest evidence pointing toward progress on two of the three sub￾results. Of STF’s 13 indicators for Objective 2 in Year 1,20 the Activity did not meet any targets and only made progress toward two output-level indicators: number of USG supported events, trainings, or activities designed to build support for peace or reconciliation among key actors to the conflict (meant to measure Sub-Result 2.1, Municipal and provincial governments are more responsive, inclusive, accountable, transparent, and effective in decision-making) and number of government officials receiving USG-supported trainings (meant to measure Sub-Result 2.3, Learning institutions are enhanced and sustainable). The Activity fared better in Year 2 and succeeded in meeting its targets for ten out of the 13 indicators. Though many of these indicators were outputs, some measured outcome-level changes as demonstrated by improved local government capacity to deliver services, local governments having increased their own-source revenue, and a high percentage of surveyed citizens (86 percent compared to a 76 percent baseline) reporting local government services satisfactorily responded to their needs. Qualitative data concurred with the monitoring data in terms of increasing government capacity to conduct better financial management, social mobilization, and deliver core services. Financial management support was especially appreciated and resulted in increased own-source revenue generation for some municipalities (see EQs 2 and 3 for more detail). For example, a GON respondent commented that due to STF’s technical assistance in introducing the Subnational Treasury Regulatory Application (SUTRA) and building staff capacity to use it, “[they] experienced substantial improvements in revenue generation.” GON respondents also provided examples of how municipalities continued to provide services during the COVID-19 pandemic, learned their legal responsibilities through STF orientation on Local Government Act 2074, developed revenue improvement plans, and overall became, as one GON respondent stated, “more confident in [their] ability to handle situations in the municipality.” Corroborating these successes, a FGD participant recounted how local judicial committees have been able to handle conflicts without the courts. Furthermore, GON respondents and CSO respondents highlighted increased involvement with CSOs, TDOs, and Ward Committees in government planning processes to promote collaboration and inclusive governance. Respondents commented that though local government capacity may have been built to facilitate marginalized groups’ access to decision-making processes, in reality municipal planning and decision￾making processes do not adequately fulfill their GESI mandates. A STF consortium respondent noted how the concept of GESI is still misunderstood among many government officials, which hinders the GESI integration process. FGD participants (youth and those from socially marginalized groups) recounted that “we have not found notable changes in these three years. Rather, the local governments/ward offices [are] not listening to the voices of women and marginalized in practice.” 20 Six out of 13 indicators aligned with Objective 2 results were not applicable for Year 1. Targets and actuals were intentionally designed to only be collected in Year 2 onward. “The issue of transparency and accountability is still there. It has not been there to the extent it is supposed to be—women's and consumers groups are here but there is no involvement or participation as it should have been done.” –CSO Respondent 11 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV One government official similarly conceded that there is a gap in the government’s capacity to ensure peoples’ voices are being heard. OBJECTIVE 3: CITIZENS AND CSOS IN TARGETED GEOGRAPHIC AREAS, INCLUDING THE HISTORICALLY MARGINALIZED, ACTIVELY PARTICIPATE IN LOCAL DECISION MAKING AND OVERSIGHT Like the other objectives, monitoring data for Objective 3 showed slower progress in Year 1 than in Year 2, though Objective 3 displayed the most indicator achievement out of all objectives in Year 2, implying improvements in mainly output-level results for increasing the space for civil-society engagement in local decision-making and civil society’s capacity for oversight. In Year 1, STF achieved only two of Objective 3’s six indicators, both of which measured progress toward Sub-Result 3.2, Improved civil society capacity for monitoring and oversight. However, in Year 2, STF met or exceeded all six indicators. Despite the positive indicator performance, qualitative data collected presented a more nuanced picture of achievement under this objective, indicating mixed results regarding how much STF’s interventions have actually resulted in civil society’s increased engagement and oversight in government processes. On the positive side, STF consortium and GON KII/SGI respondents, as well as FGD participants, reported examples of how STF trained CSOs and citizens to engage in local decision-making, assisted local governments to conduct public hearings as a platform for citizens to raise their concerns, and facilitated local residents’ inclusion in participatory planning processes. This corresponds with much of the Activity’s output-level indicator successes primarily around the establishment of mechanisms for CSOs’ and citizens’ participation. A female FGD participant furthermore provided a successful example of her CSO’s increased involvement with the ward: “When we learned about how to reach out to the local government and the laws, rules, and process, we realized that they are not for the privileged only but also for women. We focused on the collaboration and coordination with the ward, and with the support from [STF], the ward started trusting us a lot.” Even though mechanisms for participation have been put into place, CSO KII/SGI respondents and FGD participants in particular felt that civil society’s engagement has not been as robust as originally envisioned, and that these mechanisms need to be better mobilized to encourage participation. CSO respondents stated there is still a need to further develop the capacities of Good Citizen Initiatives Networks, TDOs, and other structures, and that this is an ongoing process that takes time. Meanwhile, indirect beneficiary FGD participants from CSOs representing women, as well as men and women from socially marginalized groups, asserted that they want to be engaged and have been trained, but in reality they are still not invited to government planning processes. One female FGD participant said: “People from outside train us well for three long days on our rights and role in local governance, but our own people don’t include us,” while another male FGD participant explained, “We are not invited to participate in policy making or planning. We are impoverished, uneducated, and unavailable because we are preoccupied with day-to-day survival challenges.” GESI Progress toward STF’s four output level indicators to track and measure GESI integration within STF programming were, like its objectives, slow to gain traction in Year 1 and increased in Year 2. In Year 1, the Activity only met its target for the percentage of trainings that integrated GESI components effectively but did not make headway on its other three indicators. In Year 2 the Activity made progress toward all four indicators and met or exceeded two indicators that showed the Activity helped organizations develop and practice GESI policies and integrated GESI components into the vast majority (90 percent) of its trainings. USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 12 Data from the desk review and KII/SGIs confirmed monitoring data trends around GESI integration. In Year 1, STF developed a formal GESI strategy and resulting GESI assessment to identify existing GESI efforts and gaps at the federal, provincial, and local government levels with a view toward integrating GESI across the Activity. This assessment and subsequent GESI Analysis Report resulted in the hiring of a GESI Advisor in March 2021 (Year 2) that bolstered implementation of GESI capacity building at the local level. As a STF consortium respondent indicated, “We worked with all ten municipalities to develop and pass GESI policies. These were done in cross collaboration with Objective 3 where we facilitated an inclusive policy review with CSO organizations. We provided a platform to engage with local government on the development of these important GESI policies in respective local governments.” According to STF quarterly reports, GESI policies provided a framework and foundation for municipalities to ensure GESI is integrated into annual planning and budgeting. A GON respondent at the local level noted that through STF’s GESI support, “women and marginalized communities had an opportunity to voice their needs and expectations while developing these policies.” Though STF has made good headway at implementing interventions (e.g., GESI policy development mentioned above) to promote GESI within its objectives, data suggested the interventions themselves have not yet resulted in higher-level GESI outcomes effectively materializing and that it will take both time and sustained efforts by key stakeholders to effectively address this need. As noted in the achievement descriptions for all three objectives above, data indicated gaps around operationalizing GESI policies so that women and marginalized groups experience the benefits of inclusive legislation and ensuring citizens across civil society can take part in the mechanism established for participation. EFFECTIVENESS FACTORS KII/SGI respondents reported numerous hindering and enabling factors to explain the Activity’s overall slower start up and progress toward its objectives in Year 1 and eventual ability to achieve more in Year 2. Hindering factors included: • Design delays: Though respondents appreciated the collaborative approach, an extended co￾design process between the STF consortium and USAID, as well as MoFAGA concerns over municipality selection, delayed implementation for approximately ten months. • Staff turnover: Turnover of the Chief of Party (COP), Deputy Chief of Party (DCOP), Senior Research Policy Expert, and other positions stalled progress until replacements occurred. • Year 1 priorities: STF’s primary focus in its initial year was on capacity building, policy support, and inroads with local stakeholders, all of which take time to accomplish. However, focus on these priority activities meant that other interventions and outcomes were delayed. • COVID-19 and political instability: Safety concerns and mobility restrictions resulting from the COVID-19 pandemic (which began halfway through STF’s first year), and political instability (e.g., turnover of political leadership positions, changing government policy priorities, and budgetary realignments, especially in Year 2) delayed in-person implementation. Despite the setbacks described above, STF also benefited from the following enabling factors: “The STF project made a significant contribution in the area of leadership development for women and marginalized groups. In cooperation with the STF project, we have developed GESI policy and institutional capacity development of the municipality.” –GON Respondent 13 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV • Focused approach within Nepal’s new federalism system: Given the government lockdown during COVID-19 and other constraints within the operating environment, STF focused on what it identified as “high-impact” activities in Year 2—including extensive training and capacity building—and was able to make more progress on many of its outputs. Although some respondents, mainly from USAID, were critical of the Activity’s lack of outcomes generated by Year 3, other respondents cautioned that the federalism system and processes were newly established when STF started and that it takes time for outcomes to be realized. As one beneficiary noted, “institutionalizing federalism, it is not something that can be done with limited time and effort. There has to be continuous time and effort.” In line with this, a GON respondent noted that, “the [STF] project was designed in the context of the federal model being implemented for the first time. It was designed to focus on the following components like development of structures, institutionalization of those structures, and capacity building,” all of which take time. • Integrated interventions: Respondents noted an integrated approach whereby certain interventions were meant to help progress toward multiple objectives. For example, GESI trainings for municipalities under Objective 2 were also meant to help create inclusive policies under Objective 1. • IT Support: The Activity’s effective use of information technology (IT) support and equipment helped interventions continue and municipalities deliver services during the pandemic. See EQ 5 for more information on STF’s adaptations to COVID-19. CONCLUSIONS STF has made progress under all three objectives and within its cross-cutting goal of integrating GESI in its activities, though progress primarily remains at the output level to date and further efforts are needed to ensure higher-level outcomes of good, inclusive governance and citizen participation are achieved. Part of the delays in realizing both outputs and outcomes over time had to do with internal and external factors that impeded implementation from beginning promptly. Simultaneously, it may not have been realistic to expect higher-level outcomes within only a few years. As federalism was new when STF started, this meant that many stakeholders—including GON, CSOs, and citizens themselves—needed to be oriented to working in this new system, and that STF’s heavy focus on capacity building and basic outputs in its first two years was necessary to establish a solid foundation for higher level outcomes. EQ 2 How efficient has STF been in achieving results considering the level of funding/expenditures to date? As described in the limitations section, the following findings draw on information obtained from stakeholder interviews regarding their perceptions about STF’s VFM from a qualitative perspective. FINDINGS Although USAID would like to understand STF’s VFM, as KIIs/SGIs respondents from USAID and the STF consortium noted, the contract did not require, and, therefore, the Activity did not have, an explicit VFM framework. As such there was no financial data available that would facilitate a true VFM analysis. However, according to KII/SGIs with USAID and the STF consortium, based on USAID’s request, STF is now preparing more detailed costs estimates by objective as part of its annual and ongoing planning. As one STF consortium respondent noted, “[t]his year [we] developed an implementation plan that [...] estimate[d] the cost of activities [...] This was the first time [we] had USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 14 to put money into activities, and for me, this is something that just needs to happen when you want to do work planning.” STF’S VFM: STAKEHOLDER’S MOST VALUED INTERVENTIONS In developing a VFM framework some literature recommends identifying the right balance among the “three E’s”: economy (reducing input costs), efficiency (striking a balance between inputs and outputs), and effectiveness (achieving intended outcomes). Increasingly, international development organizations working on VFM have also considered a fourth “E”—equity—to account for the importance of reaching different target groups.21 While it may not yet be possible to quantitatively assess the extent which STF is achieving any of the four E’s (the financial data required does not exist), the “four E” VFM framework provides a lens through which to examine STF’s VFM qualitatively. Findings related to the STF interventions stakeholders value the most indicate that the Activity is implementing several strategies that demonstrate all four E’s in its implementation: Cost Sharing: KII/SGI respondents from the STF consortium partners and GON noted that STF has a cost sharing approach with local government partners and CSOs, which is meant to (1) improve the economy by reducing the intervention implementation costs to USAID, (2) improve efficiency through alignment with GON frameworks and processes, and (3) improve effectiveness by contributing to local ownership. As one STF consortium respondent noted, “[w]e have applied a collaborative approach for cost sharing and so that efficiency and ownership increased. All the activities are aligned with the PLGSP framework, municipal planning process and demands created from the beneficiaries also.” An example of this was a Year 2 radio program STF organized in partnership with radio stations to create awareness of and promote greater inclusion in the fiscal year planning process. According an STF consortium respondent, five radio stations aired a series of ten episodes across 17 districts, including STF’s five districts and ten municipalities, reaching over 500,000 people. A similar approach was applied at the municipality level for policy making. Avoiding Duplication of Efforts: In avoiding duplication of efforts, STF also contributed to efficiency and effectiveness (STF synergies with other Donor/IP activities are explored more fully in EQ 3). A Donor/IP respondent whose project coordinated with STF indicated that “[r]ather than repeating the same exercises, we have supplemented each other by covering them in a different way.” The respondent elaborated, “one partner may, for example, work on the design of policies, guidelines, and legislation, while another partner could supplement with capacity building of municipalities in the implementation of these policies, guidelines, and legislation in an integrated manner.” As a result, the two activities have covered more ground facilitating wider coverage in the GON’s shifts toward federalism. Several GON respondents noted similar efforts to avoid duplication, as one explained, “we met [with] Sajhedari and we consulted to ensure how we can minimize duplication. So, we dropped those palikas [where they work]. We do cover things in the palikas that they don’t cover.” Efficiencies through IT Support: Respondents across most stakeholder groups also appreciated STF’s efficiency and effectiveness in terms of its IT support, especially in the context of COVID-19 (COVID-19 implications are discussed more fully in EQ 5). As one STF consortium respondent highlighted, the Activity’s IT support was critical as an alternative means of communication. For instance, due to this support, STF was able to provide municipal staff with training to implement the environmental framework that MoFAGA/PLGSP prepared. As a result, the Activity was able to continue its interventions despite travel and face-to-face interaction constraints. 21 Value for Money and International Development: Deconstructing Myths to Promote a More Constructive Discussion, https://www.oecd.org/development/effectiveness/49652541.pdf. 15 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV Research and Policy Support: STF consortium and local GON KII/SGI respondents also noted STF’s value-add through efficiencies and effectiveness related to research and policy support provided in the last six months. Specifically, STF is facilitating the GON’s operationalization of federalism by developing the policies and research needed for its institutionalization. As one STF consortium respondent noted, On the research side, I think that is also developed much better and becomes clearer for us in the last six months. All support the government with evidence-based research that can drive some policy decisions … About 22 specific research components … will deliver very much on the value for money proposition for the local and federal governments. [The GON] get extremely valuable help with some of the policy formulation at the federal level that will be passed down to local levels with regard to the Civil Service Reform Act, as well as the shared powers under the Constitution. Similarly, a GON respondent stated that “[t]he value-added element of the STF project is that it could provide its technical assistance in various sectors in the municipality such as policy making, planning, developing inventory [of TDO], etc.” A Donor/IP respondent shared a related view, referring STF’s “[s]upport for the drafting of a manual for local governments on how to prepare acts and legislation as well as how to run meetings. Even after the forthcoming local election, this important document for all 753 municipalities is extremely valuable. In the case of our partnership and the result we have, it is satisfactory. We have excellent results with nominal cost.” Even an FGD participant from a marginalized community commented on the value-add of STF’s policy-related work, saying, “[STF] is providing technical support in planning and preparing the policies and their implementation. One of the best example[s] is digital data base system, which will be much helpful in evidence-based planning.” Embedded Municipal Coordinators: KII/SGI respondents from several stakeholder groups noted that STF’s embedded Municipal Coordinators allow the Activity to achieve both efficiencies and effectiveness through increased responsiveness to local government partners. As one Donor/IP respondent explained, “STF has municipal advisers at the municipal level, but we don’t have advisers at the municipal [level][…] We are more focused at the provincial level.” The respondent went on to say that the Activity’s approach offered a distinct value in terms of “day-to-day coordination, [because] STF has embedded technical assistance [Municipal Coordinators], which is much easier … They can contribute more, the advisers with the municipal governments, and they work closely, which can be efficien[t].” Similarly, a local GON respondent noted, “STF works closely with local organizations and people providing direct benefit. The embedded technical assistance [Municipal Coordinators] provided ‘hands on’ coaching that was more beneficial.” Enhancing Historically Marginalized Groups’ Participation: As noted in EQ 1, data from KII/SGIs with GON respondents and FGDs with direct beneficiaries, including youth, suggested that STF is also contributing to increased equity, at least to some degree, in that it has worked with local government partners to develop and implement policies that enhance the participation of historically marginalized groups. Though more effort is needed to operationalize GESI policies and promote participation of marginalized groups, GON KIIs/SGIs and beneficiary FGD data indicated that STF’s efforts in this area have manifested in several different ways. Direct beneficiaries, particularly women and youth trainees, commented on STF’s impact around ensuring their needs are included in local government decisions. One beneficiary noted, “[t]he best or the most liked part of the program was the training, and the most important aspect of the training was that you should go and make plans and raise that to the Ward and then the Ward should prioritize your demands and needs.” Similarly, a beneficiary from a different FGD explained, “We worked in [a] Ward … office with the help of STF … setting up … good governance. We USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 16 worked to enhance coordination with the local government and different persons belonging to marginalized groups.” Furthermore, some direct and indirect beneficiary FGD participants cited equity-related STF efforts when asked what they valued more about STF’s work. For example, an FGD participant valued, “Being able to include all different groups and to discuss ways to see governance that is accessible for all and that is designed for all,” while a participant from another FGD noted STF’s efforts “mobilizing TDOs in both disaster and development activities.” Likewise, a youth FGD participant noted appreciation of STF’s exit poll training that resulted in conducting an exit poll in health facilities/primary health posts that revealed how “service from health facilities was poorly provided for […] diseases [other than] COVID-19 cases.” In contrast, the FGD participants in the men-only group did not find anything of value in STF’s work. STAKEHOLDER VALUE FOR MONEY RECOMMENDATIONS The above indications of STF’s value-add above notwithstanding, the Activity could continue to improve on its VFM according to some respondents. To assess how STF could do this, the ET asked respondents about their recommendations for STF’s VFM framework. STF consortium, GON, and other Donor/IP respondents provided specific suggestions for strengthening the Activity’s VFM: • Work better as a consortium to enhance all four E’s. • Continue cost sharing with the GON to facilitate ongoing economy, efficiency, and effectiveness. • Continue collaboration with other Donor/IPs to ensure complementarity of programming to contribute to ongoing efficiency and effectiveness. • Orient/train local government officials about policies STF supported developing to ensure their implementation will facilitate greater efficiency and effectiveness. • Select less well supported municipalities (e.g., remote municipalities) that have less donor involvement or projects to enhance equity and effectiveness. • Streamline operational processes, such as how technical experts are sourced, SOWs are revised, and work products are developed. • Consider better monitoring evaluation strategies to measure good governance outcomes, as it is difficult to measure good governance at the outcome level (primarily qualitative in nature) and easier to measure it at the output level (primarily quantitative). • Continue applying the LISA tool for the Activity to have a clear vision and strategy that is aligned with government needs and will allow the Activity to offer a better value proposition to governments. As noted above, STF is already preparing more detailed costing by objective. 17 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV CONCLUSIONS VFM has not been a focus for STF to date. Nonetheless, qualitative stakeholder feedback indicates that participants across groups appreciate and value many of STF’s efforts. More specifically, their feedback indicates that STF is providing value along all four of the proposed framework’s dimensions: efficiency, effectiveness, economy, and equity. The extent to which this is occurring is difficult to determine given the lack of quantifiable data and an established VFM framework. This is also evident in stakeholders’ recommendations for improved VFM, which largely focus on continuing several of STF’s current strategies, such as cost sharing and enhanced collaboration both within the STF consortium and with other Donor/IPs and the GON. The introduction of more detailed, objective￾wise budgeting in Year 3 will support quantitative cost-benefit analyses in the future. Increasingly, international development organizations are exploring a holistic approach to VFM with greater emphasis on the quality as well as quantity of results. Other donors and development organizations may be valuable resources for USAID as it considers country and activity-specific VFM frameworks.22 EQ 3 How has STF engaged with relevant stakeholders, including GON entities, other donor projects, and other stakeholders in the field supporting federalism? FINDINGS Respondents often spoke about STF’s added value in response to interview questions related to EQ 2 and EQ 3. This shows the natural overlap between how STF’s improved synergy with the GON and other Donor/IPs is seen as providing VFM. In addition to its improved synergy with the GON and other Donor/IPs, stakeholders also noted the quality of STF’s support, its flexible approach, and the fact that it is the only donor-supported activity working in Province 7 as other indicators of its synergy and value-add to the GON’s federalism efforts. 22 Ibid. “In Year 1 we were struggling to work as a team, and we are very much siloed in how things were implemented … we have worked together well this year across objectives [...]. Our GESI team worked with our policy team to look at all the acts and policies that we [developed or revised] and they along with objective 3 team helped facilitate [a] local meeting [with] civil society organizations [to] make sure GESI considerations were in place during the policy development process. So, the team is really proud of how we involved the citizen consultation meetings with government on policy development.” –STF Consortium Respondent “It just takes a long time for Scopes of Work to be written, approved, and for the [Request for Proposals] RFP to go out and consultants [are] hired …There seems to be a lot of reviewing that goes on … that led to some level of delay [when] the COR also changed … We spend an awful lot of time and … money which I think is a real waste.” –STF Consortium Respondent USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 18 IMPROVING SYNERGIES SGI/KII data showed that coordination and synergy between STF and other Donor/IPs, including other USAID Activities, was quite challenging at first but improved over time. A USAID respondent noted that when USAID learned STF and another related Activity were not initially coordinating this was an “example of [in]effective coordination,” and that “USAID’s message to [both] IPs was that [they] should coordinate.” USAID, GON, STF consortium, and other Donor/IP respondents all noted how STF’s synergy and coordination eventually improved with both the GON and other Donor/IPs, and the STF Year 3 Work Plan includes a table showing how the Activity will continue to prioritize coordinating and aligning interventions with the other key actors’ efforts moving forward.23 Synergy was evident in STF’s greater engagement with local government working groups such as The Asia Foundation (TAF) and GIZ; active engagement with GON coordination and partnership mechanisms; stakeholder involvement/ownership, especially at the local level; and STF’s prioritizing increased collaboration with its consortium and external partners. An STF consortium respondent noted, “we can see … really great synergy with … GIZ, [specifically their Capacity Development Support to Governance [CDSG] project, which] is also supporting [municipalities] for good governance, capacity development, [specifically] the further improvement of [the] ‘Revenue Improvement Action Plan’ (RIAP).” Furthermore, STF consortium and Donor/IP respondents purported that STF’s participation in local government working groups (e.g., TAF, GIZ) has led to more information sharing and improved synergy, while a Donor/IP respondent confirmed “there is bilateral and trilateral informal, more regular, and practical learning networking groups with IDEA, GIZ, and STF.” Regarding synergies with GON, KII/SGI respondents provided clear examples of collaboration at the local, provincial, and national levels. A GON respondent gave an example of how STF had discussions with the government to minimize duplication, and as a result, though the government continues to provide some support in the same municipalities where STF works, it “cover[s] things in the [municipalities] that [STF] doesn’t cover.” The respondent went on to explain that they “have regular meetings with STF to ensure effective coordination.” Additionally, an STF consortium respondent described how at the provincial and national levels, “[w]e now agreed that we are developing one e-learning platform, E-Chautari (digital sharing platform), that will be owned by PLGSP and PCGG.” Another STF consortium respondent noted that “MoFAGA [is] always involved in all our activities, and they used to come to the training during the formation of the initial inception meeting or other types of steering committees.” As GON and STF consortium respondents reported, one way the Activity maintains synergy with other Donor/IPs and the GON is through involvement in various coordination and partnership mechanisms, including the Project Steering Committee (PSC), PCGG, and NARMIN. An STF consortium respondent shared that there is “good coordination and learning sharing because […] MoFAGA are also part of this project in the steering committee [PSC], so they are adding value. They are putting their ideas in meetings, including in steering committee meetings.” A GON respondent corroborated this involvement in the PSC and explained, “[t]o eliminate overlaps and duplication, the [PSC] requires a quarterly breakdown of [STF’s] Annual Work Plan.” A Donor/IP respondent also commented on the Activity’s involvement in these partnership and coordination mechanisms: “[t]here is very good coordination among the STF and Capacity Development Support to Governance (CDSG) of GIZ. We are closely [aligned] among the bilateral programs.” 23 Sajhedari Year 3 Work Plan, p. 7: Exhibit 2: Sajhedari Coordination and Alignment. 19 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV As a result of improved synergy, GON respondents reported feeling greater ownership for Activity interventions, and that this sense of ownership also makes them want to share STF interventions more widely. For example, one GON respondent said that the “Municipality Profile development work is … exemplary work, which is also beneficial to other municipalities. We realize that the Municipality Association of Nepal (MUAN) should record such good practices and disseminate among the other municipalities.” An STF consortium respondent also observed that the GON’s sense of ownership makes them more enthusiastic about operationalizing interventions: “After discussion with these municipalities, [about] facilitating operational guidelines to local municipal authority, they [became] very positive [about] operationaliz[ing] the TDO concept in their municipality.” Developing this sense of ownership was an intentional part of STF’s sustainability plan. As an STF consortium respondent explained, “we provide[d] capacity development training, [and] involve[d] staff and stakeholders right from the beginning—[RIAP] and Judicial Council activities are [a] few examples. The Judicial Committee have three people, [to] whom we provided training, and it has eased their work.” QUALITY OF SUPPORT Stakeholders across all groups also noted the quality of STF’s support stemming from its proactive coordination and collaboration with the GON at all levels and other Donor/IPs, particularly in the areas of policy development, financial management, GESI, and software/digital support. Policy Development As described in EQ 1, policy development has been a primary focus for STF over its first two and half years. A GON respondent said that due to the effective collaboration, STF’s policy development support was “highly relevant to the present context of the municipality. We need such a type of support in the long run, as the terms of elected representatives is only for five years. We have less than a one-year term for the existing elected representatives. This type of support will help to bring … authority at the grassroots.” One specific element of STF’s policy development work that direct beneficiaries especially appreciated was the Activity’s intentional and proactive inclusion of marginalized groups. As one respondent explained, “STF project facilitated a series of consultative meetings between TDOs and [the] municipal authority. Women and marginalized groups such as people living with disabilities [had] the opportunity to participate in such meetings and raise their issues and concerns,” which informed STF’s GESI policy development. An example a direct beneficiary noted was, “During the GESI policy formulation process, we emphasized the need to include the issues of women and marginalized communities including women living with disabilities. Their issues would include the provision of wheelchairs, construction of disabled friendly buildings, disabled people sensitive language, etc.” Another concrete example GON officials noted as resulting from municipalities including marginalized peoples’ inputs in policy and planning formulation was introducing the Cycle City Network, an environmentally oriented program that specifically targeted “girls who have to walk more than 30 minutes to reach schools. The Cycle City program provides bicycles to such girls to facilitate their safe and effective access to education, which also minimizes the incidences of [gender-based violence] GBV.” However, indirect beneficiaries noted that, “We are not invited to participate in policy-making or planning. We are impoverished, uneducated, and unavailable because we are preoccupied with day-to-day survival challenges.” “STF has been effectively collaborating with the municipality. As a result, we feel ownership of the STF support.” –GON Respondent USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 20 Financial Management and Revenue Generation and Distribution As described in EQ 1, GON respondents valued STF’s technical assistance related to financial management. Some municipalities have already seen increased own-source revenue generation, which a GON respondent directly related to STF’s “technical assistance in the [RIAP], [which] helped in budgeting and budget forecasting for three years” on which STF collaborated with GIZ. One reason that aspects of this support was one of STF’s notable contributions was, as a GON respondent commented, “[t]here are no [other] Donor agencies that provided technical assistance in Public Financial Management (PFM).” Additionally, GON respondents reported that STF effectively collaborated with stakeholders at different levels of government, including distributing revenue sources and settling revenue disputes. GESI STF’s GESI work has also provided opportunities for collaboration with other Donor/IPs. As one donor/IP stakeholder explained, “through the platform created by the STF GESI focal person, we had an opportunity to learn from each other.” Another area of STF’s work that involved collaboration with other Donor/IPs and the GON was STF’s work on gender-responsive budgeting (GRB). As a Donor/IP noted, STF “provided substantial inputs on Tayar Nepal on [GRB], which has been really valuable.” Specifically, while GRB had been introduced under the unitary government, with the “introduction of the federal structure…elected representatives kind of [manipulated] the GRB concept…In such a context, the STF project effectively sensitized elected representatives about the importance of GRB.” The stakeholder further explained that “Tayar Nepal [was] new to GRB. While developing the Grievance Redress Mechanism (GRM) localization training session, we included the GRB component with technical assistance from the STF project.” However, as with other support areas, an STF consortium respondent noted additional work is required related to GESI. “There is a kind of gap, but we are in the process of developing the GESI policy with the Ministry of Social Development and collaborating with them.” Software Support A final area of STF support recognized for collaboration with and added value to GON and other Donor/IP efforts is software support. As some of this related to STF’s adaptation to the COVID-19 pandemic and digital platforms, it is discussed further in EQ 5. Likewise, some of this support is related to the financial management support discussed above, specifically facilitating the roll-out of the SUTRA software, in collaboration with the Foreign, Commonwealth and Development Office, which developed it. As one GON respondent noted, “[w]e received various software types of support from the STF project,” which they appreciated so much, they “want the STF project [to] continue [providing] software types of support in the future.” FLEXIBLE APPROACH The GON also appreciated the flexibility of STF’s approach relative to other donor-funded projects. GON respondents noted STF’s flexibility in terms of the areas in which it provided support such as policy making, planning, and technical assistance, and that this led to a more integrated approach. As STF consortium respondents explained, flexibility is central to the Activity’s success as interventions must be adapted to the local context. One example of this that also reflects STF’s collaboration with other Donor/IP activities is its adaptation of a Civil Society: Mutual Accountability Project (CS MAP)-developed leadership training to customize its course curricula. An STF consortium respondent explained that the Activity’s adaptation of the CS MAP training curricula allowed them to “deliver the training part and meet the expectations of the CSOs and their learning.” This example reflects both STF’s collaboration with other Donor/IPs and its responsiveness to USAID/Nepal who initially suggested they adapt the training. Another area where STF was especially adaptive was in response to COVID-19 (discussed more fully in EQ 5), on which 21 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV a USAID respondent commented that STF “adapted [their] work plan [to] refocus on COVID-19 response at [the] local level. They have adapted [their] working style in terms of how to work remotely, [and] monitor quality remotely.” GEOGRAPHIC FOCUS Another value-add STF is contributing to is its work in Sudurpaschim because, as explained by an STF consortium respondent, “not many other donors” are working there. While the other province in which STF works, “there are a lot of interventions by donors.” In contrast, in Province 7 “there are not a lot of interventions by donors, and it is far from Kathmandu. It is harder to get [to] logistically.” As such, this respondent sees STF’s work in Sudurpaschim as a “net value gain.” Additionally, STF is working synergistically with other Donor/IPs to provide wide coverage in governance support to the GON, while minimizing overlap. A Donor/IP respondent pointed out that “[t]here is no overlapping among these two projects (their own and STF). Only one municipality … we are working together.” OPPORTUNITIES FOR GREATER SYNERGY Despite STF’s improved collaboration with the GON and other Donor/IPs, there remain opportunities for even greater synergies. The two main opportunities stakeholders mentioned are collaboration within the STF consortium and providing provincial-level support. Collaboration within the STF Consortium and with Other Donor/IPs STF Consortium stakeholders noted several areas of improved collaboration, particularly with Urban Institute. This included sharing information on Objectives 2 and 3, rather than just Objective 1, to facilitate breaking down the silos within which consortium partners were working previously. Another example noted was reviewing SOWs and work products more quickly. Consortium members also noted challenges with communication and coordination among consortium partners across all objectives. Specifically, one respondent noted that initially consortium members were siloed, each working only within their own objective area without any visibility on work related to other objectives. They also indicated that it could take a long time for STF (and USAID) to approve SOWs and work products. However, an STF consortium respondent acknowledged that things have been improving as “this new COP … has been amazing because since he has arrived, he has instituted a monthly meeting between partners. We meet [one Friday every] month, to talk about what we’re respectively doing, and much more.” Despite this noted improvement, the respondent would like to see more collaboration and coordination efforts. Similarly, USAID provided an example of poor coordination, noting that they would like to see greater synergy among their IPs. While STF responded to a specific opportunity for improved coordination once USAID pointed it out, a USAID respondent noted that “CS MAP offered help to [STF], but in the utilization of resources and existing knowledge at least … municipalities where both are working is not clear” if/how that was happening. However, as noted previously, it appears that STF is now coordinating effectively with CS MAP. Nonetheless, USAID “would have liked to see more synergy as [STF] is tapping into other areas that [other] IPs are not tapping into.” Provincial-Level Support As noted in EQ 2, some respondents felt STF should engage more with the provincial-level government, though STF’s Year 3 Work Plan confirms that they are planning more work at this level, including with the PCGG. An STF consortium respondent noted “feeling [a] gap” because the Activity has “not done many things with [the provincial government].” Likewise, when looking at areas of synergy and connection, recommendations from a GON respondent included, “STF should work with PCGG for synergy in ensuring and enhancing province stake. It is important to move forward with filling the gaps in the province’s demand base activities (resource and technical gaps).” USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 22 Another GON respondent elaborated saying, “PLGSP and PCCG must cooperate with STF. Their work is extremely slow, and they do not handle issues in a timely manner. STF can assist in this area.” Another GON respondent also noted another area that STF could support, “project needs to focus on the massive capacity of elected representatives through PCCG for great contribution and changes.” An STF consortium respondent echoed this saying, “there is a unit of PCGG in every province. We need to collaborate with them.” In terms of how to begin working with the provincial-level government, a GON respondent noted “[t]here was a realization that if different donor agencies, NGOs, and INGOs activities are of similar nature, then there is a need for coordination through [an] informal mechanism or to be led by the [GON]. As a follow up to this meeting, we have decided to go for [an] initial workshop to discuss the collaboration and coordination.” According to the same GON respondent, such a workshop may be helpful in facilitating STF’s greater coordination with the GON at the provincial level because they do not yet “have the frame and model to work with the provincial government.” However, as a Donor/IP respondent noted, the “Swiss worked at the provincial level” as does GIZ. CONCLUSIONS STF has steadily improved synergies with GON and similar donor-funded activities. In doing so, the Activity has become an important resource for other activities, in several critical areas including for financial management, GESI, and policy development broadly. Moreover, its Year 3 Work Plan also indicates its ongoing commitment to continue these efforts. STF’s improved collaboration and coordination with both the GON and other donor/IPs are likely to have positive implications for the Activity’s results. Specifically, the GON’s increased sense of ownership is critical to the STF’s interventions sustainability in that if the GON feels they “own” them, they are much more likely to continue implementing them. Likewise, GON officials are much more likely to expand STF’s work through application in municipalities not currently included in the Activity (i.e., the model municipality approach). There are two areas where STF could engage more fully: first, within its own consortium to ensure clear coordination and collaboration among its partners and across objectives; and second, potentially at the provincial level. However, STF would need to coordinate their work in this area with other Donor/IPs who are already working at the provincial level. EQ 4 What can STF learn from similar projects currently being implemented in the field supporting federalism in the country, and how can STF incorporate those TEACHings in its implementation? FINDINGS LEARNING FROM OTHER PROJECTS A critical lesson STF has already integrated is the importance of collaborating and coordinating with the GON and other Donor/IPs. KII/SGI data indicate that STF is already working with and learning from other federalism projects. Other Donor/IPs commented that there is collaboration and learning between their activities and STF. For example, one respondent stated that there is “[s]trong coordination among the partners” and that “STF has a strong teamwork and a willingness to share with others.” Similarly, respondents from the STF consortium, USAID, and other stakeholders cited clear examples of STF’s learning resulting from its increased collaboration and coordination. 23 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV Learning from other donors has also reinforced the importance of government ownership and working more closely with GON partners, specifically MoFAGA and PLGSP. For example, an STF consortium respondent explained that “GIZ supported [the GON] for electronic apps … [so] now [STF plans] to support them.” In applying these lessons, STF is also collaborating with other GON and international actors to roll out trainings related to the implementation of the SUTRA. In addition to the above lessons already learned from other projects, respondents from other Donor/IPs and the GON purported that STF could better support federalism by (1) providing more digital service delivery support to municipalities, and (2) publishing more reports in Nepali instead of English. Although STF has already provided municipalities with significant digital support to facilitate their adaptation to the COVID-19 pandemic, a national GON respondent noted that “[t]he situation has changed, and the need of digitalized service delivery has become important in the current environment,” thus necessitating even more support than the project has delivered to-date. Several respondents also commented on STF’s tendency to publish reports in English, which they noted have only limited usefulness. One GON respondent commented, “[f]or government employees, producing a report in English is a waste of time,” while a Donor/IP respondent similarly stated, “[w]e learned that [STF] publications are written in English. Who the material is intended for (local government, parliamentarians, provincial level staff/representatives, or local government stakeholders) should be made extremely clear. It should be written in Nepali.” STF, however, noted that some of their work to date has already been published in Nepali and other publications in Nepali are forthcoming. Similarly, in considering the model municipality approach some respondents suggest, STF could learn from GIZ. As one Donor/IP respondent explained, “[w]e do learning workshops with our target municipalities and beyond (we call these “satellite municipalities.”) In our project, the practice is to implement in 16 municipalities. We have also identified municipalities that are in and around these 16. We share the data – best practices are shared with other municipalities via exposure visits, workshops…I think some other development partners are starting to [do this as a best practice].” ACTIVITY GENERATED LEARNING AND ADAPTATION In addition to learning from other federalism projects, KII/SGI data indicated that STF has reflected upon and learned from its own implementation challenges and adapted as a result. USAID as well as STF consortium respondents noted that the Activity was reactive at first, but eventually learned to be more proactive and focused in its efforts. One STF consortium respondent explained: [g]enerally in Year 1, because of COVID-19, we tried to do whatever we could. We threw a net out and tried to assess whatever it was that our local partners needed. And then we tried to respond and provide support to whatever they ask for, (but) as we were moving into Year 2 … we realized that we needed to be much more focused going forward; that as a small project like STF did not have the resources available to us to try to solve all the problems that local governments were having. The same STF consortium respondent elaborated that as a result of needing to be more focused the Activity began “looking more closely at … using the LISA as a framework … to prioritize interventions at the local level to address gaps that existed,” and that STF is now (in activity Year 3) using this assessment tool to “better understand how [they] can fit into some of the other development initiatives happening across the country and where [they] can provide support.” Another critical lesson for STF was that interventions need to incorporate applied learning techniques. According to one STF consortium respondent, “ideas should be translated into practice; [they] should not remain … only [a] theoretical concept.” For example, when training on a social accountability tool or offering capacity building for local champions, the Activity must also provide an opportunity for direct beneficiaries to practice using these tools and apply their learning. As a result, USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 24 STF is now trying to make their work with CSOs and municipalities more action oriented with the hope that these types of tools and training the Activity introduces will continue and be replicated in other locations. The same respondent provided the example of providing a “social accountability tool” along with “focus[ed] training [for] the local champion or local practitioner…to apply these tools.” CURRENT MODEL EFFECTIVENESS Opinions varied within and between KII/SGI respondent types (USAID, STF consortium, Donor/IPs, and GON) regarding the best modalities for supporting federalism. As a Donor/IP respondent noted, “[i]t requires different approaches … [The] right path and right modalities are still not clear.” In particular, respondents differed in their opinions concerning if and how STF should broaden its scope, such as working in more than ten municipalities or at different government levels like other federalism projects do. Continuation of Current Model and Scope Some GON respondents, however, were content with the ten municipality model and the amount of support currently provided to local governments and CSOs. As one GON respondent noted, “[w]e have done exemplary work with the support of the STF project.” Likewise, other GON respondents appreciated STF’s focus on local governments. While acknowledging the importance of provincial-level support, one GON respondent noted, “in comparison to provincial support, [we are] more concerned with and focused on local governance.” None of these cited participants indicated that STF should expand their scope. Moreover, USAID noted that STF’s scalability is not yet known, saying, “[w]e want to see a change in behavior, adoption of new policy that is implemented that is absorbed. This is important for two reasons: (1) that it can be scaled up because then you can demonstrate the process works, and (2) the whole theory behind [STF] and focusing on such a small geographic area is that we are meant to have an activity that can collaboratively work with [the] community of stakeholders from start to finish on government processes […] we want a more meaningful longer term process.” Several GON respondents noted the possibility of expansion by using the current ten municipalities as models for other municipalities. For example, one GON official suggested preparing “a model ward, including the structure and layout of the building, IT infrastructure, logistics, staff capacity building, and staff support.” Another GON respondent noted, “if they discuss about their models and if those are good systems that could be used by palikas, then that could be replicated elsewhere. And that would be really meaningful for the project.” A GON respondent explained how replication could work as they are already sharing their experiences with other municipalities: Other municipalities have been approaching us to know about the revenue generation considerations. They are not fully aware about the procedural aspects of the Municipality Revenue Committee. We have been continually sharing our thoughts and ideas on effective revenue generation. We also imparted our knowledge and skills on the functioning of the Judicial Committee. We observed that the Municipality Profile development work is an exemplary work, which is also beneficial to other municipalities. We realize that Municipality Association of Nepal (MUAN) should record such good practices and disseminate among the other municipalities. Adjustments to Current Model and Scope Some GON official respondents felt the Activity should invest more at the provincial level or criticized STF’s selection process for its ten municipalities. As one GON respondent explained, “[w]orking only at the central or [municipal] level and leaving out the provincial government will make things difficult.” 25 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV The same person also questioned the efficacy of the ten-municipality model, “we keep saying that we are working toward strengthening local governance, but we don’t know how this is possible only by selecting ten [municipalities].” In addition, some GON respondents commented that STF, like other donors, selected municipalities that were relatively more accessible in which to work. According to one GON respondent, “they have selected only easier access palikas, which are really accessible and close to roadsides.” In addition to widening its scope, several GON respondents also indicated that STF consider completely changing their operational model to reduce costs and reserve more funding for programmatic investment. Specifically, one suggested: [m]odalities must be updated in the case of an optional phase, such as cutting operational expenses. Adjustment is required, for example, in the overall number of employees. If the project has a small core team, we can outsource expertise for a variety of activities…. Reducing operational expenses requires minimizing operational budget and maximizing the program budget, which benefits the beneficiaries in the end. Similarly, another GON respondent suggested that STF examine how it sources expertise to provide technical support: “technical services are sought out through the external expert and internal expert. Both models are good, like relying on internal experts and seeking out the expertise we don’t have from external model. There is a bigger market within the private market for expertise. Like e-governance is not the capacity of the government, so outsourcing could help.” CONCLUSIONS STF has already learned and implemented several key lessons from their other Donor/IPs, the GON, and its own implementation experience, such as the importance of collaborating with other donors and USAID IPs, working closely with the GON, having a proactive and focused approach, and incorporating applied learning techniques in its interventions. That said, additional lessons STF can incorporate moving forward include focusing even more on providing digital service delivery support and writing reports in Nepali to make them more accessible and useful for GON officials. As for the best model for STF’s work, given that there is no clear consensus about the best approach and no respondents indicated any specific issues with the current approach or offered any suggestions for an alternative approach, it may be best to maintain the current 10tenmunicipality model. Simultaneously, however, as some stakeholders suggested, STF could work with the GON and other Donor/IPs to find ways to extend the reach of their work by adopting the proposed model municipality approach and ensuring that successful policies, processes, tools, and approaches are widely shared among all Nepal’s municipalities. EQ 5 In what ways has STF already adapted to the new COVID-19 context? “Provincial Government is the important component of federalism. STF doesn’t have the frame and model to work with the provincial government … We cannot strengthen federalism without working with all the three levels of government. Comparatively, looking into the age of the government, its institutionalization, [human resources] perspective, and governance, the utmost focus should be given to provincial government.” – GON Respondent USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 26 FINDINGS IMPACT ON ACTIVITY ACHIEVEMENT COVID-19 has had a tremendous impact on STF and its progress. Data indicates that both Objective 2 and 3 are especially impacted. STF’s Year 3 Work Plan and KII/SGI respondents described how the pandemic has impeded GON’s ability to create a more inclusive and responsive local government, and subsequent government-imposed safety restrictions have impeded citizen’s ability to engage with government. A CSO respondent noted, “we faced difficulties in implementing program activities at the community level due to the COVID-19 restrictions imposed by the government. We could not ensure beneficiaries’ active participation in the implementation of project activities,” while a GON respondent noted that due to a government lockdown “it was difficult to gather people and select the schemes through people’s participation.” Respondents reported fewer insights regarding how COVID-19 impacted Objective 1, but some respondents explicitly noted how the pandemic has hindered implementation of certain interventions in relation to Objective 2. One STF consortium respondent noted that the “project has provided technical capacity to municipalities to make them more responsive, inclusive, accountable, and transparent and effective [at] decision making. We carried out the activities as planned; however, because [of] the COVID-19 pandemic and transportation restrictions, some of them were postponed.” Additionally, a local GON respondent noted limited participation: “we only were able to participate in two or three programs/meetings, but due to COVID-19, we could not go further due to mobility and restrictions.” COVID-19 ADAPTATIONS KII/SGI respondents, in general, noted instances in which STF is responding to the pandemic and put strategies into place to mitigate some of the challenges COVID-19 presented. GON, STF consortium, and USAID respondents, as well as some FGD participants, noted that the Activity has made strides in informing the public about the severity of COVID-19 through various interventions. GON respondents noted that “the STF project supported in developing the social message to make the public aware about infections and provided sanitation masks,” and that “with the support of STF we organized sensitization sessions on the services provided by the District Health Office.” Additionally, in one FGD with returnee migrants, a participant said that “during the COVID-19 pandemic, [STF] was active in public awareness activities.” In particular, STF’s creation of COVID-19 Information Help Desks in municipalities has been crucial in helping municipalities spread awareness of the pandemic. Multiple STF consortium, CSO, and GON KII/SGI respondents, as well as direct beneficiaries who participated in FGDs, cited examples of the help desk’s usefulness, and stated that it should be continued. For example, an FGD participant mentioned that before the help desk there was no record of how many people came in from other countries and could potentially spread the virus, and there was a lack of COVID-19 information being distributed among communities. According to the STF consortium, GON, and CSO respondents, another way STF has adapted to the pandemic is using technology and remote work strategies. These respondents described how STF has provided virtual trainings and technological equipment that assisted GON with mitigating some of the challenges of not being able to meet in person. For example, a GON respondent noted “We were expecting a greater number of people’s participation with the government, and we were planning for lots of co-design and co-implementation. The implementation of the lockdown with Nepal and the presence of COVID-19 negatively impacted the goals of the Objective 3, as it has limited the capacity of citizen engagement that was needed.” –STF Consortium Respondent 27 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV that “due to the COVID-19 pandemic, we were unable to visit [TDOs] to select plans and schemes. We used virtual means as well as mobilized [a radio station] to seek citizens’ participation in the selection of municipality plans and schemes. With the support from the STF project, we provided orientation on a seven-step planning process to TDOs, ward representatives, and civil society.” IMPLICATIONS FOR THEORY OF CHANGE One of the assumptions in the theory of change is that “STF achievements will be possible if no natural disasters (category 3 level) or health-related pandemics occur during the life of the Activity.”24 Due to COVID-19, this assumption no longer holds and has shifted implementation modalities. This has especially impacted Objective 2, the goal of which is to build better government capacity at both the provincial and municipal level to improve decision-making and public service delivery and institutionalize capacity building. As noted above, the pandemic limited STF’s capacity to implement interventions, especially at the municipal level. To mitigate the negative effects of the pandemic on Objective 2 implementation and effectiveness, STF needs to continue to adjust its approach, specifically at the municipal level. Multiple GON respondents noted that the STF needs to do more capacity building of elected representatives as well as municipal government staff. A GON respondent noted that “the municipality needs to update the disaster management policies and procedures by the sub￾metropolitan [area].” Additionally, to further advance the Objective 2 goals, an STF stakeholder noted that “based on the project’s current experience, STF should deliver a practical session/capacity building training to upcoming newly elected representatives and staff.” The presence of COVID-19 has highlighted that there is still work needed at the municipal level and that additional trainings are needed to advance Objective 2. Additionally, as a GON respondent noted, the implementation of digital literacy and empowerment within the COVID-19 context would be beneficial in improving federalism. A GON respondent explained that “if they [STF] could help develop a system which [that] could strengthen the effectiveness of service delivery and digitize the service delivery system, that would have been useful. Specifically, in the COVID-19 context, it will prove very useful and important.” Additionally, a USAID respondent noted the benefit of transitioning the Activity into a more digital framework, citing that “if you did not want to do something virtual, it would have to be smaller numbers and more frequent, while if you do something that is virtual, you can have a much larger platform.” As respondents noted, digital platforms could provide the opportunity to train palika municipality staff with digital trainings as well as create a platform in which citizen engagement can be increased. Additionally, another assumption that has been shifted is that “All tiers of government (federal, provincial, and local) are receptive and collaborate with STF in the targeted regions, including dedicating budget and resources.” This assumption did not hold throughout the Activity because, according to a USAID respondent, GON’s priorities and resources have shifted towards handling COVID-19 and away from STF. Different GON respondents corroborated this, noting that “local governments were hesitant to see our feedback as they argued that due to COVID-19 pandemic, we couldn’t address transparency and accountability issues,” and “the implementation of PLGSP phase started after I joined the office, but we could not work as planned and use the resources due to COVID-19 first phase and subsequent phases. In our informal discussions with STF also we discussed that we could not follow the expenditure due to COVID-19.” 24 USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 28 CONCLUSIONS The presence of COVID-19 has created multiple obstacles to citizen engagement and implementation of some activities that have negatively impacted STF’s progress toward its Objectives, especially Objectives 2 and 3. Nonetheless, STF has adapted to the pandemic in two main ways: (1) remote work strategies that allowed for continued communication and trainings during COVID-19, and (2) interventions to provide communities with the necessary information to deal with COVID-19, including a help desk. These efforts have mitigated some of the implementation challenges COVID-19 created. Regarding the implications of COVID-19 on the theory of change, while a change in the overall theory of change may not be necessary, two of the assumptions within the theory of change should be re-evaluated. Specifically, STF still needs to provide continued support to municipalities to ensure better service delivery as well as act on the assumption that COVID-19 continues to influence the Activity. In addition, STF should shift its modalities to account for the on-going presence of COVID￾19 within their allocated resources. STF is currently reevaluating Assumption 3 based on the understanding that COVID challenges and restrictions will continue to effect the Activity. RECOMMENDATIONS In the future, USAID/Nepal should consider the following: 1. USAID/Nepal should exercise the STF option period (a point on which both the ET and GON concur) with two possible modifications. First, USAID and STF may consider an increased focus at the provincial level, in coordination with other Donor/IPs already working at that level. Second, USAID and STF could possibly take up respondents’ recommendation of developing the current ten municipalities as models for other municipalities. STF has not had sufficient implementation time to demonstrate the effectiveness of its approach, especially considering all the implementation challenges, including turnover in key personnel, initial difficulties coordinating with the GON particularly regarding STF’s geographic focus, and COVID-19. Moreover, STF’s efforts to improve implementation specifically around breaking down the silos that separated implementation of its three objectives and improving synergies with all levels of the GON as well as other Donor/IPs, are moving in the right direction and should be continued. Finally, given that the broad systemic work of institutionalizing federalism takes considerable time, more time is required for the Activity to realize outcome level achievements. 2. USAID should maintain STF’s current geographic scope, while working with the Activity and relevant GON respondents to develop processes for sharing proven policies, tools, and so on with non-targeted municipalities. For example, the own source revenue-generating resources such as the RIAP and training were well received and implemented. It is likely that other municipalities could benefit from them. By sharing them with national associations (MUAN and NARMIN), all remaining 743 municipalities can learn and replicate allowing STF to have a wider reach without increasing the number of municipalities within which it works. 3. If USAID/Nepal is interested in quantitatively capturing any Activity’s VFM, including STF, the Mission should consider requiring the financial reporting necessary to facilitate such an analysis in the Activity’s contract. As STF completes its base period and moves into the option period, should USAID exercise it, the Activity should consider the following: 4. Continue efforts already started to improve implementation, especially: a. better integrate work across STF objectives, 29 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV b. enhance synergies with other similar programs and GON initiatives, particularly in the areas where STF is already adding value such as GESI and PFM, and c. focus on policy operationalization rather than just policy development. 5. Develop and apply technology more, specifically related to e-Governance, which can ensure transparency and accountability among citizens at the municipal level. 6. Work more closely with municipalities to develop systems that can better mainstream marginalized communities into planning processes and project implementation. To facilitate this, STF should engage marginalized communities in dialogue more directly via CSOs/TDOs to better understand how to engage them in policy making to account for their primary focus on economic survival. 7. Ensure that all key publications, including policy documents, are translated into Nepali so that GON officials and stakeholders at all levels have access to and can use them. 8. Continue to focus on capacity building around policy implementation and support to new systems/processes operationalization. For example, STF should continue interventions such as its support of the roll-out and implementation of the SUTRA financial management support. This exemplifies effective collaboration with other similar Donor/IP programs. 9. STF should closely collaborate with other actors who are active at the provincial level to determine where STF can add value. USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 30 ANNEX I: EVALUATION DESIGN MATRIX EVALUATION QUESTIONS DATA COLLECTION TOOLS DATA SOURCE LOCATION OF DATA COLLECTION ANALYSIS AND REPORTING How has STF made progress on its three objectives? Desk review of activity documents Activity Annual Progress Reports, Quarterly Progress Reports, M&E Plans, Workplans, Results Framework Workplans, and Results Framework Desk Review: National Team Leader, and other team members Lumbini and Sudur Paschim in Nepal Municipalities where the project is operational Municipalities selected to have a representative character and coverage of operational geographic representation: Himalayan regional, Tarai, rural, and urban municipalities selected for data collection The team will analyze data collected through interview notes and desk review notes to discover the linkages with the Activity objectives, results, and assumptions, to answer the evaluation questions. The team will factor in challenges and draw conclusions. Wherever they analyze quantitative data, they will use charts, diagrams, and tables to present the data for better visibility and understanding. The team will assess the value for money based on project financial statements collected and reviewed and the views of key stakeholders. Weekly reporting channels will be established with SI team at the headquarters. National Team Members and the International Subject Matter Expert. The National Team Leader will report to the Project Director/ Program Manager, and interact to exchange views, findings, analyses, and interpretations. Draft reports will be presented for validation and review and finalized after receipt of 31 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV EVALUATION QUESTIONS DATA COLLECTION TOOLS DATA SOURCE LOCATION OF DATA COLLECTION ANALYSIS AND REPORTING comments. Necessary presentation will be made, comments, sought, and report finalized. Key informant interviews The project staff, Activity COP and DCOP, USAID/Nepal other players like the United Nations Development Programme (UNDP), GIZ, beneficiaries, municipality staff, heads of municipalities, NGOs, CSOs and other stakeholders Small group interviews Staff of municipalities, government representatives Focus group discussions Marginalized populations How efficient has STF been in achieving results considering the level of funding/expenditure to date? Desk review of documents Fact sheets, Activity documents, Annual Reports and Progress Reports, M&E Plans, Workplans, and Results Framework Key informant interviews The project staff, Activity COP and DCOP, USAID/Nepal other players like UNDP, GIZ, beneficiaries, municipality staff, heads of municipalities, NGOs, CSOs and other stakeholders, select beneficiaries Small group interviews Staff of municipalities, government representatives Focus group discussions Marginalized populations How has STF engaged with relevant stakeholders, including GON entities (e.g., MoFAGA, provincial Desk review of documents Fact sheets, Activity documents, Annual Reports and Progress Reports, M&E Plans, Workplans, and Results Framework USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 32 EVALUATION QUESTIONS DATA COLLECTION TOOLS DATA SOURCE LOCATION OF DATA COLLECTION ANALYSIS AND REPORTING government, local governments), other donor projects and other stakeholders (e.g., PLGSP, GIZ project, DRG implementing partners) supporting federalism? Key informant interviews The project staff, Activity COP and DCOP, USAID/Nepal other players like UNDP, GIZ, beneficiaries, municipality staff, heads of municipalities, NGOs, CSOs and other stakeholders, select beneficiaries What can STF learn from similar projects currently being implemented (e.g., PLGSP, GIZ) supporting federalism, and how can STF incorporate those learnings in its implementation? Desk review of documents Fact sheets, Activity documents, Annual Reports and Progress Reports, M&E Plans, Workplans, and Results Framework Key informant interviews Beneficiaries, municipalities, and other stakeholders Small group interviews Staff of municipalities, government representatives Focus group discussions Marginalized populations Source material verification and validation Meetings and interviews with the stakeholders In what ways has STF already adapted to the COVID-19 context? In addressing this question, the evaluation team must consider the implications of these adaptations on STF’s theory of change, strategic approaches, and assumptions. Desk review of documents Fact sheets, Activity documents, Annual Reports and Progress Reports, M&E Plans, Workplans, and Results Framework Key informant interviews The project staff, Activity COP and DCOP, USAID/Nepal other players like UNDP, GIZ, beneficiaries, municipality staff, heads of municipalities, NGOs, CSOs and other stakeholders, select beneficiaries Small group interviews Staff of municipalities, government representatives 33 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV EVALUATION QUESTIONS DATA COLLECTION TOOLS DATA SOURCE LOCATION OF DATA COLLECTION ANALYSIS AND REPORTING Focus group discussions Marginalized populations 33 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV ANNEX II: DATA COLLECTION TOOLS TOOL 1- INTERVIEWS WITH USAID/NEPAL Introduction Hello, my name is_______. I am part of an evaluation team from Social Impact (SI). SI is an international consulting company with its headquarters in Virginia, USA and works to improve development effectiveness around the world through evaluation, capacity building and strategic planning. SI has been asked by USAID to conduct an independent evaluation of the Sajhedari Support to Federalism (STF) Activity. STF is a USAID-funded activity to provide primarily technical assistance to the Government of Nepal in transitioning from a unitary state to a federation of provinces with more effective, accountable, responsive, and inclusive local governance. This evaluation aims to assess the performance of STF, identify best practices and lessons learned, and make recommendations needed to meet their intended objectives. As independent evaluators, SI does not represent the Government of Nepal or USAID. We selected you to participate in this evaluation because of your experience with STF and its programming. I would like to request that you read (or I will read to you) this Consent Form, to make sure that you are fully informed about this evaluation before you decide if you want to participate or not. After I have introduced this evaluation to you and have gone through what is expected of you, and if you agree to participate, I will sign this form indicating your consent. We will give you a copy of this form if you would like. You will have a chance to answer questions before consenting. Information about the evaluation If you agree to be part of this evaluation, we are going to ask you and other key informants about STF interventions that you may know about, and your perceptions of their achievements and challenges. We will also ask you about how they could be improved to achieve more significant results. We plan to conduct 61 interviews similar to this with individuals or small groups, at the national, provincial, and municipality levels and across the locations where STF works. Your participation in this interview will take approximately 60 minutes. USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 34 Your rights You have the right to refuse to answer any questions or to stop the interview at any time. You have the right to talk about this evaluation to whomever you choose. Accepting to participate or ending your participation will not affect your working relationship with STF. If you decide not to participate, we will accept your decision without holding anything against you. Your relationship with STF or other organizations that provide similar services will not be affected at all. Possible risks We do not anticipate any significant risks to you or your organization/facility because of your participation in this interview. However, please note that should you choose to participate in this interview, you will be taking time away from your regular activities, which may affect your routine tasks. Also, some aspects of the interview might affect your feelings regarding your work. We are also obligated to share a list of all participants with USAID. Although we will not link any findings or quotations directly to you in the report or share our list of participants with anyone outside USAID, your position as XXXX makes it such that we cannot guarantee confidentiality. They will, therefore, know that you spoke with us, but we will not share with them what was said. Possible benefits The results of this evaluation are expected to inform USAID’s decision to continue STF’s option period as designed, consider design modifications, or terminate the activity. Your participation in this evaluation will therefore be beneficial to STF and similar programs. By participating in this interview, you will, however, get no immediate and direct personal benefit. Confidentiality The information you share will be kept confidential and will not linked to you within the report. We will share the opinions you give us in a report to other entities outside of our team, but all your answers will be combined with those from other interviews, so nothing you share can be directly linked back to our conversation today. We want to record your interview to help us transcribe what you said accurately, but you may choose to participate and not be recorded, with no consequence. If we record this interview, the recordings will be uploaded to a password protected site and will not be made public. Only the evaluation team will have access to these recordings, and they will be permanently deleted two months after the evaluation is completed. We will not record your name in our data collection tools or notes or in this consent form. We will only include your name in an annex to the final evaluation report shared with USAID and will omit your official designation and place of work. However, given the nature of the interview and your position as XXXX, we cannot guarantee confidentiality. The public report will not include a participant list. 35 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV ONLY FOR IN-PERSON INTERVIEWS – PROTECTION FROM COVID-19 Given the COVID-19 pandemic there are several reasons you may choose not to participate in this interview. If you or someone in your household or workplace has been feeling sick, including having a cough, cold-like symptoms, or a high temperature in the past two weeks, we kindly ask you not to participate for your safety and the safety of others. Also, we plan to use safety protocols when we talk such as maintaining a distance of six feet during interviews, wearing face masks, utilizing well-ventilated areas, and using hand sanitizer before, during, and after interviews. If you are worried that these measures are not possible or you cannot adhere to these requests, the study team can contact you remotely for an interview instead. Likewise, if you do not feel comfortable the day of the interview for any reason, you can decline to participate or end the interview early without any consequence. Also, please note that due to COVID-19 we will be keeping a log of all interviews including your name and contact information to facilitate contact tracing should any member of the evaluation team become ill so that we would be able to inform you. We will delete your information from the log two weeks from today. If you have any questions about this evaluation, you may contact XX or phone number XX. You can also contact the Social Impact Internal Review Board. The contact person is XXX; Address is: 2300 Clarendon Blvd, Suite 1000, Arlington, VA 22201; phone number 703-465-1884; email address: irb@socialimpact.com. EVALUATION QUESTIONS MAIN QUESTIONS PROBING QUESTIONS RESPONSES Intro 0. What is your role in STF? Test EQ1. 1. What were your priorities and expectations for the activity? Test 2. To what extent has the activity met your expectations for achievement in the first three years? Which achievements stand out to you for each objective? Obj 1: The GON creates and implements sound policies and legislation that are informed by Test USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 36 EVALUATION QUESTIONS MAIN QUESTIONS PROBING QUESTIONS RESPONSES (Please provide examples) evidence, and empower, finance, and regulate provincial and municipal governments. Obj 2: The targeted province and selected municipal governments therein are more responsive, inclusive, accountable, transparent, and effective in decision making and delivering priority services to fulfilling core legal responsibilities. Obj 3: Citizens and civil society organizations in targeted geographic areas, including the historically marginalized, actively participate in local decision-making and oversight. 3. In your opinion, how effectively has STF been able to address challenges encountered? (Please provide examples. Please specify which objective(s) the example relates to). test 37 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV EVALUATION QUESTIONS MAIN QUESTIONS PROBING QUESTIONS RESPONSES EQ2. 4. [INTERNAL NOTE FOR ET: Beginning Oct 1, ONLY ask USAID KIs who we have follow up interviews with this question- NOT all USAID respondents.] For each objective, how would you rank the value of programming? 5. To your knowledge, does STF have a value-for-money framework (If yes, how does it work/ what does it include?) 6. Do you have any recommendations for improving value-for￾money for the activity? Test USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 38 EVALUATION QUESTIONS MAIN QUESTIONS PROBING QUESTIONS RESPONSES EQ3. 7. In your opinion, to what extent is STF achieving synergy with similar activities being implemented? (Please provide examples, such as with PLGSP, GIZ project, DRG implementing partners). In what ways is STF adding value? What are the lessons learnt as result of this collaboration? test EQ4. 8. In your opinion, what lessons and best practices could STF derive from other, similar donor-funded programming? Test EQ5. 9. In what ways has STF adapted to the COVID-19 context? 10. Do you have recommendations for revising the activity design in response to the pandemic? For each objective, how has the pandemic affected achievement? Test 39 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV TOOL 2- INTERVIEWS WITH STF STAFF Introduction Hello, my name is_______. I am part of an evaluation team from Social Impact (SI). SI is an international consulting company with its headquarters in Virginia, USA and works to improve development effectiveness around the world through evaluation, capacity building and strategic planning. SI has been asked by USAID to conduct an independent evaluation of the Sajhedari Support to Federalism (STF) Activity. STF is a USAID-funded activity to provide primarily technical assistance to the Government of Nepal in transitioning from a unitary state to a federation of provinces with more effective, accountable, responsive, and inclusive local governance. This evaluation aims to assess the performance of STF, identify best practices and lessons learned, and make recommendations needed to meet their intended objectives. As independent evaluators, SI does not represent the Government of Nepal or USAID. We selected you to participate in this evaluation because of your experience with STF and its programming. I would like to request that you read (or I will read to you) this Consent Form, to make sure that you are fully informed about this evaluation before you decide if you want to participate or not. After I have introduced this evaluation to you and have gone through what is expected of you, and if you agree to participate, I will sign this form indicating your consent. We will give you a copy of this form if you would like. You will have a chance to answer questions before consenting. Information about the evaluation If you agree to be part of this evaluation, we are going to ask you and other key informants about STF interventions that you may know about, and your perceptions of their achievements and challenges. We will also ask you about how they could be improved to achieve more significant results. We plan to conduct 61 interviews similar to this with individuals or small groups, at the national, provincial, and municipality levels and across the locations where STF works. Your participation in this interview will take approximately 60 minutes. Your rights You have the right to refuse to answer any questions or to stop the interview at any time. You have the right to talk about this evaluation to whomever you choose. Accepting to participate or ending your participation will not affect your working relationship with STF. If you decide not to participate, we will accept your decision without holding anything against you. Your relationship with STF or other organizations that provide similar services will not be affected at all. USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 40 Possible risks We do not anticipate any significant risks to you or your organization/facility because of your participation in this interview. However, please note that should you choose to participate in this interview, you will be taking time away from your regular activities, which may affect your routine tasks. Also, some aspects of the interview might affect your feelings regarding your work. We are also obligated to share a list of all participants with USAID. Although we will not link any findings or quotations directly to you in the report or share our list of participants with anyone outside USAID, your position as XXXX makes it such that we cannot guarantee confidentiality. They will, therefore, know that you spoke with us, but we will not share with them what was said. Possible benefits The results of this evaluation are expected to inform USAID’s decision to continue STF’s option period as designed, consider design modifications, or terminate the activity. Your participation in this evaluation will therefore be beneficial to STF and similar programs. By participating in this interview, you will, however, get no immediate and direct personal benefit. Confidentiality The information you share will be kept confidential and will not linked to you within the report. We will share the opinions you give us in a report to other entities outside of our team, but all your answers will be combined with those from other interviews, so nothing you share can be directly linked back to our conversation today. We want to record your interview to help us transcribe what you said accurately, but you may choose to participate and not be recorded, with no consequence. If we record this interview, the recordings will be uploaded to a password protected site and will not be made public. Only the evaluation team will have access to these recordings, and they will be permanently deleted two months after the evaluation is completed. We will not record your name in our data collection tools or notes or in this consent form. We will only include your name in an annex to the final evaluation report shared with USAID and will omit your official designation and place of work. However, given the nature of the interview and your position as XXXX, we cannot guarantee confidentiality. The public report will not include a participant list. ONLY FOR IN-PERSON INTERVIEWS – PROTECTION FROM COVID-19 Given the COVID-19 pandemic there are several reasons you may choose not to participate in this interview. If you or someone in your household or workplace has been feeling sick, including having a cough, cold-like symptoms, or a high temperature in the past two weeks, we kindly ask you not to participate for your safety and the safety of others. Also, we plan to use safety protocols when we talk such as maintaining a distance of six feet during interviews, wearing face masks, utilizing well-ventilated areas, and using hand sanitizer before, during, and after interviews. If you are worried that these measures are not possible or you cannot adhere to these requests, the study team can contact you remotely for an interview instead. Likewise, if you do not feel comfortable the day of the interview for any reason, you can decline to participate or end the interview early without any consequence. Also, 41 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV please note that due to COVID-19 we will be keeping a log of all interviews including your name and contact information to facilitate contact tracing should any member of the evaluation team become ill so that we would be able to inform you. We will delete your information from the log two weeks from today. If you have a question about the evaluation If you have any questions about this evaluation, you may contact XX or phone number XX. You can also contact the Social Impact Internal Review Board. The contact person is XX; Address is: 2300 Clarendon Blvd, Suite 1000, Arlington, VA 22201; phone number 703-465-1884; email address: irb@socialimpact.com. EVALUATION QUESTIONS MAIN QUESTIONS PROBING QUESTIONS RESPONSES Intro 0. What is your role in STF? EQ1. . 1. What were your priorities and expectations for the activity? 2. To what extent has the activity met your expectations for achievement in the first three years? (Please provide examples) Which achievements stand out to you for each objective? Obj 1: The GON creates and implements sound policies and legislation that are informed by evidence, and empower, finance, and regulate provincial and municipal governments. Obj 2: The targeted province and selected municipal governments therein are more responsive, inclusive, accountable, transparent, and effective in decision making and delivering priority services to fulfilling core legal responsibilities. Obj 3: Citizens and civil society organizations in targeted geographic USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 42 EVALUATION QUESTIONS MAIN QUESTIONS PROBING QUESTIONS RESPONSES areas, including the historically marginalized, actively participate in local decision-making and oversight. 3. In your opinion, how effectively has STF been able to address challenges encountered? (Please provide examples. Please specify which objective(s) the example relates to.) EQ2. 4. For each objective, how would you rank the value of programming? 5. To your knowledge, does STF have a value-for-money framework (If yes, how does it work/ what does it include?) 6. Do you have any recommendations for improving value-for￾money for the activity? EQ3. 7. In your opinion, to what extent is STF achieving synergy with similar activities being implemented? (Please provide examples- for example with PLGSP, GIZ project, DRG implementing partners). In what ways is STF adding value? What are the lessons learnt as result of this collaboration? 43 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV EVALUATION QUESTIONS MAIN QUESTIONS PROBING QUESTIONS RESPONSES 8. In the case of GESI, how is Sajhedari leveraging collaboration with other implementers? How (if at all) could this collaboration be improved? EQ4. 9. In your opinion, what lessons and best practices could STF derive from other, similar donor-funded programming? EQ5. 10. In what ways has STF adapted to the COVID-19 context? 11. Do you have recommendations for revising the activity design in response to the pandemic? For each objective, how has the pandemic affected achievement? TOOL 3- INTERVIEWS WITH MUNICIPALITIES AND LOCAL GOVERNMENT/ MOFAGA, PLGSP, MINISTRY OF FINANCE, MOWCSW, NNFRC, AND PROVINCIAL GOVERNMENTS/ NATIONAL ASSOCIATIONS (MUAN, NARMIN, ASSOCIATION OF DISTRICT COORDINATION COMMITTEES OF NEPAL). Introduction Hello, my name is_______. I am part of an evaluation team from Social Impact (SI). SI is an international consulting company with its headquarters in Virginia, USA and works to improve development effectiveness around the world through evaluation, capacity building and strategic planning. SI has been asked by USAID to conduct an independent evaluation of the Sajhedari Support to Federalism (STF) Activity. STF is a USAID-funded activity to provide primarily technical assistance to the GON in transitioning from a unitary state to a federation of provinces with more effective, accountable, responsive, and inclusive local governance. USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 44 This evaluation aims to assess the performance of STF, identify best practices and lessons learned, and make recommendations needed to meet their intended objectives. As independent evaluators, SI does not represent the Government of Nepal or USAID. We selected you to participate in this evaluation because of your experience with STF and its programming. I would like to request that you read (or I will read to you) this Consent Form, to make sure that you are fully informed about this evaluation before you decide if you want to participate or not. After I have introduced this evaluation to you and have gone through what is expected of you, and if you agree to participate, I will sign this form indicating your consent. We will give you a copy of this form if you would like. You will have a chance to answer questions before consenting. Information about the evaluation If you agree to be part of this evaluation, we are going to ask you and other key informants about the interventions of STF that you may know, and your perceptions of their achievements and challenges. We will also ask you about how they could be improved to achieve more significant results. We plan to conduct 61 interviews similar to this with individuals or small groups, at the national, provincial, and municipality levels and across the locations where STF works. Your participation in this interview will take no more than 60 minutes. Your rights You have the right to refuse to answer any questions or to stop the interview at any time. You have the right to talk about this evaluation to whomever you choose. Accepting to participate or ending your participation will not affect your working relationship with STF. If you decide not to participate, we will accept your decision without holding anything against you. Your relationship with STF or other organizations that provide similar services will not be affected at all. Possible risks We do not anticipate any significant risks to you or your organization/facility because of your participation in this interview. However, please note that should you choose to participate in this interview, you will be taking time away from your regular activities, which may affect your routine tasks. Also, some aspects of the interview might affect your feelings regarding your work. We are also obligated to share a list of all participants with USAID. Although we will not link any findings or quotations directly to you in the report or share our list of participants with anyone outside USAID, your position as XXXX makes it such that we cannot guarantee confidentiality. They will, therefore, know that you spoke with us, but we will not share with them what was said. Possible benefits 45 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV The results of this evaluation are expected to inform USAID’s decision regarding STF. Your participation in this evaluation will therefore be beneficial to STF and similar programs. By participating in this interview, you will, however, get no immediate and direct personal benefit. Confidentiality The information you share will be kept confidential and will not linked to you within the report. We will share the opinions you give us in a report to other entities outside of our team, but all your answers will be combined with those from other interviews, so nothing you share can be directly linked back to our conversation today. We want to record your interview to help us transcribe what you said accurately, but you may choose to participate and not be recorded, with no consequence. If we record this interview, the recordings will be uploaded to a password protected site and will not be made public. Only the evaluation team will have access to these recordings, and they will be permanently deleted two months after the evaluation is completed. We will not record your name in our data collection tools or notes or in this consent form. We will only include your name in an annex to the final evaluation report shared with USAID and will omit your official designation and place of work. However, given the nature of the interview and your position as XXXX, we cannot guarantee confidentiality. The public report will not include a participant list. ONLY FOR IN-PERSON INTERVIEWS – PROTECTION FROM COVID-19 Given the COVID-19 pandemic there are several reasons you may choose not to participate in this interview. If you or someone in your household or workplace has been feeling sick, including having a cough, cold-like symptoms, or a high temperature in the past two weeks, we kindly ask you not to participate for your safety and the safety of others. Also, we plan to use safety protocols when we talk such as maintaining a distance of six feet during interviews, wearing face masks, utilizing well-ventilated areas, and using hand sanitizer before, during, and after interviews. If you are worried that these measures are not possible or you cannot adhere to these requests, the study team can contact you remotely for an interview instead. Likewise, if you do not feel comfortable the day of the interview for any reason, you can decline to participate or end the interview early without any consequence. Also, please note that due to COVID-19 we will be keeping a log of all interviews including your name and contact information to facilitate contact tracing should any member of the evaluation team become ill so that we would be able to inform you. We will delete your information from the log two weeks from today. If you have any questions about this evaluation, you may contact XX or phone number XX. You can also contact the Social Impact Internal Review Board. The contact person is XX; Address is: 2300 Clarendon Blvd, Suite 1000, Arlington, VA 22201; phone number 703-465-1884; email address: irb@socialimpact.com. USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 46 EVALUATION QUESTIONS MAIN QUESTIONS PROBING QUESTIONS RESPONSES Intro 0. What is your involvement with Sajhedari? EQ1. 1. What are you hoping Sajhedari will accomplish for GON? 2. To what extent has Sajhedari accomplished what you expected in the first three years? (Please provide examples) 3. Which GON achievements supported by Sajhedari stand out to you? Compared with three years ago, how would you assess the following issues today [Interviewer to read each bullet point and allow respondent to answer before moving to the next bullet]: GON capacity to create and implement sound policies and legislation that are informed by evidence, and empower, finance, and regulate provincial and municipal governments. [Obj 1].  How is the research being used?  How will the policy studies be utilized in Year 3? The degree to which this municipality’s government is responsive, inclusive, accountable, transparent, and effective in 47 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV EVALUATION QUESTIONS MAIN QUESTIONS PROBING QUESTIONS RESPONSES decision making and delivering priority services [Obj 2]. The extent to which citizens and civil society organizations in this geographic area, including the historically marginalized communities, actively participate in implementation and local decision￾making processes [Obj 3]. 4. How have women and other marginalized groups been considered in policy dialogues/legislation formation? How has participation of women representatives and marginalized groups (including youth, geographically isolated communities) been ensured for these policy formulation processes? In your opinion, what is the best way to support women and marginalized groups to become confident and capable in leadership and influential decision making at local level? Can you further explain? 5. In your opinion, how effectively has Sajhedari been able to address challenges USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 48 EVALUATION QUESTIONS MAIN QUESTIONS PROBING QUESTIONS RESPONSES encountered? (Please provide examples.) [Evaluation Team to indicate in the interview notes which objective(s) each example relates to, where possible]. EQ2. 6. What support from Sejhadari do you most value? EQ3. 7. In your opinion, to what extent and in what ways is Sajhedari adding value to the GON’s decentralization efforts? Who are Sajhedari’s main contacts within GON? What is the quality of those relationships? EQ4. 8. In your opinion, what lessons and best practices from other decentralization efforts could Sajhedari apply? In your opinion, is Sajhedari’s focus on 10 municipalities the best model? Or would another model be more effective? 49 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV EVALUATION QUESTIONS MAIN QUESTIONS PROBING QUESTIONS RESPONSES 9. How effectively is Sajhedari leveraging collaboration with other actors to address D/GBV? How (if at all) could this collaboration be improved? EQ5. 10. In what ways has Sajhedari adapted to the COVID-19 context? How has the pandemic affected Sajhedari achievement? 11. Do you have recommendations for how Sajhedari can improve in response to the pandemic? TOOL 4- INTERVIEWS WITH OTHER RELEVANT DONORS & DONOR FUNDED PROJECTS Introduction Hello, my name is_______. I am part of an evaluation team from Social Impact (SI). SI is an international consulting company with its headquarters in Virginia, USA and works to improve development effectiveness around the world through evaluation, capacity building and strategic planning. SI has been asked by USAID to conduct an independent evaluation of the Sajhedari Support to Federalism (STF) Activity. STF is a USAID-funded activity to provide primarily technical assistance to the GON in transitioning from a unitary state to a federation of provinces with more effective, accountable, responsive, and inclusive local governance. USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 50 This evaluation aims to assess the performance of STF, identify best practices and lessons learned, and make recommendations needed to meet their intended objectives. As independent evaluators, SI does not represent the Government of Nepal or USAID. We selected you to participate in this evaluation because of your experience with STF and its programming. I would like to request that you read (or I will read to you) this Consent Form, to make sure that you are fully informed about this evaluation before you decide if you want to participate or not. After I have introduced this evaluation to you and have gone through what is expected of you, and if you agree to participate, I will sign this form indicating your consent. We will give you a copy of this form if you would like. You will have a chance to answer questions before consenting. Information about the evaluation If you agree to be part of this evaluation, we are going to ask you and other key informants about the interventions of STF that you may know, and your perceptions of their achievements and challenges. We will also ask you about how they could be improved to achieve more significant results. We plan to conduct 61 interviews similar to this with individuals or small groups, at the national, provincial, and municipality levels and across the locations where STF works. Your participation in this interview will take no more than 60 minutes. Your rights You have the right to refuse to answer any questions or to stop the interview at any time. You have the right to talk about this evaluation to whomever you choose. Accepting to participate or ending your participation will not affect your working relationship with STF. If you decide not to participate, we will accept your decision without holding anything against you. Your relationship with STF or other organizations that provide similar services will not be affected at all. Possible risks We do not anticipate any significant risks to you or your organization/facility because of your participation in this interview. However, please note that should you choose to participate in this interview, you will be taking time away from your regular activities, which may affect your routine tasks. Also, some aspects of the interview might affect your feelings regarding your work. We are also obligated to share a list of all participants with USAID. Although we will not link any findings or quotations directly to you in the report or share our list of participants with anyone outside USAID, your position as XXXX makes it such that we cannot guarantee confidentiality. They will, therefore, know that you spoke with us, but we will not share with them what was said. Possible benefits 51 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV The results of this evaluation are expected to inform USAID’s decision regarding STF. Your participation in this evaluation will therefore be beneficial to STF and similar programs. By participating in this interview, you will, however, get no immediate and direct personal benefit. Confidentiality The information you share will be kept confidential and will not linked to you within the report. We will share the opinions you give us in a report to other entities outside of our team, but all your answers will be combined with those from other interviews, so nothing you share can be directly linked back to our conversation today. We want to record your interview to help us transcribe what you said accurately, but you may choose to participate and not be recorded, with no consequence. If we record this interview, the recordings will be uploaded to a password protected site and will not be made public. Only the evaluation team will have access to these recordings, and they will be permanently deleted two months after the evaluation is completed. We will not record your name in our data collection tools or notes or in this consent form. We will only include your name in an annex to the final evaluation report shared with USAID and will omit your official designation and place of work. However, given the nature of the interview and your position as XXXX, we cannot guarantee confidentiality. The public report will not include a participant list. ONLY FOR IN-PERSON INTERVIEWS – PROTECTION FROM COVID-19 Given the COVID-19 pandemic there are several reasons you may choose not to participate in this interview. If you or someone in your household or workplace has been feeling sick, including having a cough, cold-like symptoms, or a high temperature in the past two weeks, we kindly ask you not to participate for your safety and the safety of others. Also, we plan to use safety protocols when we talk such as maintaining a distance of six feet during interviews, wearing face masks, utilizing well-ventilated areas, and using hand sanitizer before, during, and after interviews. If you are worried that these measures are not possible or you cannot adhere to these requests, the study team can contact you remotely for an interview instead. Likewise, if you do not feel comfortable the day of the interview for any reason, you can decline to participate or end the interview early without any consequence. Also, please note that due to COVID-19 we will be keeping a log of all interviews including your name and contact information to facilitate contact tracing should any member of the evaluation team become ill so that we would be able to inform you. We will delete your information from the log two weeks from today. If you have any questions about this evaluation, you may contact XX or phone number XX. You can also contact the Social Impact Internal Review Board. The contact person is XX; Address is: 2300 Clarendon Blvd, Suite 1000, Arlington, VA 22201; phone number 703-465-1884; email address: irb@socialimpact.com. USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 52 EVALUATION QUESTIONS MAIN QUESTIONS PROBING QUESTIONS RESPONSES EQ1. 0. How familiar are you with STF? 1. To your knowledge, to what extent has the activity met its planned achievements in the first three years? (Please provide examples) [If respondent is familiar with the objectives, ask the following:] Which achievements stand out to you for each objective? Obj 1: The GON creates and implements sound policies and legislation that are informed by evidence, and empower, finance, and regulate provincial and municipal governments. Obj 2: The targeted province and selected municipal governments therein are more responsive, inclusive, accountable, transparent, and effective in decision making and delivering priority services to fulfilling core legal responsibilities. Obj 3: Citizens and civil society organizations in targeted geographic areas, including the historically marginalized, actively participate in local decision-making and oversight. 2. In your opinion, how effectively has Sajhedari been able to address challenges encountered? (Please provide examples. EQ2. 3. Overall, how would you describe the value of Sajhedari? 4. To your knowledge, does Sajhedari have a value-for-money framework (If yes, how does it Please provide examples. 53 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV EVALUATION QUESTIONS MAIN QUESTIONS PROBING QUESTIONS RESPONSES work/what does it include?) 5. Do you have any recommendations for improving value-for- money for the activity? EQ3. 6. In your opinion, to what extent is Sajhedari achieving synergy with similar activities being implemented by other implementers and the GON? (Please provide examples.) In what ways is STF adding value and leveraging the coordination? Overall, how effective do you think the synergies are? What are the lessons learnt as result of this collaboration? EQ4. 7. In your opinion, what lessons and best practices could Sajhedari derive from similar activities being implemented? In your opinion, is Sajhedari’s focus on 10 municipalities the best model? Or would another model be more effective? EQ5. 8. In what ways has Sajhedari adapted to the COVID-19 context? 9. Do you have recommendations for revising the activity design in response to the pandemic? How has the pandemic affected achievement? TOOL 5- FGDS QUESTIONS FOR SEPARATE GROUPS OF WOMEN, MEN, AND MIXED GROUPS OF MARGINALIZED PEOPLE Introduction USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 54 Hello, my name is_______. I am part of an evaluation team from Social Impact (SI). SI is an international consulting company with its headquarters in Virginia, USA and works to improve development effectiveness around the world through evaluation, capacity building and strategic planning. SI has been asked by USAID to conduct an independent evaluation of the Sajhedari Support to Federalism (STF) Activity. STF is a USAID-funded activity to provide primarily technical assistance to the GON in transitioning from a unitary state to a federation of provinces with more effective, accountable, responsive, and inclusive local governance. This evaluation aims to assess the performance of STF, identify best practices and lessons learned, and make recommendations needed to meet their intended objectives. As independent evaluators, SI does not represent the Government of Nepal or USAID. We selected you to participate in this evaluation because of your experience with STF and its programming. I would like to request that you read (or I will read to you) this Consent Form, to make sure that you are fully informed about this evaluation before you decide if you want to participate or not. After I have introduced this evaluation to you and have gone through what is expected of you, and if you agree to participate, I will sign this form indicating your consent. We will give you a copy of this form if you would like. You will have a chance to answer questions before consenting. Information about the evaluation If you agree to be part of this evaluation, we are going to ask you and other key informants about the interventions of STF that you may know, and your perceptions of their achievements and challenges. We will also ask you about how they could be improved to achieve more significant results. We plan to conduct 61 interviews similar to this with individuals or small groups, at the national, provincial, and municipality levels and across the locations where STF works. Your participation in this interview will take no more than 60 minutes. Your rights You have the right to refuse to answer any questions or to stop the interview at any time. You have the right to talk about this evaluation to whomever you choose. Accepting to participate or ending your participation will not affect your working relationship with STF. If you decide not to participate, we will accept your decision without holding anything against you. Your relationship with STF or other organizations that provide similar services will not be affected at all. Possible risks 55 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV We do not anticipate any significant risks to you or your organization/facility because of your participation in this interview. However, please note that should you choose to participate in this interview, you will be taking time away from your regular activities, which may affect your routine tasks. Also, some aspects of the interview might affect your feelings regarding your work. Possible benefits The results of this evaluation are expected to inform USAID’s decision regarding STF. Your participation in this evaluation will therefore be beneficial to STF and similar programs. By participating in this interview, you will, however, get no immediate and direct personal benefit. Confidentiality The information you share will be kept confidential and will not be disclosed to anyone in a way that can be linked to you. We will share the opinions you give us in a report to other entities outside of our team, but all your answers will be combined with those from other interviews, so nothing you share can be directly linked back to our conversation today. We want to record your interview to help us transcribe what you said accurately, but you may choose to participate and not be recorded, with no consequence. If we record this interview, the recordings will be uploaded to a password protected site and will not be made public. Only the evaluation team of 4 people will have access to these recordings, and they will be permanently deleted two months after the evaluation is completed. We will not record your name in our data collection tools or notes or in this consent form. We will also not include your name in the final evaluation report shared with USAID and will only reference “municipality respondent” to ensure confidentiality as much as possible. ONLY FOR IN-PERSON INTERVIEWS – PROTECTION FROM COVID-19 Given the COVID-19 pandemic there are several reasons you may choose not to participate in this interview. If you or someone in your household or workplace has been feeling sick, including having a cough, cold-like symptoms, or a high temperature in the past two weeks, we kindly ask you not to participate for your safety and the safety of others. Also, we plan to use safety protocols when we talk such as maintaining a distance of six feet during interviews, wearing face masks, utilizing well-ventilated areas, and using hand sanitizer before, during, and after interviews. If you are worried that these measures are not possible or you cannot adhere to these requests, the study team can contact you remotely for an interview instead. Likewise, if you do not feel comfortable the day of the interview for any reason, you can decline to participate or end the interview early without any consequence. Also, please note that due to COVID-19 we will be keeping a log of all interviews including your name and contact information to facilitate contact tracing should any member of the evaluation team become ill so that we would be able to inform you. We will delete your information from the log two weeks from today. USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 56 If you have any questions about this evaluation, you may contact XX or phone number XX. You can also contact the Social Impact Internal Review Board. The contact person is XX; Address is: 2300 Clarendon Blvd, Suite 1000, Arlington, VA 22201; phone number 703-465-1884; email address: irb@socialimpact.com. MAIN QUESTION PROBES RESPONSES Intro What is your understanding of the main objectives of Sajhedari? 0. What do you know about the work/role of your municipalities? 1. How familiar are you the local government and the federal structures? What services in general does the municipality provide for you and your communities in your area? Interviewer Note: These introductory questions are designed to bring the FGD participants on board, most importantly to check their basic knowledge about the project as well as help them to move from simple to complex issues. EQ 1 2. Compared with three years ago, how would you assess the following issues today [Interviewer to read each bullet point and allow respondent to answer before moving to the next bullet]: 2a.GON capacity to create and implement sound policies and legislation [Obj 1]. 2b. The degree to which this municipality’s government is responsive, inclusive, accountable, transparent, and effective in decision making and delivering priority services [Obj 2]. 2c. The extent to which citizens and civil society organizations in this geographic area actively participate in local decision￾making and oversight [Obj 3]. 57 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV MAIN QUESTION PROBES RESPONSES 3. How are you and other marginalized populations you are familiar with involved in municipal planning, policy, and legislation formulation processes? Has the level of involvement changed in the past three years? What key barriers do you and other marginalized groups face in participating in these activities? EQ 2 4. What support from Sajhedari do you most value? EQ 4 5. In your opinion, what lessons and best practices from other decentralization efforts could Sajhedari apply? What specific lessons or best practices for supporting marginalized groups could Sajhedari adopt? 6. What do you understand GESI to mean? 7. How could your municipality improve GESI result? 8. How effectively is Sajhedari leveraging collaboration with other actors to address D/GBV How (is at all) could this coordination be improved? EQ 5 9. In what ways has Sajhedari adapted to the COVID-19 context? 10. Do you have recommendations for how Sajhedari can improve in response to the pandemic? USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 58 59 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV ANNEX III: DOCUMENTS REVIEWED 1. USAID. (2020). Nepal Fact Sheet: Sajhedari Support to Federalism. USAID. 2. STF Contract Base 3. USAID. (2021). Sajhedari-Support to Federalism: Baseline Perception Survey. USAID. 4. Sharma, B. (2021, February 23). Nepal's Supreme Court rules dissolved Parliament must be reinstated. The New York Times. Retrieved November 15, 2021, from https://www.nytimes.com/2021/02/23/world/asia/nepal-parliament-dissolved.html. 5. USAID. (2020). Nepal Gender Equality and Social Inclusion Analysis. USAID. 6. USAID. (2020). Sajhedari-Support to Federalism: Year One Annual Performance Report May 2019-September 2020. USAID. 7. USAID. (2019). Sajhedari-Support to Federalism (STF) Project: Year One, Quarter One Progress Report. USAID. 8. USAID. (2019). Sajhedari-Support to Federalism: Quarterly Progress Report October 1- December 31, 2019 9. USAID. (2021). Sajhedari-Support to Federalism: Quarterly Report April-June 2021. USAID. 10. USAID. (2021). Sajhedari-Support to Federalism: Quarterly Report January 2021-March 2021. USAID. 11. USAID. (2019). USAID/Nepal Support to Federalism: Annual Work Plan – Year 1 (Revised). USAID. 12. USAID. (2021). Sajhedari – Support to Federalism: Year 2 Annual Work Plan Addendum (Revised). USAID. 13. USAID. (2021). Sajhedari – Support to Federalism: Year 2 Annual Work Plan. USAID. 14. USAID. (2021). Sajhedari – Support to Federalism: Monitoring, Evaluation, and Learning (MEL) Plan. USAID. 15. Mpyisi, A., Pathak, K., Isaak, D., Rana, B., and Maharjan, G. (2021). Data Quality Assessment Report: Sajhedari Support to Federalism (STF). USAID. 16. USAID. (2020). Nepal Gender Equality and Social Inclusion Analysis 2020. USAID. 17. USAID. (2021). Sajhedari- Support to Federalism: Community Led COVID19 Awareness Grants, Consolidated Grant Narrative Report. USAID. USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 60 ANNEX IV: ILLUSTRATIVE LIST OF RESPONDENTS 1. COP, STF 2. Senior Staff, USAID 3. Director DRG, USAID 4. Senior Project Manager, ABT Associates 5. Country Director, CECI 6. Finance Director, STF 7. GESI Advisor, STF 8. M&E Director, STF 9. Citizen Engagement Specialist, STF 10. Policy Specialist, STF 11. Director, Team Consult 12. Senior Governance Specialist, STF 13. Senior Provincial Project Manager, STF 14. Mayor, Tulsipur Sub-Metropolitan 15. Deputy Mayor, Tulsipur Sub Metropolitan 16. Staff, Asia Foundation 17. DCOP, CS MAP 18. Chief Administrative Officer (CAO) and Staff, Dipayal Silgadhi Municipality 19. Joint Secretary, MoFAGA 20. CAO and Staff, Tulsipur Municipality 21. CAO and GESI Focal Person, Dhangadhi Sub Metropolitan 22. Senior Research Associate, Urban Institute 23. Deputy Mayor, Bheemdatt Municipality 24. Project Manager, Niti Simbada 25. CAO and Staff, Bittadchir 26. Deputy Mayor, Dipayal Silgadhi Municipality 61 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV 27. Deputy Mayor, Dhangadhi Sub Metropolitan 28. Executive, NARMIN 29. CAO Officers and Staff, Bheemdatt Municipality 30. Mayor, Dhangadhi Sub-Metropolitan 31. Mayor, Dipayal Silgadhi Municipality 32. Mayor, Bheemdatt Municipality 33. National Program Director, PLGSP/MoFAGA 34. Former Contracting Officer Representative, USAID 35 Chief Tech Advisor, GIZ 36. Staff, DAI/Tayar Nepal Private Sector and Social Inclusion Manager 37. Executive Director, PCGG 38. CSOs, Tulsipur 39. CSOs, Dhangadhi Sub Metropolitan 40. CSOs, Bittadchir Municipality 41. Executive Director and Staff, Dipayal Silghadi 42. CSOs, MunicipalityBheemdatt Muncipality 43. Team Leader, PLGSP USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 62 ANNEX V: TEAM MEMBERS Nahakul K.C. National Team Leader/Local Governance Expert: Mr. Nahakul K.C. is a Nepalese governance expert with extensive experience, particularly with UNDP, where he provided local governance sector support, and with USAID/Nepal, where he worked as a technical specialist on a disaster project. On the USAID project, he took the lead role in developing economic recovery planning during COVID￾19 responses in 12 program municipalities. He has also served as Team Leader on several governance-related projects in Nepal, including on the Economic Inclusion evaluation. He has also managed and provided technical input on numerous projects both in Nepal and abroad, including in Afghanistan, Vietnam, and Philippines. He has extensive experience in collecting qualitative data, creating comprehensive data collection tools, conducting quality control for research data collection, and working with the GON at both the sub-national and provincial levels. He has achieved a master’s in community development and a master’s in public administration. Balkrishna Silwal GESI Expert: Mr. Balkrishna Silwal Sharma is a gender and social consultant with more than 15 years of experience in working on conflict sensitivity and GESI analysis for humanitarian and development programs. He has worked on USAID projects such as Knowledge-Based Integrated Sustainable Agriculture and PAHAL, where he assisted with the development of desk review, assessment methodologies, and data collection instruments such as KIIs and FGDs, and conducted GESI monitoring and action planning workshops with thematic team leads and project officers. Additionally, he has experience in gender responsiveness, gender-based violence prevention activities, and monitoring and evaluation. Upama Achyara Field Research Coordinator: Ms. Upama Achyara possess more than 10 years’ working experience in the fields of research and development, good governance, GESI, and women’s and child development issues. She specializes in surveys; baseline studies; performance evaluation; monitoring, evaluation, and reporting; capacity development; and research and knowledge generation. Additionally, Ms. Achyara brings her education background in rural development and her knowledge of women, peace, and security issues to the activity. Bandana Sinha Field Research Coordinator: Ms. Bandana Sinha possess more than 10 years’ working experience in the fields of management, programming, and program implementation. She has experience working with UN, INGOs, and NGOs in areas such as peace, human rights, law, and transitional justice, reconciliation, and development. She specializes in report writing, proposal designing, document translation, and monitoring and evaluation. Additionally, Ms. Sinha brings her educational background in international humanitarian law and conflict peace and development studies. Katherine Vittum Subject Matter Advisor: Katherine Vittum is an international development practitioner with over 20 years of experience working indynamic and complex environments. She has expertise in the design, management, and evaluation of programming on electoral and constitutionalreform, devolution, local 63 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV governance, legislative strengthening, stabilization, political party strengthening, conflict management and mitigation, countering violent extremism, access to justice, and social inclusion. Ms. Vittum has worked in more than 20 countries. She has deep experience with leading multinational teams, remote management, and working with diverse stakeholders and international donors. She is trainedin outcome and impact evaluations, mixed-methods evaluations, meta-analysis, design thinking, facilitation, conflict mitigation, multi-functional teaming, team performance, multi-party negotiations, and high-impact communication skills. USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 64 ANNEX VI: CONFLICT OF INTERESTS DISCLOSURES 65 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 66 67 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV USAID.GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT | 68 69 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV ANNEX VII: SCOPE OF WORK STATEMENT OF WORK (SOW) Performance Evaluation for USAID/Nepal Sajhedari-Support to Federalism Project (STF) STF Identification Data Activity Title: Sajhedari-Support to Federalism Project Contract No: 72036719C00001 Life of Activity: May 13, 2019 – May 12, 2022 (base period) May 13, 2022 – May 12, 2025 (option period) Implementing partner: Abt Associates Inc. Total Estimated Cost: $14,576,381 Contracting Officer’s Representative: Angelina Allen-Mpyisi Alternate Contracting Officer’s Representative: Bishwas Rana BACKGROUND: Following the Comprehensive Peace Agreement in 2006 after the end of a 10-year armed insurgency, the country transitioned from a centuries-old monarchy to a republic based on a multi-party democracy. The new constitution promulgated in 2015 prescribes a three-tier governance system with central, provincial, and local governments—each consisting of an executive and an elected representative branch. Elections took place at all the three tiers of governance levels in 2017 as provisioned by the Constitution of Nepal, 2015. The creation of seven new provinces or states (Pradesh) is expected to address demands for identity￾based federalism. But so far only a few provinces have managed to agree on a name and the Madhesi￾based political parties still insist on amending the Constitution. Before holding local elections, the Government of Nepal (GON) regrouped the former 217 municipalities and 3,117 village bodies into 753 urban (Nagar) and rural (Gaun) municipalities (Palikas). The regrouping offers economies of scale for devolved service delivery, especially in remote areas. However, the new provincial and local governments continue to depend on Kathmandu for staffing and the financial resources to deliver USAID. GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT| 70 those services. To date, this has compromised the ability of the newly elected representatives to live up to their citizens’ expectations. The federalist transition also includes the streamlining and merging of Nepal’s 30-plus central ministries into 18 new federal ministries. Many local and provincial office holders are young, dynamic, and promising leaders who have created a positive enabling environment. But the magnitude and speed of change, the gaps in capacity, and the lack of a gradual or phased approach to the reforms are already tempering this initial enthusiasm and introducing significant risk to the success of the reforms. COVID-19 had a major impact in Nepal in 2020. The Government of Nepal (GON) instituted a nation￾wide lockdown on March 24, 2020. During this time the COVID-19 infection rate decreased and the GON relaxed restrictions in the Summer of 2020. The country was hit with a second wave in early 2021 and currently there are more than 5,000 infections each day with mortality of over 100 each day. The GON once more instituted a lockdown that impacted implementation of development programs as mobility and in-person meetings have been limited. The USAID/Nepal Sajhedari - Support to Federalism (STF) Activity plays a critical role in supporting implementation of the new federal system, in pursuit of the USAID Development Objective (DO) #1: “more effective, participatory and equitable democratic ecosystem.” To assist Nepal in the shift to the new system, Sajhedari-STF focuses on two provinces, Sudurpaschim (Province 7) and Lumbini (Province 5), and provides local level support to ten local governments that includes seven in Sudurpaschim and three in Lumbini province. Through an analytical, reflective operating modality, Sajhedari-STF uses evidence from the field to inform donors and the Government of Nepal’s (GON) national policy. Given the uncertainty of Nepal’s path forward, USAID built in flexibility through a co-design process and placed an emphasis on adaptive management that ensures Sajhedari-STF starts strongly, relies on evidence, and receives stakeholders’ input and buy-in as it moves along. STF is a six-year activity awarded on May 13, 2019, with a total estimated cost of $14.5 million. The activity has a base period of three years until May 12, 2022, with an option period of another three years until May 12, 2025. STF THEORY OF CHANGE AND OBJECTIVES: The Sajhedari-STF Theory of Change explains the causal links between inputs, outputs, outcome, and goal/purpose: “IF a) federalism policy and legislation is improved, b) local governments are more effective, inclusive, and accountable, and c) citizens are constructively engaged in local decision making and oversight THEN, governance in Nepal will be more effective and inclusive at all levels (federal, provincial, and municipal), resulting in improved public decision-making and access to and quality of public services for a more prosperous and peaceful Nepal.” Key assumptions are: ● Support for federalism remains a priority of USAID. ● All tiers of government (Federal, Provincial and Local) are receptive and will collaborate with Sajhedari in the targeted regions on governance issues, including dedicating budget and resources. ● No natural disasters (CAT- 3 level) or health related pandemics occur during the life of the project. 71 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV ● Civic space remains open at all levels of government. ● Political situation in the country remains stable allowing mobility for program implementation. The Sajhedari-STF project (Nepali for partnership) builds on USAID/Nepal’s prior Sajhedari Bikas (SB) (Partnership for Development) Activity (2012-2018). The objective of Sajhedari-STF is to “provide primarily technical assistance to the GON in transitioning from a unitary state to a federation of provinces with more effective, accountable, responsive, and inclusive local governance.” Sajhedari-STF supports USAID DO1: “More Effective, Participatory, and Equitable Democratic Ecosystem” three Intermediate Results (IRs) as shown under Exhibit 1. Sajhedari-STF is composed of three specific objectives: 1. The GON creates and implements sound policies and legislation that are informed by evidence, and empower, finance, and regulate provincial and municipal governments. 2. The targeted province and selected municipal governments therein are more responsive, inclusive, accountable, transparent, and effective in decision making and delivering priority services to fulfilling core legal responsibilities. 3. Citizens and civil society organizations in targeted geographic areas, including the historically marginalized, actively participate in local decision-making and oversight. A full Sajhedari-STF Results Framework is contained in Exhibit 1. Exhibit 1: STF Result Framework PROGRAMMATIC/CONCEPTUAL APPROACH OF STF: To achieve Sajhedari (STF) objectives, Abt intends to impact policies, systems, and practices across all levels of government through an integrated governance approach, with a firm focus on improving USAID. GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT| 72 horizontal and vertical coordination and integration. This will ensure that the allocation of public resources promotes equity, social inclusion, gender balance, and the peaceful integration of the large youth population into society, thereby eliminating the root causes of conflict. Exhibit 2: STF Integrated Governance Approach EVALUATION PURPOSE AND USE: Sajhedari has entered into the third year of its implementation. As the base period will come to an end at the end of the third year, a determination is required about the remaining three-year option period. The purpose of this evaluation of Sajhedari-STF is to assess the progress made towards its objectives and value for money in relation to advancing USAID’s CDCS. USAID will use evaluation findings as one of the factors to determine whether the Mission intends to continue STF’s option period as designed, consider design modifications, or terminate the activity. The mid-term evaluation will also be used by USAID and the Implementing Partner in identifying best practices and lessons learned to guide the program implementation for the remaining period, should the option be exercised. EVALUATION QUESTIONS: 1. How has STF made progress on its three objectives? In answering this question, the evaluation team must assess how progress varies among the three objectives and identify any factors that may be contributing to or hindering progress. 2. How efficient has STF been in achieving results considering the level of funding/expenditure to date? 3. How has STF engaged with relevant stakeholders, including GON entities (e.g. MoFAGA, provincial government, local governments), other donor projects and other stakeholders (e.g. PLGSP, GIZ project, DRG implementing partners) in the field supporting federalism? 4. What can STF learn from similar projects currently being implemented (e.g. PLGSP, GIZ) in the field supporting federalism in the country and how can STF incorporate those learnings in its implementation? 73 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV 5. In what ways has STF already adapted to the new COVID-19 context? In addressing this question, the evaluation team must consider the implications of these adaptations on STF’s theory of change, strategic approaches, and assumptions. EVALUATION DESIGN AND METHODOLOGY: The evaluation will mainly rely on qualitative data, but also on quantitative data to some extent. These could include primary data being collected from local as well as national level stakeholders. The methodology will apply to collecting secondary data from existing reports and records. Illustrative list of respondents for primary data collection: ● Elected leaders/officials in STF supported local governments ● STF supported CSOs ● Beneficiary citizens in STF supported local governments ● Ministry of Federal Affairs and General Administration (MoFAGA): planning and foreign aid coordination division ● Provincial and Local Government Support Program (PLGSP) ● Implementing partner (Abt Associates) ● Collaborating partners/intermediaries ● Other USAID implementing partners ● Other relevant donors ● Other similar projects (e.g. Swiss funded project, DRAFT funded project, GIZ funded project) ● USAID/Nepal staff ● Mayors/chairperson in STF supported local governments ● Deputy mayors/deputy chairpersons in STF supported local governments ● Local CSO leaders in STF supported local governments ● Local government association (NARMIN/MUAN) ● STF CORs In addition to collection of primary data, STF’s records on plan vs progress will be analyzed to assess the performance. A detailed evaluation design will be developed by the evaluation team building on methodology illustrated in this SoW. Considering the COVID-19 imposed lockdown restricts movements in the country, the evaluation methodology will be designed mainly considering the virtual mode of data collection. This includes the international team leader to be working virtually. The national consultants also will be working virtually. If the situation becomes favorable during the time of data collection, the national consultants may conduct some in-person data collection. TIMELINE FOR EVALUATION: The following is a tentative timeline for the evaluation tasks; the detailed timeline will be developed by the contractor as part of finalizing the evaluation plan. The evaluation is expected to start primary data collection in August 2021, draft report submitted in November 2021 and final report submitted by December 2021. Debrief with preliminary findings is expected in October 2021. USAID. GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT| 74 Actions Month of the Year 2021 Jun Jul Aug Sep Oct Nov Dec Share SoW draft (Mission) x Finalize SoW (Mission) x Work with the contractor to on board the evaluation team and to agree on the budget (contractor) x x Initial desk review and drafting of the evaluation design and data collection instruments (Mission and contractor) x x Finalize the evaluation design and instruments x Data collection & compilation x x x Data coding, analysis, report writing x x Debrief on the preliminary findings to USAID & IP x Submission of high-quality draft report to USAID x Comments from USAID x Submission of final report to USAID x DELIVERABLES: The following is a set of evaluation deliverables, with the expected timing of each adhering to the above Evaluation Timeline. 1. An in-briefing with USAID/Nepal upon the finalization of team lead. 2. An evaluation design containing the proposed evaluation outline; the evaluation methodology, implementation schedule and deliverables. 3. A set of data collection instruments; and field work schedule 3. A presentation to USAID (and another to STF management) of the preliminary findings, conclusions and recommendations of the evaluation one week after data collection is completed (team leader to be present). Presentations are expected in October 2021. In addition to answering the evaluation questions, the presentation should also discuss what further adjustments to interventions and implementation approaches would be needed for it to be successful. 75 | SAJHEDARI STF MTE MIDTERM EVALUATION REPORT USAID.GOV 4. A high quality draft evaluation report of not more than 25 pages in length (excluding Annexes), single-spaced in TNR 12 point font, with an executive summary of not more than 3 pages in length. Draft report is expected in November 2021. The report will also include a set of recommendations derived from the evaluation questions as well as on any adjustments required to further strengthen the STF and ensure the success. 5. A final evaluation report with the same specifications as the draft report within two weeks of receipt of consolidated comments in electronic format from USAID. Final report is expected by End of December 2021. 6. A final presentation by the contractor to USAID after the evaluation report is finalized. EVALUATION TEAM: The evaluation team should be composed of experts comprising a range of skills directly relevant to the purpose of the STF mid-term evaluation. USAID proposes the following composition, although relies on the contractor to make a final recommendation on the composition to fit the task: Team Leader: The Team Leader has demonstrated leadership and team management skills. S/he should have a postgraduate degree in a related field, with at least 10 years of experience assessing and/or evaluating development activities for USAID or other donor agencies, preferably with some experience in Asia. S/he has broad technical experience of local governance in developing countries. S/he has specific knowledge of federalism and local governance, and has profound analytical experience, which equips her/him to conduct high-quality and in-depth analysis on addressing federal governance issues. The candidate will have excellent oral and written communication skills in English; writing samples from recent evaluations or reports, and three professional references are required. Local Governance Expert (National Team Leader): The expert must have extensive experience in local governance preferably working with USAID or other donor agencies or government of Nepal. A Nepali citizen, the expert must have a minimum of Master’s Degree in a relevant subject area, and at least seven years of experience including experience working as a member of an evaluation team. Experience leading, or serving as senior technical expert in local governance and or federalism support work is required. The candidate will have excellent oral and written communication skills in English; writing samples and three professional references are required. GESI Expert (partial LOE): Gender Equality and Social Inclusion (GESI) expert must have a minimum of Master’s Degree in social science or relevant subject area, and at least five years of experience including as a member of an evaluation team. The GESI expert must have an extensive knowledge about the GESI issues especially in federal local governance in Nepal. GESI expert will work with the evaluation team to develop evaluation methodology to assess how the Activity has addressed the GESI issues as outlined in the agreement, work plans and monitoring plans. The candidate will have excellent oral and written communication skills in English; writing samples and three professional references are required. Field Researchers: The team will include Nepali Field Researchers, as required, with a Master’s degree or equivalent with at least three years of research/evaluation/assessment focused field experience working in the development or humanitarian sector with a focus on human rights. The researchers will be selected considering their sex, social background (e.g., geography where they come from and familiar with), and language prevalent in the proposed sample districts (e.g. Tharu language in western Tarai). USAID. GOV SAJHEDARI STF MTE MIDTERM EVALUATION REPORT| 76 KEY DOCUMENTS FOR REVIEW: ● Activity program description ● Annual Work plans ● Activity M&E plan ● Indicator tracking sheets ● Progress Reports (Annual, quarterly) ● Baseline and re-assessment reports ● Quality assessment reports (e.g. DQA) ● Thematic reports, e.g. studies, policy briefs (available in DEC, provided by STF if not available in DEC) KEY STAKEHOLDERS TO BE CONSULTED: ● People participating and benefiting from the Activity ● Local governments ● Local government associations ● Ministry of Federal Affairs and General Administration ● Provincial and Local Government Support Program ● Implementing partners ● Collaborating partners/intermediaries, sub-partners ● USAID-funded Activities operating locally ● Other relevant donors - NGOs, CSOs ● USAID/Nepal staff EVALUATION REPORT OUTLINE: The following page numbers should serve as approximate guidelines with a maximum page count of 30 pages. I. Executive Summary (Max 5 pp.) II. Evaluation Purpose and Questions (max 1 pp.) III. Activity Background (max 2 pp.) IV. Evaluation Methods and Limitations (max 3 pp.) V. Findings by each evaluation question (max 21 pp.) VI. Conclusions (max 3 pp.) VII. Recommendations (max 3 pp.) ANNEXES: A. Evaluation Statement of Work (SOW) B. List of documents Reviewed C. List of Informants and their Affiliations (appropriately redacted) D. Data Collection Instruments (e.g., Interview/FGD guides, questionnaires) E. List of data utilized in the Evaluation including type, source, year etc. F. Disclosure of Any Conflicts of Interest G. Statement of Differences [only if applicable]