EVALUATION Afghanistan’s Measure for Accountability and Transparency (AMANAT) Mid-term Performance Evaluation July 2021 This publication was produced with support by the American people and for review by the United States Agency for International Development. It was prepared by It was prepared by George Asatiani, Alexander Kenneth Moler, Abdul Moien Jawhary, and Massouda Kohistani, for Overseas Strategic Consulting, Ltd., for the Afghanistan Monitoring, Evaluation, and Learning Activity (AMELA), Contract #72030619C0004. The authors’ views expressed in this report do not necessarily reflect the views of the United States Agency for International Development or the United States Government. [Redacted] AMANAT MID-TERM PERFORMANCE EVALUATION FINAL EVALUATION REPORT July 2021 Contract number: 72030619C0004 Afghanistan Monitoring, Evaluation, and Learning Activity (AMELA) [Redacted] MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | iii ABSTRACT ----------------------------------------------------[Redacted]------------------------------------------------------ ----------------------------------------------------[Redacted]------------------------------------------------------ ----------------------------. Fighting corruption is an urgent priority for the government, civil society organizations, and the international community. USAID assistance focuses on preventive measures that reduce opportunities for corruption while improving services and complementing efforts by other U.S. Government partners. USAID’s Afghanistan Measure for Accountability and Transparency (AMANAT) project works with select government institutions on procedural reforms that reduce vulnerabilities to corruption in day-to-day administration. The program also strengthens the capacity of local civil society organizations to test and monitor the effectiveness of those reforms, and to advocate for their implementation. The evaluation team assessed AMANAT’s performance, including its relevance, effectiveness, efficiency, potential sustainability of results, and likelihood of impact. According to the evaluation team, select Afghanistan government institutions, supported by AMANAT, have made measurable progress in their anti-corruption efforts with increased ability to identify and address vulnerabilities to corruption. AMANAT-supported CSO initiatives have also increased public awareness, advocacy, monitoring, and oversight activities. AMANAT activities during this period have also laid the groundwork for sustainability, through installing resilient structures and systems. However, the system, in general, continues to face major challenges. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | iv CONTENTS Abstract iii Contents iv Acknowledgments vi Acronyms vii Executive Summary x Project Background x Evaluation Purpose and Use x Evaluation Methodology x Key Findings and Conclusions xi Select Recommendations xiv Introduction 1 Project Background 2 Evaluation Purpose and Use 3 Evaluation Methodology 3 Findings and Conclusions 8 Evaluation Question 1 – Reducing corruption in public service delivery 8 Evaluation Question 2 – Unintended intervention consequences 13 Evaluation Question 3 – AMANAT’s place within the Afghan Context 15 Evaluation Question 4 – Improving civil society’s anti-corruption efforts 18 Evaluation Question 5 – Sustaining AMANAT achievements 22 Gender Focus 25 Recommendations 26 Evaluation Question 1 – Reducing corruption in public service delivery 27 Evaluation Question 2 – Unintended intervention consequences 28 Evaluation Question 3 – AMANAT’s place within the Afghan context 28 Evaluation Question 4 – Improving civil society’s anti-corruption efforts 29 Evaluation Question 5 – Sustaining AMANAT achievements 29 Ad hoc Recommendation: CLIN0002 30 Annexes A-1 Annex A – Evaluation Scope of Work A-1 Annex B – Evaluation Work Plan B-1 Annex C – Core Data Collection Tools C-1 Annex C.1 – Tool for AMANAT Team C-3 Annex C.2 – Tool for AMANAT IP C-6 Annex C.3 – Tool for AMANAT STTA C-9 Annex C.4 – Tool for GIRoA and Sub-national Entities C-11 Annex C.5 – Tool for CSOs (KII) C-14 Annex C.6 – Tool for CSOs (FGD) C-17 Annex C.7 – Tool for Grantee Beneficiary C-20 Annex C.8 – Tool for Other Donor Agency C-21 Annex C.9 – Tool for Ad Hoc Conversation C-23 Annex D – List of Key Documents Reviewed D-1 Annex E – Interview Roster E-1 Annex F – Changes to Scope of Work F-1 MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | v Annex G – Statement of Differences G-1 Annex H – Sources of Information H-1 Annex I – Disclosures of Conflicts of Interest I-1 Annex J – Summary of Evaluation Team Members’ Qualifications J-1 TABLE Table 1. Gender-related performance indicators for Year 2 25 FIGURES Figure 1. AMANAT Theory of Change 1 Figure 2. Map of data collection locations 6 Figure 3. Results from FARYAD baseline and endline community surveys 19 Figure 4. Pre/post test results from select CSO capacity-building trainings 20 MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | vi ACKNOWLEDGMENTS This mid-term evaluation report was the work of the Overseas Strategic Consulting, Ltd. (OSC) team, carried out under the overall guidance of Team Leader, [Redacted], and Deputy Team Leader, [Redacted]. [Redacted] provided overall leadership on content, drafting, and incorporating comments from external reviewers, all while maintaining the intellectual frame, rigor, and intent of the deliverable. This effort was effectively supported by the team members of this mid-term performance evaluation, chosen for their evaluation and subject-matter expertise: [Redacted] and [Redacted] The team devoted careful thought and debate to the key elements of this evaluation. The early draft of the document was improved by comments and feedback from the OSC home office team and USAID/Afghanistan personnel in the field, who helped ensure clarity, reliability, and practicality for the Mission through lessons learned and recommendations for both better management of AMANAT and future project design. Aligned with this goal, OSC actively engaged USAID/Afghanistan, AMANAT project decision makers, the Government of the Islamic Republic of Afghanistan (GIRoA) officials and counterparts, other institutional donors, and other stakeholders throughout the evaluation process. Thanks to our partners and reviewers for providing invaluable assistance during the development of the document. This was the work of many. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | vii ACRONYMS 4A USAID/Assistance for Afghanistan Anti-Corruption Authority AALWDO Afghan Alternative Livelihood, Welfare and Development Organization ADS Automated Directives System AERCA Advancing Effective Reforms for Civic Accountability AMANAT Afghanistan’s Measure for Accountability and Transparency AMELA Afghanistan Monitoring, Evaluation and Learning Activity AMELP Activity Monitoring, Evaluation and Learning Plan APS Annual Program Statement BPR Business Process Redesign CCN Cooperating Country CDCS Country Development Cooperation Strategy CLA Collaborating, Learning and Adapting CLIN Contract Line-Item Numbers COR Contracting Officer’s Representative COVID-19 Coronavirus Disease CRMP Corruption Risk Mitigation Plan CS Civil Society CSCC CSO Anti-Corruption Coordination Center CSO Civil Society Organization DCAR Delegated Cooperation Agreement DFID Department for International Development DO Development Objective DO Direct Observation DOJ Department of Justice DQA Data Quality Assessment DTL Deputy Team Leader EAI Equal Access International EQ Evaluation Question EQRA World Bank’s Project in Afghanistan (EQRA means “read” in Arabic) FARYAD Faryad Radio and TV Network MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | viii FGD Focus Group Discussion FY Fiscal Year GIRoA Government of the Islamic Republic of Afghanistan GoA Government of Afghanistan GRM Grievance Redress Mechanism IA-CM IA-CM Internal Audit – Capability Model IAD Internal Audit Directorate INL International Narcotics and Law Enforcement IP Implementing Partner iPEA Institutional Political Economy Analysis IR Intermediate Result KII Key Informant Interview LOE Level of Effort M&E Monitoring and Evaluation MEC Independent Joint Anti-Corruption Monitoring and Evaluation Committee MEL Monitoring, Evaluation, and Learning MOE Ministry of Education MOHE Ministry of Higher Education MOMD Ministry of Martyrs and Disabled MOPH Ministry of Public Health MORR Ministry of Refugees and Returnees MSI Management Systems International NUG National Unity Government ODG Office of Democracy and Governance OPPD Office of Program and Project Development ORSTW Organization for Research, Skills and Training for Women OSC Overseas Strategic Consulting, Ltd. PIRS Performance Indicator Reference Sheets PS Partner Security RAP Reform Accelerating Plan SAP Survey of the Afghan People MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | ix SIGAR Special Inspector General for Afghanistan Reconstruction SOW Statement of Work SP Sub-Purpose STTA Short-Term Technical Assistance/Assistant TPM Third-Party Monitoring USAID United States Agency for International Development VCAs Vulnerability to Corruption Assessments WISE Women Integrity for Society Empowerment MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | x EXECUTIVE SUMMARY PROJECT BACKGROUND In August 2017, the United States Agency for International Development (USAID) awarded the Afghanistan’s Measure for Accountability and Transparency (AMANAT) project to Management Systems International (MSI), a Tetra Tech company [Redacted]. AMANAT is a five-year initiative to support the Afghan government’s efforts to reduce and prevent corruption in government public services. The project engages with both government and nongovernmental organizations to strengthen transparency and accountability mechanisms and build civil society’s ability to advocate for reform. Working with government beneficiaries, AMANAT seeks to enhance government institutions’ capacity to identify vulnerabilities to corruption in their processes and functions and strengthens government institutions’ capacity to implement anti-corruption reforms. Working with civil society beneficiaries, the project also seeks to build civil society’s ability to assess and improve government anti-corruption efforts. AMANAT has a special programming mechanism by which USAID can issue task orders under an Indefinite Delivery, Indefinite Quantity Contract (IDIQ) mechanism to address tasks that are not covered in the project’s main component. The start date of this USAID initiative was August 23, 2017, and the project is scheduled to conclude on August 22, 2022. EVALUATION PURPOSE AND USE The purpose of this mid-term evaluation is to assess AMANAT interventions and implementation progress in achieving its objectives, identify the successes and weaknesses of this project during implementation, provide lessons learned and recommendations to USAID/Afghanistan for better management of AMANAT’s anti-corruption efforts and help guide the Mission in future project design. The evaluation covers the period from the beginning of the AMANAT project in August 2017 until the end of February 2020. The findings and recommendations will be used to improve AMANAT interventions and, if necessary, to change interventions as well as document achievements and gaps. EVALUATION METHODOLOGY The evaluation used a mixed-methods, gender-conscious approach. Methods included (1) a systematic review of relevant AMANAT literature and program documents, (2) semi-structured key informant interviews (KIIs), (3) focus group discussions (FGDs), and (4) diverse key informants’ responses in ad hoc interviews. This evaluation addressed five evaluation questions (EQs) on the following topics: 1. Reducing corruption in public service delivery, 2. Unintended intervention consequences, 3. AMANAT’s place within the Afghan context, 4. Improving civil society’s anti-corruption efforts, and 5. Sustaining AMANAT achievements. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | xi The evaluation was conducted within the context of USAID's ADS 201.3.6.10 "Use of Evaluations" approach, processes, and protocols. In addition to assessing AMANAT performance in achieving planned objectives, the evaluation highlights lessons learned and provides recommendations, taking into consideration programming of future similar activities in Afghanistan. Aligned with this mission, OSC actively engaged USAID/Afghanistan, AMANAT decision makers, Government of the Islamic Republic of Afghanistan (GIRoA) officials, other institutional donors, and other stakeholders throughout the evaluation process. KEY FINDINGS AND CONCLUSIONS The key findings and conclusions for each research question are supplemented in the report’s main body with additional findings and conclusions and expanded on with supporting data. EVALUATION QUESTION 1 – REDUCING CORRUPTION IN PUBLIC SERVICE DELIVERY Key findings: ● AMANAT facilitated concrete and observable contributions to the reduction of corruption through AMANAT-initiated Vulnerability to Corruption Assessments (VCAs). The project directly resulted in five ministries each identifying several hundred vulnerabilities to corruption across their ministries, ministry activities, and processes ● Business Process Redesign (BPR) activities, facilitated through assistance to the internal audit directorate (IAD), have resulted in positive structural and process reform. Intervention activities were considered successful by beneficiaries when they were perceived to be designed with the needs and capabilities of beneficiaries in mind Conclusions: ● AMANAT has successfully contributed to an increased ability to implement anti￾corruption reforms through strengthening the structures through which reforms are implemented and providing concrete courses of action. Ministries have acted on this increased capacity by implementing suggested process reforms, which will have positive downstream effects on public service delivery. ● AMANAT conceived of and implemented meaningful activities that foster direct impact/corruption mitigation. The approach is largely sound and aligned with the capabilities and needs of the ministries; relevance enables interventions to work. EVALUATION QUESTION 2 – UNINTENDED INTERVENTION CONSEQUENCES Key findings: ● AMANAT grantees’ actions had the unexpected effect of encouraging sub-national government entities, for example, provincial councils, to actively promote anti￾corruption programming in their administrative areas. Grantee activities necessitated extensive engagement with government actors across the project lifecycle. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | xii ● While some sub-national government entities were receptive to grantee activities, others were reportedly actively resistant and reflected negative attention onto grantees. Conclusions: ● AMANAT's activities have had desirable second-order effects, creating a positive feedback loop, and these second-order effects have contributed to improved anti￾corruption efforts.1 ● Resistance from sub-national actors, such as local government administrators, has created unanticipated challenges for grantee activities, potentially limiting grantees’ ability to maximize anti-corruption impact. EVALUATION QUESTION 3 – AMANAT’S PLACE WITHIN THE AFGHAN CONTEXT Key findings: ● AMANAT civil society organization (CSO) trainings have produced evidence of increased knowledge and capacity in multiple topics that facilitate advocacy activities. ● Partner ministries believe programs like AMANAT will be necessary in the post-peace environment, due to increased demand for government services and therefore increased opportunities for corruption. ● Turnover of local administrators at the sub-national level has negatively impacted grantees’ activities, requiring temporary pauses to implementation. Conclusions: ● AMANAT program design and structure is largely unique within the Afghan anti￾corruption space. This contributes to AMANAT's ability to create novel impact on Afghan government and broader Afghan society. ● -----------------------------------------------[Redacted]-------------------------------------------------- -----------------------------------------------[Redacted]-------------------------------------------------- -----------------------------------------------[Redacted]----------------. ● AMANAT is not fully capable of mitigating the impact of turnover at any given level, partially due to a lack of proactive action at the managerial, sub-national and national hierarchical levels within government bodies. 1 A second-order effect is the consequence of a consequence. An anti-corruption activity may produce a desirable result (e.g., identified instance of corruption), which in turn produces another desirable result (e.g., reform minded government actors encourage further identification of corruption), contributing to a positive feedback loop. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | xiii EVALUATION QUESTION 4 – IMPROVING CIVIL SOCIETY’S ANTI-CORRUPTION EFFORTS Key findings: ● AMANAT grantees’ activities resulted in observable advocacy for reform, and further, contributed to instances of reduced corruption at the sub-national level. ● AMANAT has encouraged engagement and enthusiasm to participate in anti-corruption activities from many CSOs, creating a robust base for future activities; however, linkages are largely AMANAT to CSO, rather than CSO to CSO or GIRoA to CSO. ● As noted under evaluation question 3, AMANAT CSO trainings have produced evidence of increased knowledge and capacity in multiple topics that facilitate advocacy activities – laying the foundation for future citizen-led anti-corruption activities. ● AMANAT initially overestimated CSO capacity to implement activities that met programmatic and donor standards for impact and innovation. Conclusions: ● When mobilized, grantees are an effective avenue to generate impact — the grantee program has produced concrete, observable progress against corruption in Afghanistan. ● AMANAT trainings were effective in reaching intended audiences — clear achievement was demonstrated through internal M&E efforts verified through beneficiary reporting. ● AMANAT has made notable effort to overcome the initial challenges the grant program faced, which limited spending and potential impact in the project's early stages. EVALUATION QUESTION 5 – SUSTAINING AMANAT ACHIEVEMENTS Key findings: ● AMANAT activities have laid the groundwork for sustainability through installing resilient structures and systems in partner ministries. ● Grantees report improved effectiveness and operational efficiency as a result of what they learned during the grantee period. This learning has improved their self-reliance. Conclusions: ● AMANAT success in creating the groundwork for sustainable change is potentially limited by partner ministries’ ability to independently maintain IAD’s systems. ● AMANAT has aided (or supported) CSOs in achieving partial sustainability of results, with some evidence that sustainability is manifesting; however, currently these CSOs lack the capacity to fully sustain results. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | xiv SELECT RECOMMENDATIONS This section presents select recommendations for each evaluation question. The body of the report’s Recommendations section presents additional recommendations. EVALUATION QUESTION 1 – REDUCING CORRUPTION IN PUBLIC SERVICE DELIVERY 1. Strengthen technical assistance and support to BPR through increased capacity building, production of and revisions to Corruption Risk Mitigation Plans (CRMPs), and technical assistance to CRMP implementation. 2. Scale up ministry monitoring, evaluation, and learning (MEL) capacities through expanding support to related directorates and sub-directorates, while also improving internal capacities to ensure full attribution of activities to outcomes and objectives. EVALUATION QUESTION 2 – UNINTENDED INTERVENTION CONSEQUENCES 1. Commit to further relationship building with government actors at the sub-national level to increase grantees’ ability to mitigate adverse political will. 2. Further emphasize risk mitigation and safety/security in grantee activity plans to decrease potential harm stemming from negative repercussions of activities. EVALUATION QUESTION 3 – AMANAT’S PLACE WITHIN THE AFGHAN CONTEXT 1. Coordinate with donors and implementers to develop shared understanding of post￾peace scenario implications for the GIRoA anti-corruption commission’s activities. 2. Conduct sustained relationship building at top ministry and national executive levels to ensure resilience to government turnover and restructuring, while opening avenues for increased cooperation. EVALUATION QUESTION 4 – IMPROVING CIVIL SOCIETY’S ANTI-CORRUPTION EFFORTS 1. Focus on two avenues of CSO coordination efforts: a) strengthening linkages within Afghan civil society, and b) facilitating partnerships between the state and civil society. 2. Scale up training to improve CSO watchdog functions, legislation analysis, social auditing, and communications, while ensuring that trainings are responsive to CSO needs. EVALUATION QUESTION 5 – SUSTAINING AMANAT ACHIEVEMENTS 1. Strengthen GIRoA partner training programs through increasing relevant, long-term training engagements to ensure that trainings result in sustained behavior change. AD HOC RECOMMENDATION – CLIN0002 Identify further opportunities to utilize CLIN0002 funding and ensure that opportunities are not missed once they are identified. An example is provided in the Recommendations section at the end of the report. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 1 INTRODUCTION USAID’s Afghanistan’s Measure for Accountability and Transparency (AMANAT) project, contract number AID-306-17-H-00003, awarded on August 23, 2017, works with select government institutions to implement procedural reforms that reduce vulnerabilities to corruption in day-to-day administration. The program also strengthens the capacity of local civil society organizations (CSOs) to test and monitor the effectiveness of those reforms, and to advocate for their implementation. Some institutions have made measurable progress, supported by AMANAT, in their anti-corruption efforts. They increased their ability to identify and address vulnerabilities to corruption, and AMANAT-supported CSO initiatives have increased public awareness, advocacy, monitoring, and oversight activities. However, the system in general continues to face major challenges. AMANAT contributes to the Mission’s third development objective of further legitimizing the government by, among other initiatives, enhancing its performance in delivering services to Afghan citizens. It is designed to achieve a dual goal of improving governance at all levels and having citizens hold the Government of the Islamic Republic of Afghanistan (GIRoA) accountable. AMANAT serves two distinct but linked purposes: 1. Transparency and accountability mechanism strengthened (Purpose 1). 2. Civil society’s ability to advocate reform strengthened (Purpose 2). AMANAT’s theory of change is shown in Figure 1. Figure 1. AMANAT Theory of Change MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 2 PROJECT BACKGROUND Afghanistan remains one of the most corrupt countries in the world. Recent studies show that corruption in Afghanistan is still serious and widespread throughout government institutions. The 2019 Corruption Perception Index of Transparency International ranks Afghanistan 173 out of 180 countries surveyed, with an overall score of just 16 out of 100. The 2018 national corruption survey by Integrity Watch Afghanistan finds that corruption remains central to Afghanistan’s challenges. According to the survey, the vast majority of Afghans (83 percent) believed that corruption had affected the life of people in their local areas, and almost three￾quarters of those surveyed did not trust public services because of corruption. The 2019 Survey of the Afghan People (SAP) found that 81.5 percent view corruption as a major problem. In the same survey, a record 67.9 percent say corruption is a “major problem” in their daily life. The Afghan government has long regarded tackling corruption as one of its top priorities. Dating back to December 2014, the National Unity Government of Afghanistan (NUG) announced its commitment to fight corruption through the Realizing Self-Reliance: Commitments to Reforms and Renewed Partnerships policy framework paper at the London Conference on Afghanistan. This policy framework paper outlined strategic responses to tackle the underlying drivers of corruption identified by the Afghan government. GIRoA has also promised to institute better governance by increasing government accountability, performance, and ability to deliver results in key areas such as judicial and public administration reform. The recent Geneva Conference on Afghanistan on November 28, 2019, co-hosted by the Government of Afghanistan and the United Nations, noted corruption as one of the most serious challenges, besides insecurity and poverty, that require a renewed focus and commitment by the Afghan government and the international donor community. On December 25, 2018, the Afghan government approved a revised anti-corruption strategy that contained 103 benchmarks with completion deadlines between December 2016 and June 2020. The Afghan government updated the strategy again in February 2019 to include 102 of the 103 benchmarks from the December 2018 version. In June 2019 the Special Inspector General for Afghanistan Reconstruction (SIGAR) determined that the Afghan government met 57 of 76 benchmarks, or about 75 percent. Recognizing the urgency to restore trust in existing anti-corruption efforts and institutions that can effectively implement government programs, the Afghan Government is accelerating the pace of reform through a mutually agreed upon set of benchmarks with the international community. The Afghanistan Anti-Corruption Reform Accelerating Plan (RAP) is an integral part of the national efforts to combat corruption with specific benchmarks and timelines for fiscal year 2020. The benchmarks and timelines are subject to revision around the middle of the year to ensure they restore trust with Afghan citizens and the international community. USAID has been providing anti-corruption assistance to the Afghan government and CSOs for about a decade. The past interventions include USAID’s Assistance for Afghanistan Anti￾Corruption Authority (4A) Activity, which supported training the Corruption Prevention Directorate of the High Office of Oversight and Anti-Corruption (HOOAC). Following the conclusion of 4A, USAID supported the implementation of the Advancing Effective Reforms for MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 3 Civic Accountability (AERCA) that provided technical assistance and financial support to government entities to plan and implement reforms recommended by Vulnerability to Corruption Assessments (VCAs). AERCA also supported watchdog efforts by CSOs to test and monitor the implementation of those reforms and to advocate their implementation in instances where target government entities fail to follow through on implementation of reforms. USAID support to the Independent Joint Anti-Corruption Monitoring and Evaluation Committee (MEC) to conduct VCAs ended in December 2019. EVALUATION PURPOSE AND USE The purpose of this mid-term evaluation is to assess AMANAT interventions and implementation progress in achieving its objectives, identify the successes and weaknesses of the project, and provide lessons learned and recommendations to USAID/Afghanistan. This mid-term evaluation also assesses the relevance and usefulness of AMANAT interventions for its target beneficiaries and the sustainability of the results. The evaluation covers the period from the beginning of the AMANAT activity on August 23, 2017, until February 29, 2020. The findings and recommendations of the evaluation will be used to improve AMANAT interventions and, if necessary, to change interventions as well as document achievements and gaps of interventions. The audience of the evaluation report will be the USAID/Afghanistan Mission, including the Office of Democracy and Governance, the Office of Program and Project Development, the Government of Afghanistan and the Implementing Partner. The report will also be of value to International Narcotics and Law Enforcement (INL), the Department of Justice (DOJ), and the SIGAR team. Furthermore, this evaluation will provide information for USAID’s future Country Development Cooperation Strategy (CDCS) and future programming in anti-corruption. To achieve this, the evaluation assesses project performance, including its relevance, effectiveness, efficiency, and likelihood of impact. The evaluation was conducted within the context of updated USAID's ADS 201.3.6.10 "Use of Evaluations" approach, processes, and protocols. In addition to assessing AMANAT’s performance in achieving planned objectives, the evaluation highlights lessons learned and provides recommendations, taking into consideration programming of future similar activities in Afghanistan. Aligned with this mission, the evaluation team actively engaged USAID/Afghanistan, AMANAT decision makers, GIRoA officials, other institutional donors, and other stakeholders throughout the evaluation process. USAID/Afghanistan will also use the evaluation findings primarily to: 1. Learn lessons and recommendations that will guide the design of future similar activities. 2. Inform the development of the next CDCS. EVALUATION METHODOLOGY The evaluation methodology was designed to answer the five evaluation questions (EQs) using a mixed-methods, gender-conscious approach. Annex A provides the Evaluation Scope of Work, on which Overseas Strategic Consulting, Ltd. (OSC), based the design. Annex B contains a MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 4 more detailed overview of the work plan used for this evaluation. See Annex C for data collection tools and interview protocols. EVALUATION QUESTIONS The mid-term evaluation addressed the following questions related to the performance, relevance, and sustainability of AMANAT. This question was partially answered through a desk review of programmatic data, reporting, and internal databases provided. It was augmented by key informant interviews (KIIs), following social distancing protocols, with implementing partner staff, GIRoA counterparts, and other stakeholders. Additionally, the team undertook in-person ad hoc interviews and compared results with feedback from KIIs with relevant documents, strategies, reporting, etc. The analysis included comparison of program indicators, targets against achievements, as well analytical comparison of observed reality and documentation, including limited GIRoA data. This question was answered through a desk review of programmatic data, reporting, specific questions in the KIIs, AMANAT and GIRoA feedback, and ad hoc interviews. The analysis required qualitative data analysis emphasizing contextual analysis of AMANAT activities such as content analysis, as well as qualitative assessment of direct beneficiaries’ perceptions. To answer this question the team performed analysis and synthesis of notes of impressions obtained through KIIs and ad hoc interviews. The evaluation team also considered external feedback from GIRoA counterparts and other institutional donor agencies to assess how AMANAT’s interventions fit in with the overall anti-corruption context in Afghanistan. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 5 Through qualitative data collection, including KIIs and focus group discussions (FGDs), the evaluation team assessed the extent to which interventions were successful through the prism of how CSOs consider AMANAT empowering their aspirations for change. The team also utilized data derived from external document reviews and AMANAT staff KIIs to augment the data collected from Afghan CSOs directly.2 The evaluation team assessed the extent to which the results achieved by the intervention, in the long run, will continue or are likely to continue in the absence of AMANAT support. In KIIs and FGDs, stakeholders provided informed conjecture about results that have exhibited sustained outcomes and the likelihood of the outcomes being sustained. The evaluation team also asked key informants about the factors that were most important to sustained outcomes in anti-corruption efforts and the likelihood that the factors will continue to have a similar influence on intended outcomes. The team also reviewed internal and external documents, including progress reports and documentation from partner ministries, other donor agencies, and CSOs. DATA COLLECTION The team collected data using the methods described below. Review of Project Data — The team conducted a systematic review of literature and program documents from AMANAT’s startup through February 2020. The team reviewed 121 key documents (see Annex D). The team identified key findings and conclusions and then linked them to the evaluation’s objectives. The desk review informed the development of data collection tools as well as the report. Semi-Structured Key Informant Interviews (KIIs) — The team drew informants from a range of stakeholder types: implementing partners (IPs), GIRoA, CSOs, AMANAT grantees, and other donors. The team employed purposive sampling to select respondents. The team combined recommendations from USAID and the IPs with information from the document review and strove to maximize representativeness. A total of 43 KIIs were conducted in [Redacted] and the target 2 Annex E, the interview roster, lists the CSOs that provided respondents, both from core staff and beneficiaries. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 6 provinces. Ten KIIs were conducted with female respondents and 33 were conducted with male respondents. Gender balance was limited by underlying imbalance in employee composition among stakeholders. Focus Group Discussions (FGDS) — The team conducted FGDs in [Redacted] with AMANAT-supported CSOs and beneficiaries, as well as women beneficiaries. The team employed purposive sampling to select respondents and conducted two face-to-face FGDs with a total of six participants over both groups. One FGD was conducted with female participants and one was conducted with male participants. The number of FGDs and the size of the groups were limited by COVID-imposed restrictions; see “Limitations” for further details. Ad Hoc/Follow-Up Interviews — The team conducted ad hoc interviews in [Redacted] with AMANAT’s key team members. The evaluation team employed purposive sampling to select respondents based on results from previous KIIs and FGDs. GEOGRAPHIC COVERAGE – SIX LOCATIONS/PROVINCES SELECTED The evaluation team distributed the field work described in the preceding section across six provinces. KIIs were conducted in the following provinces: [Redacted], [Redacted], [Redacted], [Redacted], [Redacted], and [Redacted]. FGDs and ad hoc interviews were conducted in [Redacted] province. These locations ensured that the evaluation’s findings represent the differing perspectives amongst beneficiaries across geographies (see Figure 3). Figure 2. Map of data collection locations [Redacted] MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 7 ● [Redacted] ● [Redacted] ● [Redacted] ● [Redacted] ● [Redacted] ● [Redacted] ● [Redacted] Province- Few program activities, and no grantee activities, took place in [Redacted] during the evaluation period. For this reason, the team shifted its focus to other provinces with more relevant and frequent AMANAT activity. DATA ANALYSIS The evaluation used qualitative analytical techniques to code, collate, and interpret data from KIIs, discussion groups, and ad hoc interviews. The evaluation team used response themes and relational content analysis to identify response categories and patterns as well as to elucidate emergent themes, contextual factors, and trends. The evaluation team analyzed quantitative data with Microsoft Excel. The evaluation team compared data from the document review, KIIs, FGDs, and ad hoc interviews to determine whether findings were divergent or convergent. When multiple informational streams provided consistent information, these were included as findings. The evaluation team drew conclusions by reflecting on the key findings for each evaluation question. Analysis accounted for gender and social dimensions wherever relevant. The evaluation team also conducted limited descriptive statistical analysis on provided quantitative data. Quantitative data were limited to two output tracking workbooks. LIMITATIONS The evaluation team dealt with several risks and limitations during data collection. Implementation Delays: Movement restrictions imposed by COVID-19 measures stalled the evaluation kick-off until in-person field work was possible. This delay contributed to recall bias. Recall Bias: After an extended period, some respondents’ recall of AMANAT activities may have faded. Informants may have had difficulty accurately recalling changes and innovations since the evaluation covered nearly a three-year initial period. Alternatively, respondents may have been unaware that AMANAT conducted interventions that had beneficial results. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 8 Response Bias: Informants may have formed their responses based on personal motivation rather than the most accurate information. For example, informants may have given the evaluation team positive remarks about the AMANAT project because they expect to receive more assistance in the future. In some cases, informants may have thought that a negative evaluation could mean the end of future program opportunities. In addition, there is an incentive for some GIRoA officials to offer positive responses because their directorates/units benefit from USAID and AMANAT support. Data Gaps: Key informants from early stages of the project were not accessible, which could have limited the depth of information available. The evaluation team reviewed all documentation from the early implementation periods to ensure that core facts were reviewed; however, the team was limited in its ability to extract significant detail from these early stages. Movement Within Afghanistan: ---------[Redacted]--------------------------------------------------- -------------------------------------------------------[Redacted]--------------------------------------------------- -------------------------------------------------------[Redacted]--------------------------------------------------- -------------------------------------------------------[Redacted]--------------------------------------------------- -------------------------------------------------------[Redacted]--------------------------------------------------- -------------------------------------------------------[Redacted]-------------------------------- COVID-19: The ongoing novel coronavirus pandemic contributed to the aforementioned implementation delay. It also limited face-to-face field work for KIIs and FGDs; the team had to conduct many KIIs by phone. For each FGD, the evaluation team contacted eight to ten potential respondents. Some respondents were unwilling to attend in-person focus groups. Additionally, the team had to restrict the number of attendees to a maximum of four per focus group in order to meet room occupancy and social distance requirements. As a result, the group size was below the optimal six to eight attendees per group. FINDINGS AND CONCLUSIONS This section discusses the evaluation’s findings for each evaluation question. Conclusions are then derived from these findings and nested under the evaluation question sub-section. EVALUATION QUESTION 1 – REDUCING CORRUPTION IN PUBLIC SERVICE DELIVERY How and to what extent do AMANAT interventions contribute to reduced corruption in Afghan government institutions and improve public service delivery by the Afghan government? What interventions work? Why? What interventions do not work? Why? AMANAT contributes to government legitimization (USAID/Afghanistan Mission Results Framework Development Objective 3) through two Intermediate Results (IRs): 1) governance at national and provincial levels strengthened (IR 3.2) and 2) citizens hold GIRoA accountable (IR 3.1). The findings and conclusions nested under evaluation question 1 (EQ1) related to IR 3.2. Findings and conclusions relating to IR 3.1 are nested under EQ4. IR 3.2 comprises one sub-IR (Sub-IR 3.2.3), “transparency and accountability mechanisms strengthened,” which in turn comprises two sub-purposes: 1) “GIRoA-led initiatives to identify vulnerabilities enhanced” MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 9 (SP 1.1) and 2) “Capacity of GIRoA institutions to implement anti-corruption reforms strengthened” (SP 1.2). FINDINGS Demonstrable contribution to anti-corruption. AMANAT facilitated concrete and observable contributions to the reduction of corruption in Afghanistan. AMANAT’s activities resulted in structural changes to how government ministries function and deliver public services. Structural changes that lessen opportunities for corruption, paired with improved capacity of reform-minded civil servants, fosters an environment in which corruption can be minimized. AMANAT therefore contributed the broader effort to reduce corruption in Afghanistan. These contributions were made at the intermediate result level; the ability to connect AMANAT to outcome-level changes (i.e., quantified reduction in corruption) is discussed under “robust MEL, with room to grow”. The project directly resulted in five ministries each identifying several hundred vulnerabilities to corruption across their ministries, ministry activities, and supporting processes; these vulnerabilities were identified through the Vulnerability to Corruption Assessments (VCAs).3 AMANAT support also included direct assistance to each ministries’ internal audit directorates (IADs). This support served to align IAD policies, processes, and procedures with international standards, specifically the Institute of Internal Auditors. The output from this effort, the Internal Audit – Capability Model (IA-CM) report for each ministry “assessed gaps and deficiencies in internal audit capacity.”4 The IA-CM clearly defines areas for improvement and reforms, supplementing these areas with an action plan. Through implementing these reforms, the IADs will achieve a higher degree of functionality, enabling more effective anti-corruption activities. While this document serves as a valuable tool to the ministries in these pursuits, its utilization is potentially limited by inaccessibility. Respondents from three out of five ministries were unfamiliar with the IA-CM report, with one citing that it was not available in Dari or Pashto. AMANAT also provided a series of trainings aimed at increasing the capacity of the IADs, to bring them in line with international standards and rectify gaps identified through the IA-CM. Through improved capacity, IAD staff can make more effective contributions to anti-corruption activities, in turn lessening instances of corruption. Trainings were well regarded overall, with some issues of relevance (see “‘Factors of intervention success” below). Ministry respondents self-reported growth in knowledge and capacity. One ministry respondent stated that the trainings were particularly valuable for new auditors to introduce concepts and international standards. These trainings were provided to no less than 145 internal audit staff. 3 Partner ministries are the Ministry of Education, Ministry of Higher Education, Ministry of Martyrs and Disabled, Ministry of Public Health and Ministry of Refugees and Returnees. AMANAT also worked with the Independent Joint Anti-Corruption Monitoring and Evaluation Committee and Supreme Audit Office. Responses from the latter two are identified as ‘government official’ rather than ministry member. 4 Y2 Annual Report, page 2. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 10 During the evaluation period, AMANAT also conducted Business Process Redesign (BPR) activities with three partner ministries, with reforms implemented in accordance with a Corruption Risk Mitigation Plan (CRMP). BPR activities, facilitated through assistance to IADs, resulted in positive structural and process reform. These structural and process reforms will minimize opportunities for corruption within the IADs, while also improving the directorates’ effectiveness, minimizing corruption within ministries to enable more efficient public service delivery. Specifically, AMANAT aided partner ministries by, “developing and adopting effective audit processes that are consistent with international standards, IA [internal audit] charters, committee charters, codes of ethics, policies, procedures, tools, and checklists.”5 AMANAT also conducted foundational work for a series of activities that will be completed in the years following the evaluation period. These activities include reforming the Ministry of Public Health’s (MOPH) complaint handling, supporting implementation of an equitable land redistribution system for the Ministry of Refugees and Returnees (MORR), and developing the capacity of local school shuras. While the outcome of these activities is not determinable at this stage, preliminary results suggest that the activities are progressing through their program logic and achieving intermediate results which could enable outcome-level achievement. During the evaluation period, AMANAT initiated its pilot program for school shura capacity development.6 The school shura pilot demonstrated promising progress in initial stages, according to AMANAT and ministry staff, as well as direct beneficiaries (i.e., training recipients). One such beneficiary stated: Now we understand our role and responsibilities in managing complaints regarding schools and students’ issues. Most importantly, we learned that school shuras could play an active role in motivating parents to send their children to schools and if we notice any problems related to students, we can raise it with the school and or parents. We can mobilize local resources to solve our school problems instead of waiting for government to do it for us. Meeting partner expectations. AMANAT generally met ministry partner expectations at a program level, although reception at the activity level was inconsistent. Five out of five partner ministry respondents reported that overall, AMANAT met their expectations from the perspective of program design and program logic and were eager for AMANAT to expand and extend its assistance. However, four of these five reported that specific elements of the program fell short of expectations, particularly related to capacity building. Factors of intervention success. Intervention activities were considered successful by beneficiaries when they were perceived to be designed with the needs and capabilities of beneficiaries in mind. Respondents reported that successful interventions, such as those derived 5 Year 3 Annual Report, page 7. 6 Pilot activities during the evaluation period were oriented around standing-up the pilot and implementing capacity building sessions. Direct anti-corruption activities by beneficiaries did not occur during the evaluation period. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 11 from the line of activities supporting BPR, were successful because they directly addressed the ministry’s needs. One AMANAT short-term technical assistant (STTA) described this lesson: A fundamental lesson is any intervention should be based on a deep understanding of the context of ministries and aligned with strengths of government and that particular ministry – remove bottlenecks and make connections between existing capabilities and need for reform. However, ministry respondents reported that interventions were not fully successful when they did not meet criteria for relevance. Specifically, trainings were well regarded overall, but ministry partners report that elements are not fully relevant. Respondents from three out of five ministries found elements of the training not fully tailored to their needs. For example, one ministry respondent stated that their employees found the trainings rudimentary, while a respondent from a different ministry found the trainings too challenging for staff. Trainings were also not fully successful if not implemented with capable instructors. A set of trainings in critical thinking offered to the MEC were found lacking for this reason. One respondent from the MEC stated that, “the supplier did not manage engagement very well.” These concerns were echoed in AMANAT quarterly reporting to USAID. Aligned with national and institutional priorities. AMANAT is aligned with national and institutional anti-corruption priorities. Four out of five interviewed ministry respondents reported that AMANAT’s activities are aligned. This alignment is highly valued, as all ministry respondents recognize the importance of conducting anti-corruption work. As one respondent stated, “Corruption is very dangerous and problematic for [the] government of Afghanistan; it is like terrorism to the government…most of our problems, I believe, are because of corruption.” During the evaluation period, AMANAT worked closely with the MEC. At the close of the evaluation period, AMANAT had discontinued work with the MEC; and later, by early 2021, the MEC had largely ceased to function as an independent body.7 Nonetheless, MEC respondents found AMANAT’s assistance strongly valuable and directly contributing to the MEC’s institutional priority to address national anti-corruption priorities across ministries. Robust monitoring, evaluation, and learning (MEL), with room to grow. AMANAT employs a robust, multilevel MEL system directly and with its implementing partner, Equal Access International (EAI). This MEL system is guided by an Activity MEL Plan and revised indicators as needed. The MEL system incorporates data quality assessment(s), utilizes site visits as a data source, and is done in cooperation with activity stakeholders. The MEL system therefore meets USAID standards, as outlined by ADS 201.3.4.10. AMANAT also utilizes USAID collaborating, learning and adapting (CLA) principles to “promote more effective implementation of project activities and operational processes that will enable improved 7 MEC ceasing to function independently in 2021 occurred outside of the evaluation period but is mentioned here for context. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 12 project outcomes.”8 Findings derived from CLA exercises were essential in shaping program implementation over the evaluation period. The MEL system and CLA activities directly contributed to shaping the program and facilitating positive achievements discussed under EQ1 and EQ4. Decisions derived from CLA activities include but are not limited to the following: ● Identifying and selecting partner institutions; ● Developing proposed plans of action and related activities in collaboration with each selected institution; ● Revising and adapting the planned approaches throughout implementation; and, ● Modifying its grant process and related capacity-building efforts to better utilize this line of activities (see EQ4). While the MEL and CLA approaches are robust, AMANAT team members believe there is room for growth. Specifically, staff members with knowledge of MEL principles believe that indicators utilized during the evaluation period were too output-oriented, rather than outcome/impact-oriented. Consequently, AMANAT has limited ability to establish causal linkages between its activities and IR 3.1/IR 3.2 and DO3. Robust MEL capacity does not necessarily extend to the partner institutions. An AMANAT staff member and two ministry respondents reported that MEL capacity within ministries is largely oriented around audits. One ministry respondent believes AMANAT is not sufficiently supporting development of M&E capacity. AMANAT’s ability to establish causal linkages from ministry-reported outcomes to AMANAT objectives is therefore limited. Unrealized potential of CLIN0002. AMANAT did not utilize the CLIN0002 funding mechanism during the evaluation period. At the close of the evaluation period, however, a proposal for assistance to [Redacted] was under consideration by USAID. AMANAT staff report that an element of this underutilization was borne from unclearly defined ownership of the process. Specifically, this staff member reported that it was his/her initial understanding that CLIN0002 areas of assistance would come from USAID. However, when this did not materialize, the team shifted its understanding of CLIN0002 ideation to be fully AMANAT-led. Moreover, one potential CLIN0002 partner reported that the process to conceptualize, propose, and initiate activities under this line of assistance was lengthy and difficult. Internal growing pains. AMANAT faced initial growing pains when operationalizing the program. Representatives from three AMANAT departments each separately reported that their department faced understaffing issues which directly impeded their operations. One staff member reported that a key management position was unfilled for nearly six months. These staffing gaps are partially due to long periods to approve and engage staff members, according to two AMANAT staff members. 8 Y2 CLA Progress Report MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 13 Additionally, AMANAT faced difficulties with its financial management system. An AMANAT staff member and an AMANAT STTA separately reported that salary payments were occasionally delayed. This STTA reported that salary payments were delayed due to AMANAT accounts being frozen, although this specific assertion could not be triangulated. These payment difficulties have impacted the perception of AMANAT. One ministry respondent reported receiving complaints and requests for assistance from AMANAT staff members who contended with delayed salary payments. CONCLUSIONS Achievement. AMANAT has successfully contributed to an increased ability to implement anti-corruption reforms by strengthening the audit directorate structures through which reforms are implemented and providing concrete courses of action via VCA/IA-CM. Ministries have acted on this increased capacity by implementing suggested process reforms, which will manifest in positive downstream effects on public service delivery. Achievement. AMANAT conceived of and implemented meaningful activities through BPR/CRMP that foster direct impact/corruption mitigation by closing vulnerabilities to corruption and reforming process elements. The approach is largely sound and aligned with ministry capabilities and needs relevance enables interventions to work. Challenge. AMANAT’s activities are not perceived as comprehensive enough at this period to ensure full effectiveness and achievement of the program’s objectives; while intermediate results are being realized, at the midterm period of the program AMANAT has yet to realize its outcomes. Further effort is required to address remaining institutional capacity gaps over the remaining years of the project or any future anti-corruption initiatives. Challenge. AMANAT’s MEL system is ADS-compliant but is not sufficiently developed to concretely identify the chain of causality from inputs to outputs, to outcomes and goals. This limited ability to measure outcomes creates an attribution gap, limiting the potential to fully assess programmatic impact. EVALUATION QUESTION 2 – UNINTENDED INTERVENTION CONSEQUENCES What are the unintended consequences (positive and negative) of those interventions? How are the consequences, if any, affecting anti-corruption efforts in Afghanistan? FINDINGS Minimal reported unintended consequences. A total of four of five ministry respondents and four of seven grantee respondents did not report any unintended consequences, positive or negative. The sole ministry respondent did not expect the “start and stop” nature of the intervention; communication findings are detailed under EQ1. Unexpected consequences for grantees are detailed in the subsequent paragraphs. Uneven reception to grantees. AMANAT grantees’ actions had the unexpected effect of encouraging sub-national government entities to actively promote anti-corruption programming MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 14 in their administrative areas. Grantee activities necessitated extensive engagement with government actors across the project lifecycle — from permission to conduct activities, to having government officials serve as both beneficiaries and activity participants. In support of the grantees’ activities, sub-national government actors actively encouraged grantee anti-corruption work. Specifically, the Social Affairs Deputy of the [Redacted] Ministry of Justice publicly pledged to “fight effectively against corruption;”9 the [Redacted] government also requested that Women Integrity for Society Empowerment (WISE) provide further trainings to government employees. In support of the activities of the Organization for Research, Skills and Training for Women (ORSTW), the Deputy Governor of [Redacted] stated, “I really appreciate these kinds of useful anti-corruption efforts; these efforts are for the better improvement of transparency in society.”10 While some sub-national government entities were receptive to grantee activities, others were reportedly actively resistant and reflected negative attention (e.g., alleged interference) onto grantees. Specifically, grantee Faryad Radio and TV Network (FARYAD) reported considerable difficulty in securing the participation of government officials for their article production. This reluctance to participate resulted in delivery delays. Moreover, a representative from FARYAD stated in an interview, “The journalists have been threatened… [government directorate] has been trying to sabotage the media operations.” As one AMANAT IP succinctly stated in support, “If you’re seen to be doing things that disturb the current arrangement, you could be targeted for harassment.” Priming engagement with anti-corruption topics. Initial research efforts, such as the institutional Political Economy Analysis (iPEA), helped prime the ministries’ engagement with anti-corruption issues and topics. That is, the process of conducting preliminary or foundational research provided an avenue for a soft introduction to anti-corruption principles and topics. One AMANAT STTA observed that querying on a given topic naturally forces government respondents to begin thinking about the topic and their position relative to the topic. Consequently, the STTA interviewee believes that this process can foster active engagement with anti-corruption topics when it then comes time for AMANAT to provide direct assistance. Managing beneficiary expectations. Solicitation of secondary beneficiaries (i.e., grantee direct beneficiaries) contributed to instances of negative incentivization. An AMANAT staff member reported that grantees offered unequal incentivization to participate (e.g., subsidizing transportation costs) at select events, which contributed to an expectation that incentives would be provided for future events. 9 Y3Q2 Quarterly Report – Annex 3: Wise Final Report (pg. 14). Attributed to official, but not indicated whether verbatim quote. 10 Abbreviations for all grantees reflect the abbreviations used in AMANAT contracting/reporting and do not necessarily reflect the organization’s self-description. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 15 CONCLUSIONS Achievement. AMANAT's activities have had desirable second-order effects, particularly at the sub-national level, creating a positive feedback loop, and these second-order effects have contributed to improved anti-corruption efforts.11 Challenge. Resistance from sub-national actors, such as local government administrators, has created unanticipated challenges for grantee activities, potentially limiting grantees’ ability to maximize impact. EVALUATION QUESTION 3 – AMANAT’S PLACE WITHIN THE AFGHAN CONTEXT How and to what extent do AMANAT’s interventions fit in with the overall Afghan anti-corruption context as regards political uncertainty, lack of political will, insecurity, and the post-peace scenario? FINDINGS Novel position within the anti-corruption sector. AMANAT’s program elements constitute unique approaches to combatting corruption in Afghanistan. With the government￾oriented line of activities (IR 3.2), AMANAT focuses its efforts at the technical/working level, specifically within IADs. This is a unique approach for donor-funded programming, according to AMANAT staff and two government officials;12 one of these government respondents specifically could not recall his/her ministry ever receiving such assistance.13 Another government respondent considers AMANAT one of the largest programs with this focus area and that, consequently, AMANAT has a vital role to play in keeping attention to its focus areas. This approach of working at the IAD technical level fits the Afghan context well, as civil servants enjoy a degree of job security, with consistency in activities, where political appointees at the higher levels (e.g., Minister and Deputy Minister) are subject to turnover due to changes in political alignment/realities and national priorities. AMANAT assistance to CSOs through its grants program also constitutes a unique intervention. As one AMANAT staff member stated, “the program is very supportive for the CSOs, and before there was not such a platform to fight corruption through a grants program.” The grants program was recognized as unique for its role in prevention (e.g., advocacy, citizen monitoring and service-delivery reform). One grantee stated that the AMANAT intervention is “one of the few in memory” specifically oriented toward prevention, generally, with assistance to 11 A second order effect is the consequence of a consequence. An anti-corruption activity may produce a desirable result (e.g., identified instance of corruption), which in turn produces another desirable result (e.g., reform minded government actors encourage further identification of corruption), contributing to a positive feedback loop. 12 One such official was no longer employed at the government body at the time of the interview, but was employed there during the period of AMANAT assistance. 13 AMANAT is not the only donor-funded programming with beneficiaries in select IADs. The World Banks’ Capacity Building for Results (CBR) engages with IADs, including placing staff within the directorate. However, respondents do not recall CBR engaging at the technical level in a manner like AMANAT. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 16 CSOs. This unique role, and emphasis on prevention, facilitated grant activities which were novel approaches to addressing corruption concerns in the Afghan context – participatory accountability dialogues involving culturally-relevant stakeholders by ORSTW, for example.14 AMANAT training to CSOs, which may not necessarily be grantees, also introduced concepts that were unfamiliar to most participants (See EQ4 findings). Preparing for a post-peace environment. AMANAT staff members and government officials report that the ongoing peace negotiations are producing uncertainty, including an unwillingness to commit to long-term strategies or action plans. In the face of this uncertainty, according to a senior AMANAT staff member, the program is unable to plan for contingencies because post-peace decision-making is taking place at a higher level. Nonetheless, AMANAT is preparing for a variety of potential post-peace scenarios through its work at the technical level. On one hand, AMANAT is guided by a belief that structural reforms are resilient. Undoing AMANAT-supported reforms through the IADs would take active efforts of deconstruction, according to an AMANAT staff member. AMANAT is also building relationships at the technical level, with employees who may remain in place under a new administrative configuration. As this same staff member stated, “there are always technocrats.” This latter approach aligns with government officials’ perceptions of a post-peace environment. Four out of five ministry respondents believe that anti-corruption programming, such as AMANAT’s, will be necessary in a post-peace environment. Specifically, ministry respondents note that a post-peace scenario will dramatically increase the number of beneficiaries. For example, MORR will have to contend with land claims of returnees. This perceived need is also shared by grantee respondents. Four of seven grantee respondents wanted to work with AMANAT in a post-peace environment, on related topics. One grantee believes AMANAT can serve a crucial role in ensuring equitable distribution of resources. Nonetheless, AMANAT’s effectiveness in a post-peace environment depends on the administrative configuration, according to one former government official. --------------------------------------------------------------- --------------------------------------------------[Redacted]-------------------------------------------------------- ------------ Accounting for security context. AMANAT employs several measures to mitigate the impact of insecurity on programming. AMANAT engages a network of contractors and employees from the areas in which they live to ensure that activities and activity monitoring can continue in the areas that core project staff are unable to access. AMANAT does face challenges due to the prevailing security situation in Afghanistan and [Redacted], specifically. AMANAT staff members reported that security restrictions can cause coordination challenges. This challenge was echoed by two ministry officials. Partial resilience to political uncertainty. AMANAT is partially resilient against the impact of government restructuring due to its focus on working at the technical level. AMANAT 14 While awarded grantees were able to implement novel solutions fitting the Afghan context, the grant program as a whole faced difficulties identifying these novel and innovative solutions; see EQ4 “learning from initial challenges” for further information. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 17 works with civil servants who are partially immune to turnover amongst senior, more politically aligned staff. Furthermore, beneficiaries at the working level are reportedly highly engaged in their respective directorates’ activities and are reform minded; all ministry respondents reportedly value the type of work AMANAT was conducting. This also provides a degree of resilience against adverse political will, particularly that derived from political uncertainty. However, one ministry official reported that AMANAT does not substantially engage at the senior management or national level, to the extent necessary. This concern was also echoed by a grantee respondent and a former government official. Consequently, there are reportedly low levels of awareness of AMANAT within the ministries, even within cooperating directorates. To that end, AMANAT is vulnerable to having programmatic progress reversed due to decisions made above the level at which AMANAT is operating. The aforementioned ministry official provided the example of laws and procedures guiding IAD practice; if changes are made at a national level that run contrary to the procedures AMANAT assists in installing, the ministry will be obliged to deconstruct the AMANAT-guided systems. At the sub-national level, turnover of local administrators has negatively impacted grantees’ activities requiring temporary pauses to implementation. As one grantee relayed, changes at the sub-national level shift focal points and require reorientation and rebuilding relationships, which delays implementation and absorbs institutional energy. AMANAT staff echoed this concern. Limited intra-sector engagement. AMANAT engaged with other donors at the working level, specifically with the World Bank’s grievance redress mechanism (GRM) team on the mapping of complaint-handling mechanisms, as well as the World Bank EQRA team on the approach for training of school management shuras. However, according to two major donors in the anti-corruption space, AMANAT engagement at the program level is relatively limited. These donors reported awareness of AMANAT as a program but had limited awareness of specific AMANAT activities.15 This may contribute to duplication of efforts; one donor reported conducting capacity-building exercises with CSOs on analyzing the national budget, an activity AMANAT also conducted. CONCLUSIONS Achievement. AMANAT program design and structure is largely unique within the Afghan anti-corruption space. This contributes to AMANAT's ability to create novel impact on Afghan government and broader Afghan society. Achievement. AMANAT has largely taken the steps within its control to develop contingencies for a post-peace scenario, given the level at which post-peace decisions are made. Consequences of the post-peace government arrangement will flow down to the level at which AMANAT is working. 15 The evaluation team acknowledges that certain donor engagements (e.g., Anti-Corruption Stakeholders Working Group) may be outside of AMANAT’s area of responsibility. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 18 Challenge. AMANAT is not fully resilient against turnover, partially due to a lack of proactive action at non-working, national, and sub-national hierarchical levels to mitigate the impact of turnover at any given level. EVALUATION QUESTION 4 – IMPROVING CIVIL SOCIETY’S ANTI￾CORRUPTION EFFORTS To what extent are AMANAT’s interventions contributing to Civil Society’s ability to advocate for reform? AMANAT contributes to IR 3.1 “citizens hold the GoA accountable” through sub-IR 3.1.2, “Civil society’s ability to advocate for reform strengthened”. Sub-IR 3.1.2 includes one sub￾purpose (SP 2.1), “civil society capacity to assess and improve GoA anti-corruption efforts strengthened.” FINDINGS Leading reform efforts. AMANAT grantees’ activities resulted in observable advocacy for reform, and further, contributed to instances of reduced corruption at the sub-national level. AMANAT grantee activities took a multifaceted approach to combatting corruption, through building awareness among community members, conducting strategic communications campaigns, directly engaging government officials in advocacy efforts, reporting on instances of corruption, and several other efforts. Overall, grantees held no less than 130 trainings or workshops in support of their activities, reaching over 1,500 direct beneficiaries and 5,000 indirect beneficiaries. These efforts have directly impacted secondary beneficiaries (i.e., community members). The grantees conducted several activities that produced notable impacts. The following is a selection of the impacts from the grantees that had closed programming and submitted final narratives during the evaluation period. The Afghan Alternative Livelihood, Welfare and Development Organization (AALWDO) established and stood up joint monitoring bodies across [Redacted] through capacity building and community awareness workshops. This empowered community members and victims of corruption to seek redress through a safe, nongovernment advocacy body. For example, in one district, the joint monitoring body found that a local official was signing illegal contracts, disregarding procurement policy. As a result of the monitoring body’s advocacy, the district governor called for contract annulment and realignment in accordance with procurement policy. This was not the only example; an AALWDO respondent relayed the following anecdote: We do engage Mullahs, community elders, youths and other people…We had a positive result from the project, because it paves the way [for] Mullahs [to get] courage to talk about corruption sin during each Friday Mosque by loudspeaker and inform people to avoid corruption in society. ORSTW established anti-corruption coalitions in [Redacted] supported by accountability and transparency awareness sessions. They collected over 150 citizen complaints, particularly from MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 19 rural communities, and advocated for their resolution at the relevant sub-national level. For example, the anti-corruption coalition in [Redacted] District received a complaint about improper treatment of patients at a local clinic. This complaint was escalated to the District Governor, who sanctioned the clinic and requested that ORSTW ensure that the anti￾corruption coalition function permanently in the district. FARYAD trained 20 journalists in [Redacted], [Redacted], [Redacted], and [Redacted] in anti￾corruption investigative reporting, who in turn produced 100 investigative reports. These trainings increased journalists’ knowledge of and skills in investigative reporting, with 80 percent of trained journalists reporting improvement. According to a community survey conducted by FARYAD as part of their M&E activities, awareness of investigative reports among news consumers rose from 5 percent to 72 percent.16 Additionally, positive attitudes toward bribery decreased from 80 percent pre-campaign to 12 percent post-campaign.17 Further results are displayed in Figure 5. A beneficiary in [Redacted] produced a report on corrupt practices within the Provincial Customs Department; the report made its way to the governor’s desk, who ordered the arrest and investigation of the named officials. Figure 3. Results from FARYAD baseline and endline community surveys Women Integrity for Society Empowerment (WISE) conducted quasi-vulnerability to corruption assessments and impact analyses in 14 departments of the [Redacted] provincial government, paired with capacity building to empower local advocates. Impact analyses identified that, “the majority of local government entities implemented changes to their service delivery processes.” 16 FARYAD did not detail the methodology utilized in conducting the community surveys. Consequently, these results are taken at face value, without assessment of methodological rigor. 17 The specific question stated that “in order to get work done by an office in a timely manner, it is necessary to pay bribes – but that will not cause any corruption in the office.” MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 20 Overall, all seven interviewed grantees report that AMANAT made a perceived impact and was effective in its assistance to promote anti-corruption efforts. The grantees also stated that AMANAT’s assistance was successful and useful. These interventions were also found useful by the secondary beneficiaries (i.e., grantee beneficiaries). As one beneficiary stated: After the training provided, with some other participants [I] took part in advocacy here in the province for anti-corruption within the public institutions that enabled us to better monitor and advocate for reform in the public service provision…These training sessions supported me and fulfilled my expectations to better know what corruption is and how to advocate in society about injustices. Overall, all six interviewed secondary beneficiaries were satisfied with the activities they participated in, finding them useful in increasing their capacities. Building foundations for future advocacy. AMANAT CSO trainings have produced evidence of increased knowledge and capacity in multiple topics, such as citizen monitoring, that facilitate advocacy activities. AMANAT implemented approximately 18 CSO-focused advocacy trainings (including training for trainers [ToTs]) during the evaluation period; pre/post-test results were not available for every training.18 Training sessions on watchdog functions, in one instance, resulted in a 34-percentage point increase, from 55 percent to 89 percent, in related knowledge, according to AMANAT reporting (see Figure 6). Training sessions on the legislative process, in one instance, resulted in a 46-percentage point increase, from 21 percent to 67 percent, according to this same report.19 Figure 4. Pre/post test results from select CSO capacity-building trainings 18 AMANAT conducted several trainings in support of the grants program during the evaluation period. Findings related to this set of trainings are discussed under EQ5: Improving self-reliance. 19 Y2Q2 quarterly report MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 21 Mobilizing civil society. AMANAT has encouraged engagement and enthusiasm to participate in anti-corruption activities from many CSOs, creating a robust base for future activities. According to an AMANAT staff member, the program has created a database of more than 400 CSOs to engage in future programming activities. At least 50 CSOs have been directly engaged in AMANAT’s activities.20 While AMANAT has engaged several CSOs on a program to link CSOs, engagement between CSOs has been limited. AMANAT’s initial efforts to create the CSO Anti-Corruption Coordination Center (CSCC) were abandoned for reasons outside the scope of this evaluation. However, multiple AMANAT staff members, as well as grantee respondents, found this effort valuable. AMANAT grantees would like to improve coordination and establish linkages between them. As one grantee stated, “To be frank, AMANAT is very conservative in [coordination] regard. We don’t know other partners.…” In the absence of AMANAT-driven coordination efforts, some grantees are building connections themselves. One grantee in [Redacted] province reported that other CSOs in the province saw AMANAT’s activities and were encouraged to mobilize and engage with AMANAT for future programming. This mobilization of civil society did not necessarily extend to fostering state–citizen interactions. No government official respondents reported AMANAT-driven coordination with CSOs, although one official explicitly stated a desire to increase such coordination. Aligned with organizational priorities. All seven interviewed grantees reported that AMANAT aligns with their organizational and broader sectoral anti-corruption activities. One grantee reported that it had previously worked tangentially on anti-corruption programming but because of its accomplishments with AMANAT, it has formally incorporated “anti￾corruption” into its organizational strategy. Learning from initial challenges. AMANAT faced several challenges in standing up activities in support of IR 3.1. AMANAT initially overestimated CSO capacity, their subject matter knowledge and ability to implement. Multiple AMANAT staff respondents reported that the program overestimated CSOs’ general familiarity with and ability to grasp specific concepts related to anti-corruption efforts, such as budget watchdog functions and legislation analysis. This necessitated AMANAT to adjust programming, introducing more foundational training to its curriculum within each topical focus. The grants program also faced challenges, partially resulting in AMANAT disbursing two percent of initially (i.e., under first version/unmodified contract) earmarked funds during its first two years of programming.21 AMANAT overestimated CSO capacity to implement activities that met programmatic and donor standards for impact and innovation. Consequently, approximately four percent of grant applications have been accepted. The grant process was 20 This figure does not include unsuccessful offerors for the grant program. 21 [Redacted] MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 22 stymied in its early stages by a complex and lengthy multistage application process that resulted in a solicitation cycle that could run over 12 months from application to contract ratification. AMANAT has taken multiple steps to address these challenges to grant applications, working through multiple revisions of its annual program statement (APS). AMANAT simplified the grant application process from multiple steps to a single step. The process was also digitized through an in-house application portal. AMANAT increased utilization of the RFA mechanism to improve efficiency and solicit more appropriate, purpose-fit applications. Capacity building in support of the grant process also incorporated an additional training module aimed at improving the relevance of applications through developing more anti-corruption solution￾focused applications. Finally, AMANAT increased the period of performance and award amount, and allowed for current awardees to fast-track program expansions. Concerns about transparency. The AMANAT project does not have formal external transparency mechanisms for the grant activity, for example, published grant application/acceptance rates or diversity and inclusion statistics for awardees.22 The grant timeline is also opaque (see above, “Learning from initial challenges”). Multiple CSO respondents voiced concerns about this lack of transparency. These concerns reflect the opaque nature of the procurement process – unverifiable innuendo about favoritism or preferential treatment borne from a lack of clarity on which organizations were awarded grants. AMANAT staff also noted select CSOs’ perception that it lacks sufficient transparency. CONCLUSIONS Achievement. When mobilized, grantees are an effective avenue to generate impact. The grantee program has produced concrete, observable progress against corruption in Afghanistan.23 Achievement. AMANAT trainings were effective in reaching intended audiences. Clear achievement was demonstrated through internal M&E efforts, verified through beneficiary reporting. Achievement. AMANAT has made notable effort to overcome the initial challenges the grant program faced, which limited spending and potential impact in the project's early stages. EVALUATION QUESTION 5 – SUSTAINING AMANAT ACHIEVEMENTS To what extent are AMANAT’s interventions sustainable for the Afghan government and for CSOs? 22 The evaluation team acknowledges that due to the nature of anti-corruption work and its context in Afghanistan, real limitations exist for external transparency mechanisms. 23 Progress is qualitatively assessed through self-reported instances of corruption identified, which are then corrected or resolved by relevant sub-national government actors. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 23 FINDINGS Laying groundwork for sustainability. AMANAT activities during this period have laid the groundwork for sustainability through installing resilient structures and systems in partner ministries. Specifically, the reforms made through the VCA/BPR line of activities discussed under EQ1—such as reforming recruitment strategies, establishing conflict of interest principles and strengthening complaints handling—can be sustained at the close of AMANAT. AMANAT is guided by a belief in the resilience of structures, as undoing AMANAT-driven reforms would require active acts of deconstruction. As a senior AMANAT staff member stated, “You create structures, and they remain.” Partner ministries support AMANAT's approach to strengthening internal systems to ensure sustainability and generally share the belief in the resilience of structures and systems. Three out of five interviewed ministry respondents believe that process-oriented reforms completed with AMANAT assistance will remain after the close of AMANAT support. As one government official stated, “When we make a system, our promise is sustainability of that system.” Another government official stated: Yes, everything that was done, can be left unchanged. In the case AMANAT stops support, the government system will still remain the same. This groundwork is not sufficient to ensure sustainability, however. Ministries do not necessarily have the resources — human resources, funding, political capital, etc. — to maintain and continue to improve reform elements completed with AMANAT support. As one senior AMANAT staff member stated: We are developing a comprehensive model for the IA-CM system, but the public institutions do not have the resources to run the system after one year. We should tie the program based on resources the institution has. Sustainability is also related to the availability of capable, reform-minded civil servants for positions within the directorates. One challenge to ensuring this talent pipeline is the lack of a dedicated concentration and department teaching auditing at the national universities. Without this department, the IADs will be at least partially dependent on external assistance in the near to midterm future. This notion of AMANAT laying the groundwork for sustainability also extends to capacity building. As discussed under EQ1, ministry partners value trainings. However, they are divided in their perceptions of the trainings’ sustainability. One government official believes his/her ministry employees retained the knowledge from the trainings and continue to utilize these gains; one former official shares this belief about his/her institution’s employees. However, respondents from two out of five partner ministries believe the trainings were insufficient to, as one respondent stated, “contribute to a change in mindset.” AMANAT is limited in its ability to fully assess whether results are sustained. Specifically, AMANAT does not have a dedicated mechanism to determine whether results are sustained — measured through sustainability-oriented indicators, GIRoA-led sustainability assessments, or MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 24 reporting, etc. Rather, sustainability monitoring is currently largely done on an informal basis. One senior AMANAT staff member states that the program is limited in this respect by access and ability to observe, for example, assessing the extent to which knowledge from capacity￾building situations is utilized in real-world environments. Improving self-reliance.24 Grantees report improved effectiveness and operational efficiency, as a result of knowledge gained during the grantee period. Specifically, grantees highlight the trainings in proposal development, budgeting, and related processes that were extended to potential offerors as valuable. As one grantee stated, “As a result of AMANAT grant implementation, we have a team of trained staff who have received training in anti-corruption [and] budget making, and our policy review processes increased from once at all to every six months.” Moreover, AMANAT grantees and other CSOs can sustain learnings from subject matter and proposal development trainings. AMANAT training materials are accessible to CSOs beyond the life of the project due to their self-guided nature and availability in local languages. Grantees have utilized this increased capacity to improve their self-reliance. Specifically, multiple grantees report that they are developing or have developed proposals to continue activities established under AMANAT. One grantee stated that activities conducted under AMANAT and the reporting of these activities serve as useful “case studies” to present to potential funders. While CSOs are on the path to self-reliance, it has not been fully realized at this time. Grantees find it necessary to solicit funding from donors outside of AMANAT; most grantees lack core funding to fully sustain AMANAT activities absent AMANAT funding. Only three out of seven grantees believe that results realized under AMANAT will be fully sustained. At the secondary beneficiary level, four of six interviewed beneficiaries stated that they continued their advocacy efforts to some degree following the close of their respective grants. However, these respondents implied that further support would be necessary to ensure continued activities. CONCLUSIONS Challenge. AMANAT success in creating the groundwork for sustainable change through increased ability to implement reforms is potentially limited by partner ministries’ ability to independently maintain IAD systems. Challenge. AMANAT has aided CSOs in achieving partial sustainability of results, with some evidence that sustainability is manifesting; however, currently these CSOs lack the capacity to fully sustain results. 24 Per ADS 201.3.4.3, the ‘self-reliance vision’ is borne through “increasing local ownership or “commitment” and strengthening the “capacity” of local actors to lead their own development.” MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 25 GENDER FOCUS The evaluation incorporated a gender focus that cut across evaluation questions. This section highlights the findings and conclusions from this line of investigation. FINDINGS Unintended mainstreaming. Training sessions resulted in unintentional gender mainstreaming, as beneficiaries brought children to sessions and added childcare concerns into discussions. AMANAT’s facilitation of this unintended mainstreaming, in turn, improved conditions for women’s meaningful participation in program activities. Partial achievement of gender-related indicator targets. The AMANAT program did not meet all of its gender-related indicator targets during annual reporting periods. For Year 1, indicators were not fully developed or relevant. For Year 2, AMANAT met indicators. 1.1.1 and 2.1.4. It did not meet indicator 2.1.1. Indicator 2.1 was not met, as measurement was scheduled to be introduced during Year 3. The table below presents targets and results. Table 1. Gender-related performance indicators for Year 2 Indicator Y2 Target Y2 Achievement 1.1.1 Number of women participating in technical and professional positions as trainees and monitors in beneficiary institutions. +5% over baseline (8) +62.5% over baseline (13) 2.1 Proportion of target population reporting increased agreement with the concept that males and females should have equal access to social, economic and political opportunities. +5% over baseline (no baseline) 025 2.1.1 Proportion of females who report increased self-efficacy at the conclusion of USG-supported programming. 75% 74% 2.1.4 Number of training and capacity-building activities conducted with USG assistance that are designed to promote the participation of women or integration of gender perspectives in anti-corruption activities. 5 23 Uneven focus on gender equality and women’s empowerment. Select grantees, such as ORSTW and WISE have an explicit gender-oriented focus in their programming. The organizations ensure that women are well represented, including over-represented in activities relative to the population. Activities are designed and led by women, with gender mainstreaming serving as a core element of the activity philosophy. Other grantees, such as 25 Not assessed in Year 2; scheduled to be introduced through Year 3. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 26 FARYAD, do not explicitly incorporate equality and empowerment in activities, attributing their achievements solely to having women beneficiaries. Perceived need for improvement in grantee support. A focus group of women-led CSOs were eager to work with AMANAT, through receiving training or the grant program. However, they reportedly felt unsupported in this manner. The respondents, who represented organizations that had participated in at least one AMANAT-led workshop or networking session, reported limited further interaction with AMANAT. Additionally, respondents who submitted proposals, one of which reportedly made it past the concept stage, said that the process was opaque, with minimal to no feedback. As one respondent stated upon being contacted, “As we had applied to AMANAT for an anti-corruption project, we [did not] have any response from their side; [so] it is funny how AMANAT has our organization contact in their contact list.” The respondents recognize AMANAT’s value. One respondent reported awareness of an AMANAT grantee’s activities and stated, “Therefore, [the] AMANAT project and training was good through [this] CSO to conduct their project, [such] that people should know about the behavior of officials in organizations.” Low awareness of the grant process and requirements, along with low awareness of gender￾specific RFAs, may be inhibiting deeper engagement from these CSOs. Nonetheless, each respondent reported a willingness and desire to work with AMANAT, in one capacity or another. As one respondent stated: If I were [to get an] AMANAT project, it would be very useful, because in the place I work most of the key community members, leaders, and high-ranking government [officials] misuse projects and cheat the poor people. People really need to know about their rights, and they really need to raise their voice [for] anti-corruption. CONCLUSIONS Achievement. AMANAT exceeded performance on two of three assessed gender-related performance indicator targets for Year 2. Challenge. AMANAT is progressing in its objective to improve gender mainstreaming; however, the program falls short of fully meeting the needs of women beneficiaries and secondary beneficiaries. RECOMMENDATIONS This section details the recommendations derived from each evaluation question’s findings and conclusions. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 27 EVALUATION QUESTION 1 – REDUCING CORRUPTION IN PUBLIC SERVICE DELIVERY 1. Strengthen technical assistance and support to Business Process Redesign (BPR) through increased capacity building, production of and revision to Corruption Risk Mitigation Plans (CRMPs), and technical assistance to CRMP implementation. a. Recommendation derived from findings nested under keyword “demonstrable contribution to anti-corruption.” Support through BPR/CRMP resulted in concrete and observable reform efforts. Moreover, the ministry partners involved in these activities recognize their value and encourage their continuation. AMANAT is making a novel impact through this line of activities, and results indicate that sustained and strengthened efforts would result in further progress against corruption. 2. Improve Internal Audit-Capability Model (IA-CM) report accessibility to ministry partners through translating into local languages, increasing dissemination efforts, and encouraging engagement with updated action plans. a. Recommendation derived from findings nested under keyword “demonstrable contribution to anti-corruption.” The IA-CM report is limited as an institutional guide by limited accessibility. The report has the potential to be a foundational piece of work for partner ministries, but further effort through improving accessibility and raising awareness of the report’s utility is required. 3. Scale up ministry monitoring, evaluation, and learning (MEL) capacities through expanding support to related directorates and sub-directorates, while also improving internal capacities to ensure full attribution of activities to outcomes and objectives. a. Recommendation derived from findings nested under keyword “robust MEL, with room to grow.” Ministry partners have limited capacity to implement robust MEL activities. This inhibits partners’ abilities to assess their contributions to anti-corruption efforts. Moreover, AMANAT MEL is limited in its ability to attribute its own activities to higher-level impact at the outcome/objective level. 4. Assess avenues for assistance to and cooperation with newly established GIRoA commissions (e.g., Access to Information Commission or Independent Joint Anti￾Corruption Monitoring and Evaluation Committee (MEC)-successor) that support anti￾corruption activities across the ministries. a. Recommendation derived from findings nested under keyword “aligned with national and institutional priorities,” with tangential relationship to EQ3 “preparing for a post-peace environment.” AMANAT succeeded in its support to MEC, assisting MEC’s work that cuts across ministries. AMANAT would do well to identify related anti-corruption bodies or commissions that work across the government, allowing AMANAT to indirectly assist ministries it does not directly partner with. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 28 EVALUATION QUESTION 2 – UNINTENDED INTERVENTION CONSEQUENCES 1. Commit to further relationship building with government actors at the sub-national level to increase grantees’ ability to mitigate adverse political will. a. Recommendation derived from findings nested under keyword “uneven reception to grantees.” AMANAT grantees’ activities are delayed and impeded by sub-national government actors. AMANAT could further leverage its relationships with government actors to mitigate these impediments. 2. Further emphasize risk mitigation and safety/security in grantee activity plans, to decrease potential harm stemming from negative repercussions from anti-corruption activities. a. Recommendation derived from findings nested under keyword “uneven reception to grantees.” Civil society broadly, and anti-corruption practitioners specifically, are exposed to potential harm on account of their activities. AMANAT should look to incorporate risk mitigation and safety/security planning into its assistance, for example, including capacity building for risk mitigation for anti-corruption practitioners as a distinct line of activities. 3. Standardize partner/grantee incentive schemes for secondary beneficiaries. a. Recommendation derived from findings nested under keyword “managing beneficiary expectations.” AMANAT should look to standardize incentive schemes both for direct activities and across grantees to ensure that AMANAT programming does not involuntarily contribute to negative incentives. EVALUATION QUESTION 3 – AMANAT’S PLACE WITHIN THE AFGHAN CONTEXT 1. Coordinate with donors and related implementers to develop shared understanding of post-peace scenario implications, particularly with respect to GIRoA anti-corruption commissions’ activities. a. Recommendation derived from findings nested under keyword “preparing for a post-peace environment.” The prospective post-peace environment has unanswered questions on how anti-corruption activities will take shape. AMANAT will be forced to contend with a new political paradigm, which will potentially reform how anti-corruption bodies behave. AMANAT could look to coordinating its approach and engagement strategy for these new bodies with donors and other implementers. A coordinated engagement strategy could reflect shared best practices, knowledge and lessons learned. 2. Conduct sustained relationship building at the top ministry and national executive levels to ensure resilience to government turnover and restructuring, while opening avenues for increased cooperation. Conduct these efforts in accordance with an institutional engagement strategy. a. Recommendation derived from findings nested under keyword “partial resilience to political uncertainty.” AMANAT’s efforts at the technical/working level are MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 29 not sufficient to fully embed AMANAT in the ministries’ and national government’s consciousness. AMANAT is at risk of having its activities invalidated, in part due to a lack of awareness of AMANAT at different hierarchical levels within the GIRoA. EVALUATION QUESTION 4 – IMPROVING CIVIL SOCIETY’S ANTI￾CORRUPTION EFFORTS 1. Focus on two avenues of CSO coordination efforts: a) strengthening linkages within Afghan civil society to mobilize member networks, and b) facilitating relationships and partnerships between the state and civil society. a. Recommendation derived from findings nested under keyword “mobilizing civil society.” While AMANAT has succeeded in establishing linkages between the project and a given CSO, linkages facilitated by AMANAT are minimal among CSOs and between CSOs and government. 2. Scale up training to improve civil society watchdog functions, legislation analysis, social auditing, and strategic communications, while ensuring that trainings are responsive to CSO needs and capacities. a. Recommendation derived from findings nested under keyword “building foundations for future advocacy.” AMANAT training efforts have produced demonstrated gains in participant competency. As capacity increases, training should further align with this increased level of knowledge. 3. Continue coordination efforts with USAID to identify bottlenecks in the grant procurement lifecycle to facilitate the expansion of awards to current and new grantees. a. Recommendation derived from findings nested under keyword “learning from initial challenges.” The grant program has produced demonstrated impact on anti-corruption (see EQ4 “Leading reform efforts”). Potential impact during the project period was limited, however, by the considerable effort in onboarding grantees, in part, because of process inefficiency. 4. Develop and install an external transparency mechanism. a. Recommendation derived from findings nested under keyword “concerns about transparency.” AMANAT can improve transparency of its procurement process. For example, publishing diversity and inclusion statistics about grantees can prevent perceptions that grants are not awarded to minority groups. EVALUATION QUESTION 5 – SUSTAINING AMANAT ACHIEVEMENTS 1. Strengthen GIRoA partner training programs through increasing relevant, long-term training engagements to ensure that trainings result in sustained behavior change. a. Recommendation derived from findings nested under keyword “laying groundwork for sustainability.” AMANAT training programs result in demonstrated improved capacity amongst participants. However, sustainable behavior change is not necessarily produced through single sessions. This issue is MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | 30 exacerbated when trainings are not sufficiently relevant to respondents or tailored to their capacities. 2. Conduct a feasibility study to support efforts of the Ministry of Higher Education (MOHE) to establish an internal audit curriculum and department at [Redacted] University. a. Recommendation derived from findings nested under keyword “laying groundwork for sustainability.” The MOHE has a strong desire to create, as one MOHE respondent stated, a “Center for Audit Learning” at [Redacted] University. While establishing such a center is outside of AMANAT’s scope, AMANAT can leverage its strong research capabilities and expertise to assist in scoping. 3. Identify potential alternative funding mechanisms for the grant program, such as core funding instead of activity funding alone, to promote long-term sustainability. a. Recommendation derived from findings nested under keyword “improving self￾reliance.” At the close of the evaluation period, AMANAT had identified multiple grantees for repeat programming. Continued demonstrated success on the activity or project level could indicate suitability for programming that looks beyond the AMANAT project life. AD HOC RECOMMENDATION: CLIN0002 1) Identify further opportunities to utilize CLIN0002 funding and ensure that opportunities are not missed once they are identified. a. Recommendation derived from findings nested under evaluation question 1 keyword “unrealized potential of CLIN0002.” There are expanded possibilities under the innovative and dynamic feature of the AMANAT project called “special programming mechanism,” that allows USAID and the project to react quickly to changing circumstances through special task orders for activities that could not have been anticipated at the time AMANAT commenced operations. AMANAT has thus far submitted two possibilities to USAID for consideration, but at the close of the evaluation period, had yet to finalize agreements for programming under CLIN0002. Another idea, which was mentioned during KIIs with officials from partner ministries and favorably received, is that USAID/Afghanistan may commission AMANAT under CLIN0002 to design workshops for individual partner ministries to inform the GIRoA about best practices and lessons learned from successful USAID-supported anti-corruption efforts in other countries. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-1 ANNEXES ANNEX A – EVALUATION SCOPE OF WORK OFFICE OF DEMOCRACY AND GOVERNANCE (ODG) & OFFICE OF PROGRAM AND PROJECT DEVELOPMENT (OPPD) STATEMENT OF WORK MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) I. PROGRAM INFORMATION Program/Project Name: Afghanistan’s Measure for Accountability and Transparency (AMANAT) Contractor: Management Systems International Inc. (MSI) Contract #: AID-306-17-H-00003 Total Estimated Cost: [Redacted] Life of Program/Project: August 23, 2017 – August 22, 2022 Active Provinces: [Redacted] Mission Development Objective (DO): DO3: Government Accountability to Citizens Increased. Linkage to Standard Program Structure (SPS): Democracy, Human Rights and Governance (DR) Required? One evaluation per project II. INTRODUCTION The USAID/Afghanistan Mission awarded Management Systems International Inc. (MSI) the Afghanistan’s Measure for Accountability and Transparency (AMANAT) contract for five years (August 23, 2017 – August 22, 2022) to support the Afghan government’s efforts to reduce and prevent corruption in public services. The project is the Mission’s primary anti-corruption activity that focuses on increasing transparency and accountability within institutions and systems of the Government of the Islamic Republic of Afghanistan (GIRoA) and on strengthening the watchdog MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-2 capacity of Afghanistan’s civil society organizations (CSOs) in public service delivery monitoring and anti-corruption advocacy campaigns. USAID/Afghanistan’s ODG wishes to conduct an external mid-term performance evaluation of AMANAT to assess the progress and achievements made towards the intended results, challenges/constraints preventing the project from achieving its expected results, and the way forward to improve the project performance in the remaining period. III. BACKGROUND Afghanistan remains one of the most corrupt countries in the world. Almost two decades after the fall of the Taliban, the Afghan government continues to struggle to provide reliable public services to its citizens. This failure poses a serious threat to Afghanistan state building and development as well as the legitimacy of the Afghan government. Recent studies show that corruption in Afghanistan is still serious and widespread throughout government institutions. The 2019 Corruption Perception Index, of Transparency International, ranks Afghanistan 173 out of the total 180 countries surveyed, with an overall score of just 16 out of 100. The 2018 national corruption survey by Integrity Watch Afghanistan finds that corruption remains central to Afghanistan’s challenges. According to the survey, the vast majority of Afghans (83 percent) believed that corruption had negatively affected the life of people in their local areas, and almost three quarters of those surveyed did not trust public services because of corruption. The 2019 Survey of the Afghan People (SAP) found that 81.5 percent of Afghans view corruption as a major problem for Afghanistan. In the same survey, a record 67.9 percent of Afghans say corruption is a “major problem” in their daily life lower than the 70.6 percent in 2018. The Afghan government has long regarded tackling corruption as one of its top priorities. Dating back to 2014, the National Unity Government of Afghanistan (NUG) announced its commitment to fight corruption through the Realizing Self-Reliance: Commitments to Reforms and Renewed Partnerships policy framework paper at the London Conference on Afghanistan in December 2014. This policy framework paper outlined strategic responses to tackle the underlying drivers of corruption identified by the Afghan government. The Government of Afghanistan has also promised to institute better governance by increasing government accountability, performance, and ability to deliver results in key areas such as judicial and public administration reform. The recent Geneva Conference on Afghanistan November 28, 2019, co-hosted by the Government of Afghanistan and the United Nations, noted corruption as one of the most serious challenges, besides insecurity and poverty, that require a renewed focus and commitment by the Afghan government and the international donor community. The National Unity government developed an anti-corruption strategy for fighting corruption in 2017. The strategy provides the conceptual framework for assessing corruption, describes the overall approaches to striking the most effective balance possible between prevention and punitive action, and summarizes the government’s action program, drawing on the diagnostic to set priorities and propose a sequence of interventions26. 26 Afghanistan National Strategy for Combating Corruption Strategy October 2017 MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-3 On December 25, 2018, the Afghan government approved a revised anti-corruption strategy that contained 103 benchmarks with completion deadlines between December 2016 and June 2020. The Afghan government updated the strategy again in February 2019 to include 102 of the 103 benchmarks from the December 2018 version. SIGAR determined that the Afghan government met 57 of 76 benchmarks, or about 75 percent, due by June 2019. In November 2018, the President of Afghanistan attended the Geneva Conference on Afghanistan and agreed to the “Geneva Mutual Accountability Framework” (GMAF). This framework includes four commitments from the Afghan government to (1) formally approve new indicators for the 2017 anti-corruption strategy; (2) increase prosecutions for anti-corruption cases; (3) verify government official’s asset declarations; and (4) improve citizens’ access to Afghan government information.27 Recognizing the urgency to act to restore trust in existing anti-corruption efforts and institutions that can effectively implement Government programs, the Afghan Government is accelerating the pace of reform through a mutually agreed upon set of benchmarks with the international community. Afghanistan Anti-Corruption Reform Accelerating Plan (RAP) is an integral part of the National efforts to combat corruption with specific benchmarks and timelines during the fiscal year 2020. The benchmarks and timelines are subject to revision around the middle of the year to ensure they are relevant to restore trust with Afghan citizens and the international community.28 USAID has been providing anti-corruption assistance to the Afghan government and CSOs for about a decade. The past interventions include USAID’s Assistance for Afghanistan Anti￾Corruption Authority (4A) Activity, which supported training the Corruption Prevention Directorate of the High Office of Oversight and Anti-Corruption (HOOAC). Following the 4A conclusion, USAID supported the implementation of the Advancing Effective Reforms for Civic Accountability (AERCA) that provided technical assistance and financial support to government entities to plan and implement reforms recommended by Vulnerability to Corruption Assessments (VCA)s. AERCA also supported watchdog efforts by CSOs to test and monitor the implementation of those reforms and to advocate their implementation in instances where target government entities fail to follow through on implementation of reforms. USAID support to the Independent Joint Anti-Corruption Monitoring and Evaluation Committee (MEC) to conduct VCAs ended in December 2019. Building on the past interventions mentioned above, USAID/Afghanistan designed the AMANAT program to further support the Afghan government to reduce and prevent corruption in its public services. AMANAT, which also means “entrusted” in Dari and Pashto, also builds on the previous 27 Geneva Conference on Afghanistan, “Geneva Mutual Accountability Framework,” Geneva, Switzerland, November 27–28, 2018. 28 Afghanistan Anti-Corruption Reform Accelerating Plan, October 2019 MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-4 Delegated Cooperation Agreement (DCAR) between the United Kingdom’s Department for International Development (DFID) and USAID, which provided technical assistance to the MEC to become an independent anti-corruption organization. AMANAT employs a three-pronged approach to support anti-corruption in Afghanistan: strengthening Afghan government and non￾government institutions to identify vulnerabilities to corruption and develop recommendations to address the vulnerabilities; supporting the Afghan government to plan and implement reforms based on the recommendations; and strengthening local CSOs to monitor reform implementation. The AMANAT project hypothesizes on the following theory of change: ● If Afghan government officials demonstrate sufficient commitment to decrease corruption in government services; and ● If Afghan government officials are provided with the appropriate technical and financial assistance to develop and implement strategies to address and reduce opportunities for corruption in accessing public services; and ● If Afghan non-governmental institutions can develop a sufficient level of technical expertise and institutional capacity to identify vulnerabilities to corruption, to develop workable recommendations to address these vulnerabilities, to access resources to implement the recommendations to address identified vulnerabilities, and to monitor the implementation of these recommendations; ● Then there will be a corresponding decrease in opportunities for corruption and government services will be provided more effectively and efficiently. ● And there will be a resulting increase in public confidence in the performance and legitimacy of the Afghan government. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-5 AMANAT Theory of Change Diagram AMANAT contributes to the Mission’s third development objective of further legitimizing the government by, among other initiatives, enhancing its performance in delivering services to Afghan citizens. It is designed to achieve a dual goal of improving governance practices at all levels of the system and having citizens hold the GIRoA accountable. MSI understands that holding governments accountable includes holding its officials accountable for corrupt business practices. AMANAT specifically serves two distinct but linked purposes: 1. Transparency and accountability mechanism strengthened (Purpose 1). 2. Civil society’s ability to advocate reform strengthened (Purpose 2). IV. PROGRAM GOALS AND OBJECTIVES AMANAT’s goal is to reduce and prevent corruption in public services to strengthen the credibility and legitimacy of the Afghan government and improve service delivery for the Afghan people. This goal is supported by three main objectives: 1) Strengthen Afghan non-governmental and governmental institutions in identifying vulnerabilities to corruption in the delivery of public services and developing corresponding recommendations to address these vulnerabilities. 2) Support select Afghan government institutions with technical assistance to plan for and implement procedural reforms based on those recommendations; and 3) Strengthen local CSOs capacity to test and monitor the effectiveness of those reforms and to advocate for their implementation where necessary. The AMANAT Activity is aligned with the USAID/Afghanistan Mission’s Results Framework Development Objective (DO) 3, of the current Country Development Cooperation Strategy (CDCS) 2019-2023. The DO seeks to increase Afghan government accountability to citizens, by increasing citizen participation in decision making processes, improving the effectiveness of the Afghan government, and reducing vulnerabilities to corruption. Interventions carried out under MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-6 the AMANAT project also contribute to the fulfillment of the following corresponding Intermediate Results (IR) and Sub-Intermediate Results (Sub-IR): ● IR 3.1: Citizen Participation in the Decision-Making Processes Improved o Sub-IR 3.1.3: Civil Society Advocacy for Reform Strengthened ● IR 3.2: Improved Effectiveness of Government Institutions o Sub-IR 3.2.1: Government Administrations and Municipalities Better Respond to Citizen Needs ● IR 3.3: Reduced Vulnerability to Corruption o Sub-IR 3.3.1: Implementation of Laws and Regulations that Impede Corruption Advanced o Sub-IR 3.3.2: Effective Legal Enforcement of Formal and Informal Laws Increased o Sub-IR 3.3.3: Independent Monitoring of Government Functions (CSOs and Independent Anti-Corruption Agencies) Improved. AMANAT’s Result Framework IRs and Sub-IRs also correspond to the following purposes and sub-purposes: Goal: Citizens Hold the Government of Islamic Republic of Afghanistan (GIRoA) Accountable and Governance at the National and Sub-National Level Strengthened Purpose 1: Transparency and Accountability Mechanisms Strengthened ● Sub-purpose 1.1: GIRoA-led initiatives to identify vulnerabilities to corruption enhanced ● Sub-purpose 1.2: Capacity of GIRoA institutions to implement anti-corruption reforms strengthened Purpose 2: Civil Society’s Ability to Advocate for Reform Strengthened ● Sub-purpose 2.1: Civil society capabilities to assess and improve GIRoA anti-corruption efforts strengthened To achieve the goal, purposes, and sub-purposes above, AMANAT implements key activities such as the following: ● Assist government institutions to conduct facilitated self-assessments to identify corruption risks ● Strengthen internal audit functions in selected government institutions ● Strengthen the MEC’s capacity to conduct VCAs and other initiatives29 ● Mitigate corruption risks using Business Process Redesign (BPR) and develop Corruption Risk Mitigation Plans (CRMPs) ● Implement corruption risk mitigation plans ● Identify and provide technical assistance at the sub-national level ● Mobilize civil society groups in [Redacted] and the provinces working on anti-corruption 29 USAID has supported MEC since 2015, our support to MEC ended in December 2019. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-7 ● Train CSOs on specific subjects/sectors so they can perform watchdog functions ● Strengthen civil society groups’ capacity to analyze shortcomings in legislation ● Build the capacity of CSOs in social auditing and citizen monitoring ● Build the capacity of CSOs to communicate, educate, and promote learning ● Support CSO monitoring and oversight of anti-corruption reforms through a grants program ● Utilize a flexible programming mechanism for supplemental technical assistance to Afghan stakeholders ● Administer Grants under Contract to CSOs and other non-governmental partners to support the activities under the relevant components of the AMANAT activity. V. PURPOSE OF THE EVALUATION The purpose of this evaluation is to (1) assess AMANAT interventions and implementation progress in achieving its objectives, (2) Identify the success and weakness of this project during implementation (3) provide lessons learned and recommendations to USAID/ODG for better management of AMANAT’s anti-corruption efforts and the design of future similar projects/activities. The findings and recommendations of the evaluation will be used to improve interventions and, if necessary, to change interventions as well as document achievements and gaps of interventions. The audience of the evaluation report will be the USAID/Afghanistan Mission, including the Office of Democracy and Governance, the Office of Program and Project Development, the Government of Afghanistan and the implementing partner. The report will also be of value to International Narcotics and Law Enforcement (INL), Department of Justice (DOJ), and the Special Inspector General for Afghanistan Reconstruction (SIGAR) colleagues. The evaluation will cover the period from the beginning of the AMANAT activity on August 23, 2017 until February 29, 2020. VI. EVALUATION QUESTIONS The Contractor must answer the evaluation questions below: 1. How and to what extent do AMANAT interventions contribute to reduced corruption in Afghan government institutions and improve public service delivery by the Afghan government? What interventions work? Why? What interventions do not work? Why? 2. What are the unintended consequences (positive or negative) of those interventions? How are the consequences, if any, affecting anti-corruption efforts in Afghanistan? 3. How and to what extent do AMANAT’s interventions fit in with the overall Afghan anti￾corruption context as regards political uncertainty, lack of political will, insecurity and post-peace scenario. 4. To what extent are AMANAT's interventions contributing to Civil Society's ability to advocate for reform? 5. To what extent are AMANAT’s interventions sustainable for the Afghan government and for CSOs? MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-8 VII. EVALUATION DESIGN & METHODOLOGY The Contractor must address the evaluation questions presented above for this mid-term performance evaluation. USAID/Afghanistan requires that a mixed methods approach, comprising both quantitative and qualitative methods for the evaluation methodology. Data requirements, collection methods, and required analyses must be determined collaboratively with USAID/Afghanistan under the direction of an independent, external evaluation Team Leader. Details on final datasets, data collection methods and instruments, and analytical framework(s) must be approved by the Technical Office (TO) COR as part of the work plan approval. Data must be disaggregated by sex, age, and/or other aspects (e.g. geographical local), where relevant. The evaluation team must conduct a document review to develop an analytical framework to guide this evaluation. Based on the desk review of AMANAT documentation, the team must develop a comprehensive evaluation work plan that clearly shows how each evaluation question will be answered with specific data collection and analysis methods that take into account COVID-19 travel and meeting restrictions. Tools and methodologies to collect data from program participants must be clear and must indicate whether remote or in-person data collection approaches will be used. This includes collecting data directly from program participants/beneficiaries as well as program participation records. Given the fluidity of the operating environment, the remote or in-person data collection may change in consultation with USAID. Due to uncertainly around the change in timing of the change of international travel due to COVID-19, The Contractor will initiate the evaluation remotely, which will desk review and finalization of the evaluation work plan. Following the lift of international travel restrictions, the Contractor’s expat team will travel to Afghanistan to conduct interviews with KI and collect additional field data, including direct observations with participation of the local team members, as needed. The exact dates of both international and in-country travel will be coordinated with USAID based on guidance from the US and Afghan governments and WHO. In the meantime, the Contractor will proceed with the desk review and other activities that can be conducted remotely with the understanding that ODG’s preference is to have the expatriate team members in country as soon as feasible to conduct the bulk of the evaluation. USAID/Afghanistan will determine during in-briefing between USAID/Afghanistan and the evaluation team what provinces the Contractor should travel to for collecting more data from the subset of provinces listed above. The Contractor is responsible for arranging all meetings and appointments needed to complete the required work outlined in this Statement of Work. The Contractor is confident that commencement of remote work described above is feasible and will ensure the evaluation will remain on track despite travel uncertainties. USAID/Afghanistan requests that the evaluation team complete the following table as part of its detailed design and evaluation work plan. The Contractor may choose to use another format if the below table is not preferred; however, any chosen format should contain all the information specified for each question as presented in the Evaluation Questions section above. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-9 Questions30 Suggested Data Sources Suggested Data Collection Methods Data Analysis Methods 1. How and to what extent do AMANAT interventions contribute to reduced corruption in Afghan government institutions and improved public service delivery by Afghan government? What interventions work? Why? What interventions do not work? Why? Progress Reports, Documents, registers and records, COR, IP staff, key informants at the government, sub grantees, beneficiaries, and other relevant stakeholders Desk review, Key Informant Interview (KII), Focus Group Discussions (FGDs) face-to-face with social distance norms and Direct Observation (DO) of service delivery and compare DO with plans, documents, strategies etc. Comparison of realities with what is written on papers. DRAFT (to be finalized by the evaluation team): The Contractor will compare proposed indicators v achievement of the indicators and expected throughout the evaluation period. This will qualitatively be validated based on the results of the interviews and analysis of the political, security, and economic developments during the evaluation period to determine key factors that contributed to the progress towards project objectives. 2. Generally, what anti-corruption effects (unanticipated results, unintended positive and negative consequences) are the AMANAT interventions generating, irrespective of the activity objectives? Documents (IP reports, AMELP and PIRS), TPM reports, IP staff, key informants, COR and other stakeholders Key Informants Interviews, Focus Group Discussions, Programmatic Documents, External Documents, AMANAT and GIRoA Data Sets and Observations. DRAFT (to be finalized by the evaluation team): The Contractor will review project documents and juxtapose them against public and KI perceptions during interviews and focus group discussion using 30 The evaluations questions will be finalized during the co-design process and may re-articulated or broken up further for clarity, or both. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-10 a guided, standardized questionnaires. 3. How and to what extent AMANAT interventions fit in with overall Afghan context such as political uncertainty, lack of political will and insecurity. Progress Reports, Documents, registers and records, COR, IP staff, key informants at the government, sub grantees, beneficiaries, and other relevant stakeholders Key informant interviews, FGDs, Documents Review DRAFT (to be finalized by the evaluation team): The Contractor will review project documents and juxtapose them against public and KI perceptions during interviews and focus group discussion using a guided, standardized questionnaires. . 4. To what extent are AMANAT's interventions contributing to Civil Society's ability to advocate for Reform? Progress Reports, IP, Key Informants interview, CSOs, Beneficiaries and COR Documents, KII, FGD DRAFT (to be finalized by the evaluation team): The Contractor will evaluate activates and capacities of CSO’s directly or indirectly involved in project activities and measure the change in the number of interventions implemented by the CSO’s, as well their efficacy and sustainability. 5. To what extent are AMANAT’s interventions sustainable for the Afghan government and for CSOs. Progress reports, Documents, IP staff, Ministries, Sub-grantees and other relevant stakeholders Progress reports, Documents, IP staff, Ministries, Sub-grantees and other relevant stakeholders DRAFT (to be finalized by the evaluation team): The Contractor will analyze AMANAT’s expenditures in sub￾grants to CSO’s and MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-11 Documents review, Interviews and FGDs Documents review, Interviews and FGDs the Afghan government and compare the change in the level of other resources these entities contributed to continuation or expansion of these activities. In addition, The Contractor will evaluate the role of AMANAT in CSO’s and government’s ability to identify resource needs and gaps and mobilize additional resources for anti-corruption activities. VIII. EXISTING PERFORMANCE INFORMATION SOURCES USAID/Afghanistan will share the documents below with the Contractor upon award of the Contract. The Contractor may request additional documents from USAID/Afghanistan. Those documents include but are not limited to: ● AMANAT Contract and Contract Modifications ● Annual Work Plans ● Activity Monitoring, Evaluation, and Learning Plan (AMELP); MSI has a database for M&E and grants ● Project Progress Reports (quarterly and annual reports) ● Data Quality Assessment (DQA) reports ● Other internal/ad hoc assessment/verification reports ● Other Activity-related documentation (such as Grants under Contract award documents, grantees’ reports, government coordination documents) IX. EVALUATION TEAM COMPOSITION The Contractor must provide a team of four key personnel. Overall, the team must have the following expertise: experience relating to anti-corruption programming, experience conducting assessments/evaluations for USAID or other major donors, and experience or evaluating DG programs in Afghanistan. Specific qualifications, skills, and expertise for each team member include: Evaluation Team Leader, International The Team Leader is a senior-level evaluation specialist with a minimum of a Master’s degree; preferably Ph.D., in Political Science, Social Science or a related subject area or equivalent years of relevant experience. The Team Leader must have at least 5 years of experience in evaluating MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-12 anti-corruption programs. The preference is an evaluation expert with anti-corruption experience. The Team Leader must also be an excellent English writer and have previous experience writing evaluation reports. He or she must be familiar with USAID evaluation policy and guidance included in the USAID Automated Directive System (ADS) Chapter 201. In addition, he or she must have a general understanding of development issues in Afghanistan. The Team Leader will be responsible for the development of the evaluation work plan and overall team coordination, including ensuring that team members adequately understand their roles and responsibilities, and for assigning individual data/information collection, and reporting responsibilities. Deputy Team Leader (DTL), International The DTL should have a strong background in anti-corruption activities as well as evaluations. S/he must have at least a Master’s degree, in Political Science, Social Science or a related subject area or equivalent years of relevant experience. S/he must have at least five years of experience in anti-corruption activities and managing or conducting evaluations in various sectors. S/he must have extensive analytical and data triangulation. experience, which equips him/her to conduct high-quality and in-depth analysis of anti-corruption issues in Afghanistan, including capacity building programs to Afghan key actors and system management development for Afghan government institutions to prevent and combat corruption in public service delivery to Afghan people. He/she should also have in-depth experience conducting anti-corruption research and assessments and developing anti-corruption strategy strategies and programming options. S/he will be responsible for designing the tools and methodology as well as training the national staff who will collect the data. An ability to conduct interviews and discussions as well as professional level report writing in English is required. Evaluation Team Members, Local Two mid-level Afghan evaluation specialists (a male and female) with at least a Master’s degree and five years of evaluation or anti-corruption work experience, or a Bachelor’s degree and eight years of evaluation/anti-corruption work experience. Both team members must have practical work experience in either anti-corruption programming or in conducting DG evaluations and have a comprehensive knowledge of the Afghan political and social context. The team members should have a deep understanding of development issues in Afghanistan. Both need to have strong English language skills in both written and verbal communications. X. EVALUATION SCHEDULE The contractor is to submit a detailed schedule of evaluation, however the tentative date for starting this evaluation will be June 8, 2020. This period includes preparatory work before arrival in-country and finalization after the trip. During this period the evaluation team should develop the draft work plan and submit it to USAID for review and comments before the in￾brief. USAID and the evaluation team will then discuss the work plan during the in-brief. The tools and instruments for data collection should be submitted to USAID after finalization of work plan. A six-day work week is authorized for this activity while in Afghanistan. See the Table of Evaluation Schedule below: MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-13 Activity Team Location Schedule (Week) Document review-desk review and preparation Remote Week 1 USAID review of draft work plan Remote Week 2 Final work plan submission to USAID Remote Week 3 In-brief In-country Week 4 Meetings with USAID and [Redacted] -based interviews In-country Weeks 5 - 6 Interviews in provinces outside [Redacted] In-country Week 7 - 8 Data analysis, preliminary report and presentation to USAID and IPs In-country Weeks 9 - 11 Draft report submission to USAID In-country Weeks 12 - 13 USAID/Afghanistan review of the draft report In-country Weeks 14 - 15 Final report submission to USAID Remote Weeks 16 - 17 Level of Effort (LOE) OSC requests three additional days for the team lead and two additional days for the deputy team lead in light of the large volume of activity-related documentation and the potentially significant volume of GIRoA documents and datasets. The desk review period is also the time during which initial design/drafting of research instruments takes place. It is anticipated that both of these processes may take additional time and back and forth with staff working remotely. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-14 Table 2: Minimum LOE in days by activity Activity LOE for Expat Team Lead Deputy Team Lead Expat Evaluation Team Member National Total LOE in Days Desk review and draft Work Plan (remote) 6 4 0 10 Team Planning Workshop (finalize and approve work plan, evaluation tools and design) (remote) 6 6 12 24 Travel to and from Afghanistan 4 4 0 8 In-brief 1 1 2 4 Field work (data collection and interviews) including preparation (organizing travel, schedules and travel days included)31 20 20 60 100 Mid brief to USAID 1 1 2 4 Preliminary data analysis and synthesis, drafting report and presentation materials 15 15 20 50 Exit brief to USAID 1 1 2 4 Finalize the evaluation report (remote) 8 5 0 13 Totals 62 57 98 217 Table 2: LOE in days by position 31 It is anticipated that expatriate team members will conduct some KIIs and interviews remotely. Evaluation team members will conduct FGDs, observations and some KIIs in person, when lockdowns have been lifted. The LOE will be reduced if the lock down and ban on international travel are not lifted during the period when the evaluation is conducted. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-15 Position Remote prep/Desk Review32 Travel to/from [Redacted] In￾country33 Finalization of Report Total Expat Team Leader (Expat) 12 4 38 8 62 Deputy Team Lead (Expat) 10 4 38 5 57 Afghan Specialist 6 0 43 0 49 Afghan Specialist 6 0 43 0 49 Totals 34 8 162 13 217 XI. DELIVERABLES AND REPORTING REQUIREMENTS All deliverables are internal to USAID/Afghanistan and the Contractor unless otherwise instructed by USAID/Afghanistan. The following deliverables are required for this evaluation Activity. 1. In-briefing: Within 3 working days of completing the desk review, the Evaluation Team will organize and host a remote in-briefing with the OPPD PST unit and the Office of Democracy and Governance (ODG) Team for introductions and to discuss the team’s understanding of the assignment, initial assumptions, evaluation questions, methodology, and work plan, and/or to adjust the SOW, if necessary. The in-briefing will be conducted using a mission-acceptable teleconferencing platform. 2. Evaluation Work Plan: Within 2 calendar days following the in-brief, the Evaluation Team Leader shall provide a detailed initial work plan to OPPD’s PST unit and ODG Team. The initial work plan will include: (a) the overall evaluation design, including the proposed methodology, data collection and analysis plan, and data collection instruments; (b) a list of the team members and their primary contact details while in-country, including the email address and mobile phone number for the team leader; and (c) the team’s proposed schedule 32 Includes desk review and remote team planning workshop 33 Includes in-brief, field data collection, mid-brief, preliminary data analysis, and exit brief MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-16 for the evaluation. USAID offices and relevant stakeholders are asked to take up to 2 days to review and consolidate comments on the work plan. Once the evaluation team receives the consolidated comments on the initial work plan, they are expected to return with a final work plan within 2 days. The final work plan shall include the list of potential interviewees and sites to be visited. 3. Mid-term Briefing and Interim Meetings: The evaluation team is expected to hold a mid￾term briefing with USAID/Afghanistan on the status of the evaluation activity, including potential challenges and emerging opportunities. Such meetings may be virtual if the evaluation is conducted during the COVID-19 era. If necessary, USAID/Afghanistan may ask the Contractor to provide weekly briefings by phone or email. Any delays or complications must be communicated to the TOCOR as early as possible to allow for quick resolution and to minimize any disruptions to the evaluation. 4. PowerPoint and Final Exit Presentation: The evaluation team is expected to hold a final exit presentation to discuss the summary of findings and recommendations to USAID/Afghanistan. This presentation will be scheduled as agreed upon during the in-briefing. Presentation slides should not exceed 18 in total. This presentation may be virtual if the evaluation is conducted during the COVID-19 era. 5. Draft Evaluation Report: The draft evaluation report should be consistent with the guidance provided in Section XIII: “Final Report Format.” The report will address each of the issues and questions identified in the SOW and any other factors the team considers having a bearing on the objectives of the evaluation. Any such factors can be included in the report only after consultation with USAID. The submission date for the draft evaluation report will be decided upon during the mid-term or exit briefing and submitted to OPPD’s PST unit and ODG by OSC, the non-conflicted sub-contractor under AMELA. Once the initial draft evaluation report is submitted, the following deadlines should be followed: a. ODG will have 10 working days in which to review and comment on the initial draft, after which point USAID/OPPD’s PST unit will have 2 working days to review and consolidate all USAID comments (total of 12 working days). OPPD will submit the consolidated comments to OSC. b. The evaluation team will then have 5 working days to make appropriate edits and revisions to the draft and re-submit the revised final draft report to USAID. c. ODG and the OPPD’s PST will have 10 working days after the submission of the second revised draft to again review and send any final comments. d. Recommendations Workshop(s): A workshop (or multiple workshops) with the key stakeholders to develop, discuss, and validate the recommendations. Final Evaluation Report: The evaluation team and contractor will be asked to take no more than 3 days to respond/incorporate the final comments from USAID. The Evaluation Team Leader will then submit the final report to the Evaluation Activity Manager for AMELA and the AMELA COR. Evaluation Final report should include all elements described in ADS 201mah, USAID MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-17 Evaluation Report Requirements (also listed below). The evaluation report should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the strategy, project, or activity. The evaluation must adhere to the criteria outlined in ADS 201maa, Criteria to Ensure the Quality of the Evaluation Report. All data and records (dataset, supporting documents such as code books, data dictionaries, scope and methodology used to collect and analyze the data) is the property of USAID and will, as applicable, be submitted in full in an easily readable electronic format, per the requirements outlined in ADS 579; organized and documented to be submitted to the USAID Development Data Library and for use by those not fully familiar with the project or evaluation; and owned by USAID. The evaluation report must: 1. Identify the evaluation as either an impact or performance evaluation per the definitions in ADS 201. 2. A 1–2-page evaluation fact sheet(s) highlighting target findings for decision-making. These fact sheets will respond to the question: Who needs to know what, and by when? The fact sheet(s) may be developed during report drafting, based on discussions with the evaluation team and the technical team. 3. Include an abstract of not more than 250 words briefly describing what was evaluated, evaluation questions, methods, and key findings or conclusions. The abstract should appear on its own page immediately after the evaluation report cover. 4. Include an Executive Summary 2–5 pages in length that summarizes key points (purpose and background, evaluation questions, methods, findings, and conclusions). 5. State the purpose of, audience for, and anticipated use(s) of the evaluation. 6. Describe the specific strategy, project, activity, or intervention to be evaluated including (if available) award numbers, award dates, funding levels, and implementing partners. 7. Provide brief background information. This should include country and/or sector context; specific problems or opportunities the intervention addresses; and the development hypothesis, theory of change, or simply how the intervention addresses the problem. 8. State the evaluation questions. 9. In an impact evaluation, state evaluations questions about measuring the change in specific outcomes attributable to a specific USAID intervention. 10. Describe the evaluation method(s) for data collection and analysis. 11. Describe limitations of the evaluation methodology. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-18 12. In an impact evaluation, use specific experimental or quasi-experimental methods to answer impact evaluation questions. 13. Include evaluation findings and conclusions. 14. If recommendations are included, separate them from findings and conclusions. 15. Include infographics and visuals where necessary. 16. Address all evaluation questions in the Statement of Work (SOW) or document approval by USAID for not addressing an evaluation question. 17. Include the annexes listed under the evaluation report format section. XII. MANAGEMENT AMELA’s non-conflicted sub-contractor OSC will identify and hire the evaluation team, pending the AMELA COR’s concurrence and CO approval, to assist in facilitating the work plan, and arrange meetings with key stakeholders identified prior to the initiation of the fieldwork. The evaluation team will organize other meetings as identified during the course of the evaluation, in consultation with AMELA’s non-conflicted sub-contractor OSC and USAID/Afghanistan. AMELA’s non-conflicted subcontractor OSC is responsible for all logistical support required for the evaluation team, including arranging accommodation, security, office space, computers, Internet access, printing, communication, and transportation. The evaluation team will officially report to AMELA’s non-conflicted sub-contractor OSC management. AMELA’s non-conflicted sub-contractor OSC is responsible for all direct coordination with USAID/Afghanistan/OPPD, through [Redacted] COR for AMELA. From a technical management perspective, the evaluation team will work closely with AMANAT COR, [Redacted]. In order to maintain objectivity, OPPD’s Evaluation will make all final decisions about the evaluation. The length of the evaluation final report should be within the range of 25-30 pages in length, excluding executive summary and annexes. It should be written in English, using Gil Sans MT12 point font, single space spacing, and be consistent with USAID branding policy. The report should be structured as follows: ● Include enough information on the cover of the evaluation report so that a reader can immediately understand that it is an evaluation and what was evaluated. The evaluation cover must: ● Include a title block in USAID light blue background color. ● Include the word “Evaluation” at the top of the title block and center the report title underneath that. The title should also include the word “evaluation.” ● Include the following statement across the bottom of the cover page: “This publication was produced at the request of the United States Agency for International Development. It was prepared independently by [list authors and organizations involved in the preparation of the report].” For an internal evaluation team, use the MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-19 following statement: “This publication was produced at the request of [USAID/Mission or OU] and prepared by an internal evaluation team comprised of [list authors and affiliation].” ● Feature one high-quality photograph representative of the project being evaluated and include a brief caption on the inside front cover describing the image with photographer credit. ● State the month and year of report publication (e.g., when final and approved by USAID Operating Unit). 1. Title Page 2. Table of Contents 3. List of any acronyms, tables and/or figures 4. Acknowledgements or Preface (optional) 5. Evaluation abstract 6. Executive Summary (2-5 pages): This section should represent a concise and accurate statement of the most critical elements of the report. 7. Introduction a. Description of the project evaluated, including goal and expected results b. Brief statement on purpose of the evaluation, plus a list of the evaluation questions c. Description of the methods used in the evaluation (such as desk/document review, interviews, site visits, surveys, etc.), the rationale and location for field visits (if any), and a description of the numbers and types of respondents d. Limitations to the evaluation, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.) 8. Findings Findings should be specific, concise, and supported by strong quantitative or qualitative evidence. a. Findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions. b. Describe findings, focusing on each of the evaluation questions and providing gender disaggregation where appropriate c. If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females. d. Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or the compilation of people’s opinions 9. Conclusions ● Conclusions should be specific, concise, and supported by strong quantitative or qualitative evidence. ● Conclusions are value statements drawn from the data gathered during the evaluation process 10.Recommendations ● Recommendations should be actionable, practical and specific statements for existing programming and for the design and performance of future programming ● Each recommendation should be supported by a specific set of findings MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-20 ● Include recommended future objectives and types of activities based on lessons learned 11.Annexes I. Evaluation Statement of Work: If the SOW was revised over the course of the evaluation, the evaluation report should include the updated SOW as an Annex rather than the original SOW. The Contracting Officer’s Representative of the evaluation must agree upon, in writing, all modifications to the SOW, whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline. II. Final Work Plan including methodology description (include any pertinent details not captured in the report) III. Copies of all data collection and analysis tools used, such as questionnaires, checklists, survey instruments, and discussion guides. IV. List of critical and key documents reviewed V. Schedule of Meetings and sources of information (If confidentiality is a concern, the team should discuss and agree upon an approach with USAID) VI. Notes from key interviews, focus group discussions and other meetings VII. Documentation of any changes to the SOW or evaluation process VIII. Statement of differences (if applicable), any “statements of difference” regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team. IX. All sources of information—properly identified and listed. X. Signed disclosures of conflicts of interest from evaluation team members. XI. Summary information about evaluation team members, including qualifications, experience, and role on the team. XII. Matrix of Findings, Conclusions and Recommendations (FCR) 12.One- or two-page briefer of findings, recommendations and lessons learned (optional) XIV. OVERALL REPORTING GUIDELINES The evaluation report should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the validity of the project’s TOC and the effectiveness of the project. Evaluation reports shall address all evaluation questions included in the SOW and be written in highly professional English, free of grammatical and typographical error, and with professional formatting. Annexes should be free of personally-identifiable information (PII), and any/all information which may compromise the privacy and protection of evaluation participants will be reviewed by USAID for potential redaction/removal. Any modifications to the statement of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline need to be agreed upon in writing by the AMELA COR. 5. DISSEMINATION REQUIREMENTS In accordance with ADS Chapter 201, the Contractor will make the final AMANAT mid-term performance evaluation report publicly available through the Development Experience MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | A-21 Clearinghouse (DEC). This publication of the report must be made within 90 days of the evaluation conclusion (approval of the final report by USAID/Afghanistan). The evaluation findings, conclusions and recommendations will be discussed with AMANAT’s Implementing Partner, Government entities and other key stakeholders. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | B-1 ANNEX B – EVALUATION WORK PLAN AMANAT mid-term performance evaluation schedule: Day Date Day of the Week Location Tasks Day 1 Dec 15 Tuesday Remote Virtual In-brief with USAID Day 2 Dec 28 Monday Remote Document review Day 3 Dec 29 Tuesday Remote Document review Day 4 Dec 30 Wednesday Remote Document review Day 5 Dec 31 Thursday Remote Document review/draft work plan Day 6 Jan 1 Friday Remote Document review/draft work plan Day 7 Jan 2 Saturday Remote Day off Day 8 Jan 3 Sunday Remote Day off Day 9 Jan 4 Monday Remote Document review/draft work plan Day 10 Jan 5 Tuesday Remote Team Planning Workshop, finalization of the evaluation design and tools, and the work plan Day 11 Jan 6 Wednesday Remote Team Planning Workshop, finalization of the evaluation design and tools, and the work plan Day 12 Jan 7 Thursday Remote Team Planning Workshop, finalization of the evaluation design and tools, and the work plan Day 13 Jan 8 Friday Remote Team Planning Workshop finalization of the evaluation design and tools, and the work plan Day 14 Jan 9 Saturday Remote Day off Day 15 Jan 10 Sunday Remote Day off Day 16 Jan 11 Monday Remote Team Planning Workshop, finalization of the evaluation design and tools, and MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | B-2 the work plan; Field work CCN (data collection and interviews) Day 17 Jan 12 Tuesday Remote Team Planning Workshop, finalization of the evaluation design and tools, and the work plan; Field work CCN (data collection and interviews) Day 18 Jan 13 Wednesday Travel Travel to [Redacted], UAE; Field work CCN (data collection and interviews) Day 19 Jan 14 Thursday Travel Obtain Visa in [Redacted]; Field work CCN (data collection and interviews) Day 20 Jan 15 Friday Travel Travel to [Redacted], Afghanistan Day 21 Jan 16 Saturday [Redacted] Preparation work (organizing travel, schedules, and travel days) and introductory meetings with GIRoA partners; Field work CCN (data collection and interviews) Day 22 Jan 17 Sunday [Redacted] Preparation work (organizing travel, schedules, and travel days) and introductory meetings with GIRoA partners; Field work CCN (data collection and interviews) Day 23 Jan 18 Monday [Redacted] Field work (data collection and interviews) Day 24 Jan 19 Tuesday [Redacted] Field work (data collection and interviews) Day 25 Jan 20 Wednesday [Redacted] Field work (data collection and interviews) Day 26 Jan 21 Thursday [Redacted] Field work (data collection and interviews) Day 27 Jan 22 Friday [Redacted] Day off Day 28 Jan 23 Saturday [Redacted] Field work (data collection and interviews) Day 29 Jan 24 Sunday [Redacted] Field work (data collection and interviews) MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | B-3 Day 30 Jan 25 Monday [Redacted] Field work (data collection and interviews) Day 31 Jan 26 Tuesday [Redacted] Field work (data collection and interviews) Day 32 Jan 27 Wednesday [Redacted] Field work (data collection and interviews) Day 33 Jan 28 Thursday [Redacted] Field work (data collection and interviews) Day 34 Jan 29 Friday [Redacted] Day off Day 35 Jan 30 Saturday [Redacted] Field work (data collection and interviews) Day 36 Jan 31 Sunday [Redacted] Field work (data collection and interviews) Day 37 Feb 1 Monday [Redacted] Field work (data collection and interviews) Day 38 Feb 2 Tuesday [Redacted] Field work (data collection and interviews) Day 39 Feb 3 Wednesday [Redacted] Field work (data collection and interviews) Day 40 Feb 4 Thursday [Redacted] Field work (data collection and interviews) Day 41 Feb 5 Friday [Redacted] Day off Day 42 Feb 6 Saturday [Redacted] Field work (data collection and interviews) Day 43 Feb 7 Sunday [Redacted] Field work (data collection and interviews) Day 44 Feb 8 Monday [Redacted] Mid/Exit-brief with USAID/Afghanistan Day 45 Feb 9 Tuesday Travel Travel from [Redacted], Afghanistan; Field work CCN (data collection and interviews) Day 46 Feb 10 Wednesday Travel Travel from [Redacted], Afghanistan; Field work CCN (data collection and interviews) MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | B-4 Day 47 Feb 11 Thursday Remote Preliminary data analysis, drafting report and presentation materials; Field work CCN (data collection and interviews) Day 48 Feb 12 Friday Remote Preliminary data analysis, drafting report and presentation materials; Field work CCN (data collection and interviews) Day 49 Feb 13 Saturday Remote Day off; Field work CCN (data collection and interviews) Day 50 Feb 14 Sunday Remote Day off; Field work CCN (data collection and interviews) Day 51 Feb 15 Monday Remote Preliminary data analysis, drafting report and presentation materials Day 52 Feb 16 Tuesday Remote Preliminary data analysis and synthesis, drafting report, and presentation materials Day 53 Feb 17 Wednesday Remote Data analysis and synthesis, drafting report, and presentation materials Day 54 Feb 18 Thursday Remote Data analysis and synthesis, drafting report, and presentation materials Day 55 Feb 19 Friday Remote Data analysis and synthesis, drafting report, and presentation materials Day 56 Feb 20 Saturday Remote Day off Day 57 Feb 21 Sunday Remote Day off Day 58 Feb 22 Monday Remote Data analysis and synthesis, drafting report, and presentation materials Day 59 Feb 23 Tuesday Remote Data analysis and synthesis, drafting report, and presentation materials Day 60 Feb 24 Wednesday Remote Data analysis and synthesis, drafting report, and presentation materials Day 61 Feb 25 Thursday Remote Data analysis and synthesis, drafting report, and presentation materials MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | B-5 Day 62 Feb 26 Friday Remote Data analysis and synthesis, drafting report, and presentation materials Day 63 Feb 27 Saturday Remote Day off Day 64 Feb 28 Sunday Remote Day off Day 65 Mar 1 Monday Remote Data analysis and synthesis, drafting report, and presentation materials Day 66 Mar 2 Tuesday Remote Data analysis and synthesis, drafting report, and presentation materials Day 67 Mar 3 Wednesday Remote Data analysis and synthesis, drafting report, and presentation materials Day 68 Mar 4 Thursday Remote Initial draft of the evaluation report Day 69 Mar 5 Friday Remote Initial draft of the evaluation report Day 70 Mar 6 Saturday Remote Day off Day 71 Mar 7 Sunday Remote Day off Day 72 Mar 8 Monday Remote Initial draft of the evaluation Day 73 Mar 9 Tuesday Remote Submission of the draft evaluation report Day 74 Mar 10 Wednesday Remote USAID Review Day 75 Mar 11 Thursday Remote USAID Review Day 76 Mar 12 Friday Remote USAID Review Day 77 Mar 13 Saturday Remote Day off Day 78 Mar 14 Sunday Remote Day off Day 79 Mar 15 Monday Remote USAID Review Day 80 Mar 16 Tuesday Remote USAID Review Day 81 Mar 17 Wednesday Remote USAID Review Day 82 Mar 18 Thursday Remote USAID Review Day 83 Mar 19 Friday Remote USAID Review Day 84 Mar 20 Saturday Remote Day off MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | B-6 Day 85 Mar 21 Sunday Remote Day off Day 86 Mar 22 Monday Remote USAID Review Day 87 Mar 23 Tuesday Remote USAID Review Day 88 Mar 24 Wednesday Remote USAID Review Day 89 Mar 25 Thursday Remote USAID Review Day 90 Mar 26 Friday Remote USAID reverts draft for finalizing Day 91 Mar 27 Saturday Remote Day off Day 92 Mar 28 Sunday Remote Day off Day 93 Mar 29 Monday Remote Finalize the evaluation report Day 94 Mar 30 Tuesday Remote Finalize the evaluation report Day 95 Mar 31 Wednesday Remote Finalize the evaluation report, AMELA Review Day 96 April 1 Thursday Remote Final submission AMANAT Evaluation questions, data sources, data collection methods, and data analysis methods. The team collected data to answer the evaluation questions and identify evidence of progress toward intended outcomes as well as findings related to opportunities, and corrective or adaptive actions needed to improve outcomes and programming. Data requirements, collection methods, and required analyses were further refined collaboratively with USAID/Afghanistan by the evaluation team. Evaluation Question Data Source(s) Data Collection Method(s) Data Analysis Method(s) 1. How and to what extent do AMANAT interventions contribute to reduced corruption in Afghan government institutions and improve public service delivery by the Afghan government? What AMANAT documents, progress reports, registers and records. AMANAT COR, IP staff, key informants at the government, stakeholders, beneficiaries, and Desk review, KIIs (purposive), FGDs (vis-à-vis following social distancing rules). DOs Comparison of program indicators and targets against achievements. Qualitative data analysis emphasizing contextual analysis of program activities such as MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | B-7 interventions work? Why? What interventions do not work? Why? other relevant parties. content analysis, comparison and pattern analysis and tally sheets. Quantitative data analysis of program progress through secondary analysis of program reporting data Triangulating and cross￾checking AMANAT’s reporting through KIIs with USAID, project stakeholders, and CSOs. 2. What are the unintended consequences (positive or negative) of those interventions? How are the consequences, if any, affecting anti￾corruption efforts in Afghanistan? USAID programmatic documents, IP programmatic documents. GIRoA officials; other stakeholders. Desk review, KIIs, FGDs, and DOs. Desk review of relevant external documents, AMANAT and GIRoA data sets Qualitative data analysis emphasizing contextual analysis of AMANAT activities such as content analysis, comparison and pattern analysis and tally sheets. Analysis of targeted results, outputs, and outcomes of PPR in each of its major areas of intervention. Verification of AMANAT’s reporting and stated results versus the actual results in each project component. Analysis and synthesis of notes of impressions obtained through on￾site visits/direct observation in municipalities and MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | B-8 relevant government branches. 3. How and to what extent do AMANAT’s interventions fit in with the overall Afghan anti￾corruption context as regards political uncertainty, lack of political will, insecurity and post-peace scenario. AMANAT progress reports, documents, registers and records. COR, IP staff, key informants at the government, sub grantees, beneficiaries, other relevant stakeholders. Desk review, KIIs, FGDs, DOs. Desk review of reports and programmatic documents. Analysis of the GIRoA datasets. Qualitative: content analysis, comparison and pattern analysis and tally sheets. Analysis and synthesis of notes of impressions obtained through FGDs and on-site visits/direct observation. Quantitative: inferential statistical analysis. Verification of AMANAT’s reporting with the actual results. 4. To what extent are AMANAT's interventions contributing to Civil Society's ability to advocate for reform? AMANAT progress reports, documents, registers and records. COR, IP staff, key informants at CSOs, GIRoA, sub grantees, beneficiaries, other relevant stakeholders. Desk review, KIIs, FGDs, DOs. Qualitative content analysis, comparison and pattern analysis. Disaggregated by gender. Analysis and synthesis of notes of impressions obtained through FGDs, on-site visits/direct observation. 5. To what extent are AMANAT’s interventions sustainable for the Afghan government and for CSOs? AMANAT reports, documents, registers and records. Key informants at Afghan Ministries, GIRoA, sub grantees, and other Desk review of relevant internal and external documents, including progress reports and any documentation obtained from relevant line ministries, other Analysis of AMANAT’s expenditures in sub￾grants to CSO’s and the Afghan government and compare the change in the level of other resources these entities contributed to continuation or MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | B-9 relevant stakeholders. government branches, and CSOs. KIIs and FGDs, Site Visits/Direct Observation. expansion of these activities. The Contractor will evaluate the role of AMANAT in CSO’s and government’s ability to identify resource needs and gaps and mobilize additional resources for anti-corruption activities. In addition, the Contractor will assess resources allocated for sustainability and capacity building efforts provided by the AMANAT team and seek if there is a mechanism in place to sustain the achievements. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-1 ANNEX C – CORE DATA COLLECTION TOOLS The following documents represent the core data collection tools for each respondent type. These core tools were then adapted for each individual respondent. That is, each respondent from a specific “type” was asked the same core set of questions. Then they were asked questions specific to their background and experience, as derived from desk research and preliminary analysis of initial interviews. As a result of this approach, the team constructed approximately 40 unique data collection tools. All instruments have the following cover page. Key Informant Interview (KII) Guide Interview DETAILS: Client: Project Name: Researcher Name: Interview Data: Start Time: End Time: Respondent Name: Respondent Position: Organization/Institution: Informed Consent Obtained: Hello, I am/we are _______________ and I/we work with Overseas Strategic Consulting (OSC), an American company conducting research for the United States Agency for International Development (USAID). We are evaluating the Afghanistan’s Measure for Accountability and Transparency (AMANAT) project implemented in the Islamic Republic of Afghanistan since 2017. We want to understand how USAID’s support to the Afghanistan government’s efforts to reduce and prevent corruption in the provision of public services has worked and what might be improved for future similar in Afghanistan. You have been selected to participate in this research because we believe you can provide a unique perspective on the AMANAT program. We believe the information that you can provide on your engagement with AMANAT-implemented activities would be very helpful to our research. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-2 Your participation in this interview is completely voluntary and can be stopped at any time. You will not be provided with any payment or direct benefit for your participation. We will record your answers to a series of project-related questions by hand or electronically via type-written notes. We will not discuss your responses with anyone and your name will not appear in connection to the information you give us. The interview will last approximately 1 hour. Do you understand that your participation in this interview is voluntary and unpaid and that you can end the interview at any point? Are you willing to participate? MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-3 ANNEX C.1 – TOOL FOR AMANAT TEAM General questions on project design and implementation: 1. How long have you worked on the AMANAT project? 2. What role do/did you play in implementing the AMANAT project? 3. How is coordination between USAID, the GIRoA, and CSOs under one mechanism (AMANAT)? 4. Are the different components producing synergies or are there coordination challenges? 5. What plans does AMANAT have in place to strengthen sustainability of its efforts in the remaining term? EQ 1: HOW AND TO WHAT EXTENT DO AMANAT INTERVENTIONS CONTRIBUTE TO REDUCED CORRUPTION IN AFGHAN GOVERNMENT INSTITUTIONS AND IMPROVE PUBLIC SERVICE DELIVERY BY THE AFGHAN GOVERNMENT? WHAT INTERVENTIONS WORK? WHY? WHAT INTERVENTIONS DO NOT WORK? WHY? 1. What mechanisms are in place to monitor and evaluate progress? a. At the program level? Intervention level? 2. How does AMANAT determine whether an intervention contributes to corruption reduction? a. Has progress reflected program logic? How is this determined? 3. AMANAT has supported a series of knowledge generation exercises. For each one, can you tell me how it has contributed to GIRoA ability to identify vulnerabilities to corruption? a. iPEA b. VCA c. IA d. How effective were trainings and capacity building workshops that supported these activities? e. How do the ministries differ in their utilization of these tools? Do any ministries lead in their adoption and utilization? Lag behind? f. To what extent did MEC benefit from AMANAT support to its training needs? i. Why was support to MEC withdrawn? 4. To what extent has adoption of BPR/CRMP strengthened the capacity of GIRoA institutions to implement anti-corruption reforms? a. How do the three ministries differ in their utilization of these tools? 5. What role has the support to MOPH with trainings, strategic communications and complaints handling mapping played in strengthened capacity? 6. To what extent has AMANAT’s support to the School Management Shuras strengthened capacity to implement anti-corruption reforms? a. Successes of the pilot? Challenges and lessons learned? 7. Regarding CLIN002 – what is AMANAT’s readiness to address opportunities as they arise? a. To what extent does AMANAT solicit opportunities? 8. How has AMANAT’s CLA activities contributed to strengthen the program’s ability to contribute to program goals? 9. Overall, did AMANAT meet its midterm objectives and project milestones? MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-4 a. What interventions worked? i. What are the shared characteristics of successful interventions? ii. How does the definition of success vary across partners? iii. What intervention contributed to the most significant change? b. What interventions did not work? i. What are the shared characteristics of unsuccessful interventions? Unique characteristic of leas successful intervention? 10. Can you describe the nature of your external relationships? 11. And finally, how did internal process-driven elements contribute to program success? (e.g., staffing experience, coordination between teams, etc.) EQ 2: WHAT ARE THE UNINTENDED CONSEQUENCES (POSITIVE OR NEGATIVE) OF THOSE INTERVENTIONS? HOW ARE THE CONSEQUENCES, IF ANY, AFFECTING ANTI￾CORRUPTION EFFORTS IN AFGHANISTAN? 1. How are unintended consequences recognized or detected? a. What is the evidence or recognized unintended consequences, both positive and negative? 2. To what extent are unintended consequences positively impacting anti-corruption efforts? Negatively? 3. Are any unintended consequences impeding AMANAT’s work? EQ 3: HOW AND TO WHAT EXTENT DO AMANAT’S INTERVENTIONS FIT IN WITH THE OVERALL AFGHAN ANTI-CORRUPTION CONTEXT AS REGARDS POLITICAL UNCERTAINTY, LACK OF POLITICAL WILL, INSECURITY AND POST-PEACE SCENARIO. 1. How do AMANAT interventions mitigate the effect of government turnover or restructuring? a. To what extent did the MOMD restructure impact programming? b. To what extent did the Presidential election impact programming decisions? 2. How does AMANAT mitigate lack of political will in partner institutions? a. How does AMANAT account for the presence of corruption cases in partner institutions where reforms are targeted? 3. How does AMANAT mitigate the impact of insecurity on programming? 4. How has AMANAT adapted programming and developed contingencies for the ongoing peace negotiations? a. To what extent will AMANAT results be sustainable in a post-peace environment? 5. Considering these factors, how does AMANAT fit within the broader anti-corruption programmatic landscape? a. What are AMANAT’s unique elements that allow it to navigate these complexities? EQ 4: TO WHAT EXTENT ARE AMANAT'S INTERVENTIONS CONTRIBUTING TO CIVIL SOCIETY'S ABILITY TO ADVOCATE FOR REFORM? 1. To what extent has mobilizing CSOs strengthened the overall CS’ ability to advocate for reform? MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-5 a. To what extent did the groundwork for the CSCC creation contribute? b. What has been the contribution of AMANAT’s extensive networking and meetings with CSOs? 2. To what extent did watchdog trainings strengthen CS’ ability to advocate for reform? a. What has been the contribution of the budget process and oversight trainings? b. Case studies? 3. To what extent did legislative process (and related) trainings strengthen CSG capacity to analyze shortcomings in legislation? a. Case studies? 4. To what extent did social auditing/citizen monitoring trainings strengthen CS’ ability to advocate for reform? a. Case studies? 5. To what extent did strategic communications training and campaign support strengthen CS’ ability to advocate for reform? a. Case studies? 6. Overall, how has the grants program strengthened CS’ ability to advocate for reform? a. How has proposal development and related subject matter training (i.e., Magenta-led training) contributed to CS capacity? b. What have been the contributions of specific grantees? i. Most significant change? ii. Challenges? EQ 5: TO WHAT EXTENT ARE AMANAT’S INTERVENTIONS SUSTAINABLE FOR THE AFGHAN GOVERNMENT AND FOR CSOS? 1. What mechanisms exist to assess ability for the Afghan government to sustain results during and following AMANAT support? a. And mechanisms for CSOs? i. How is ability to sustain results prioritized when assessment potential beneficiary CSOs? 2. How are enhanced GIRoA-led initiatives to identify vulnerabilities to corruption sustained? 3. How is strengthened capacity of GIRoA institutions to implement anti-corruption reforms sustained? 4. How are strengthened CS capabilities to assess and improve GoA anti-corruption efforts sustained? 5. Which CSOs are maintaining results following close of AMANAT support? Donor coordination approach and good practices 1. Have similar projects/programs operated by other donors in Afghanistan offered any lessons learned to support AMANAT initiatives? Future activities in the anticorruption sector 1. What are some of the positive lessons learned during this period of the AMANAT implementation? 2. What areas within the anticorruption sector require further guidance and support by international donor agencies like USAID? MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-6 ANNEX C.2 – TOOL FOR AMANAT IP General questions: 1. What role do/did you play with the AMANAT project? 2. How long did you serve in this position? EQ 1: HOW AND TO WHAT EXTENT DO AMANAT INTERVENTIONS CONTRIBUTE TO REDUCED CORRUPTION IN AFGHAN GOVERNMENT INSTITUTIONS AND IMPROVE PUBLIC SERVICE DELIVERY BY THE AFGHAN GOVERNMENT? WHAT INTERVENTIONS WORK? WHY? WHAT INTERVENTIONS DO NOT WORK? WHY? 1. (For EAI) What mechanisms are in place to monitor and evaluate progress? a. At the program level? Intervention level? 2. (For EAI) What data base platforms and database products do EAI use? a. How is data stored and accessed, in the event of external DQAs? 3. (For EAI) How does AMANAT determine whether an intervention contributes to corruption reduction? a. What further MEL elements can be incorporated to better evaluate program contributions? 4. Considering the program element you supported, can you tell me how it contributed to GIRoA ability to identify vulnerabilities to corruption? 5. Overall, did AMANAT meet its goals for the specific program element you supported? a. What aspects were particularly successful? Why? b. What aspects were particularly challenging? Why? 6. If AMANAT were to replicate this activity in a different ministry, how would you alter the approach to be more effective? 7. Can you describe the nature of your working relationship with your government partners? a. How can AMANAT improve the process through which these sorts of activities are undertaken? EQ 2: WHAT ARE THE UNINTENDED CONSEQUENCES (POSITIVE OR NEGATIVE) OF THOSE INTERVENTIONS? HOW ARE THE CONSEQUENCES, IF ANY, AFFECTING ANTI￾CORRUPTION EFFORTS IN AFGHANISTAN? 1. (For Govt. IP) Did you notice any unintended consequences, both positive and negative, of the activity you supported? 2. (For Govt. IP) To what extent are any unintended consequences positively impacting anti￾corruption efforts? Negatively? EQ 3: HOW AND TO WHAT EXTENT DO AMANAT’S INTERVENTIONS FIT IN WITH THE OVERALL AFGHAN ANTI-CORRUPTION CONTEXT AS REGARDS POLITICAL UNCERTAINTY, LACK OF POLITICAL WILL, INSECURITY AND POST-PEACE SCENARIO. 1. (For Govt. IPs) How did the program elements you contributed to mitigate the effects of the following: a. Government turnover/restructuring MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-7 b. Lack of political will c. Impact of insecurity d. Prospect of post-peace environment 2. Considering these factors, how does AMANAT’s assistance to GIRoA fit within the broader anti-corruption programmatic landscape? EQ 4: TO WHAT EXTENT ARE AMANAT'S INTERVENTIONS CONTRIBUTING TO CIVIL SOCIETY'S ABILITY TO ADVOCATE FOR REFORM? 1. (For CSO training) To what extent did trainings strengthen CS ability to advocate for reform? a. Watchdog trainings b. Legislative process c. Social auditing/citizen monitoring d. Strategic communications 2. (For Magenta) How has anti-corruption subject matter training contributed to CS capacity? 3. What mechanisms were in place to monitor and evaluate activity progress and success? 4. Overall, which elements of the trainings succeeded? a. Which require additional consideration or revision? 5. Were there any unanticipated difficulties (e.g., comprehension, relevance, etc.) considering the training subject matter? 6. How can future programming adapt the trainings to better meet the beneficiaries’ needs? a. Do the existing trainings require placement in a broader curriculum? If so, what does that look like? EQ 5: TO WHAT EXTENT ARE AMANAT’S INTERVENTIONS SUSTAINABLE FOR THE AFGHAN GOVERNMENT AND FOR CSOS? 1. (For Govt. IP) How are enhanced GIRoA-led initiatives to identify vulnerabilities to corruption or implement anti-corruption reforms strengthened? That is, do you believe the program element you worked on will be sustained by the relevant institution? a. Why/why not? b. How can future AMANAT programming strengthen potential sustainability in its partner institutions? 2. (For CSO IP) How have strengthened CS capabilities to assess and improve GoA anti￾corruption efforts sustained? 3. (For Govt. IP) Are there any specific mechanisms you are aware of for the Afghan government to sustain results during and following AMANAT support? 4. (For Govt. IP) Do you believe the program element you contributed to is replicable in other parts of government? a. What aspects will prove challenging? How can these challenging elements be overcome? Other questions 1. In your experience, are there lessons from other projects/programs operated by other donors in Afghanistan that AMANAT can incorporate into their programming? MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-8 2. What are some of the positive lessons learned during this period of your assistance to AMANAT? 3. Are there any other areas you would like to bring up? MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-9 ANNEX C.3 – TOOL FOR AMANAT STTA General questions: 1. What role do/did you play with the AMANAT project? 2. How long did you serve in this position? General questions: 3. Can you please tell us a bit about your professional background? 4. How did you become associated with the AMANAT project? EQ 1: HOW AND TO WHAT EXTENT DO AMANAT INTERVENTIONS CONTRIBUTE TO REDUCED CORRUPTION IN AFGHAN GOVERNMENT INSTITUTIONS AND IMPROVE PUBLIC SERVICE DELIVERY BY THE AFGHAN GOVERNMENT? WHAT INTERVENTIONS WORK? WHY? WHAT INTERVENTIONS DO NOT WORK? WHY? 1. Can you please provide us an overview of the role you played with AMANAT, specifically the GIRoA-focused activities you supported? 2. Considering the program element/activity you supported, can you tell me how it contributed to GIRoA ability to identify vulnerabilities to corruption or implement anti￾corruption reform? 3. Overall, did AMANAT meet its goals for the specific program element you supported? a. What aspects were particularly successful? Why? b. What aspects were particularly challenging? Why? 4. If AMANAT were to replicate this activity in a different ministry, how would you alter the approach to be more effective? 5. Can you describe the nature of your working relationship with your government partners? a. How can AMANAT improve the process through which these sorts of activities are undertaken? 6. Can you describe the nature of your working relationship with the AMANAT team? EQ 2: WHAT ARE THE UNINTENDED CONSEQUENCES (POSITIVE OR NEGATIVE) OF THOSE INTERVENTIONS? HOW ARE THE CONSEQUENCES, IF ANY, AFFECTING ANTI￾CORRUPTION EFFORTS IN AFGHANISTAN? 1. Did you notice any unintended consequences, both positive and negative, of the activity you supported? EQ 3: HOW AND TO WHAT EXTENT DO AMANAT’S INTERVENTIONS FIT IN WITH THE OVERALL AFGHAN ANTI-CORRUPTION CONTEXT AS REGARDS POLITICAL UNCERTAINTY, LACK OF POLITICAL WILL, INSECURITY AND POST-PEACE SCENARIO. 1. How does AMANAT’s assistance to GIRoA fit within the broader anti-corruption programmatic landscape? EQ 5: TO WHAT EXTENT ARE AMANAT’S INTERVENTIONS SUSTAINABLE FOR THE AFGHAN GOVERNMENT AND FOR CSOS? MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-10 1. Do you believe the program element you worked on will be sustained by the relevant institution? a. Why/why not? b. How can future AMANAT programming strengthen potential sustainability in its partner institutions? 2. Are there any specific mechanisms you are aware of for the Afghan government to sustain results during and following AMANAT support? 3. Do you believe the program element you contributed to is replicable in other parts of government? a. What aspects will prove challenging? How can these challenging elements be overcome? Other questions 1. In your experience, are there lessons from other projects/programs operated by other donors in Afghanistan that AMANAT can incorporate into their programming? 2. What are some of the positive lessons learned during this period of your assistance to AMANAT? 3. Are there any other areas you would like to bring up? MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-11 ANNEX C.4 – TOOL FOR GIROA AND SUB-NATIONAL ENTITIES EQ 1: HOW AND TO WHAT EXTENT DO AMANAT INTERVENTIONS CONTRIBUTE TO REDUCED CORRUPTION IN AFGHAN GOVERNMENT INSTITUTIONS AND IMPROVE PUBLIC SERVICE DELIVERY BY THE AFGHAN GOVERNMENT? WHAT INTERVENTIONS WORK? WHY? WHAT INTERVENTIONS DO NOT WORK? WHY? 1. Could you explain the areas of assistance, and the key objectives, of the AMANAT program with your Directorate and its staff? 2. What offices or units of your Ministry/Directorate were the primary targets of assistance under cooperation with the USAID/AMANAT program? 3. AMANAT has supported a series of knowledge generation exercises. For each one, can you tell me how it has contributed to [Ministry] ability to identify vulnerabilities to corruption? a. iPEA (all 5) b. VCA (MOHE, MOPH, MOMD) c. IA (all 5) d. How effective were trainings and capacity building workshops that supported these activities? i. Which trainings or workshops were most effective? e. How beneficial was MEC support to [Ministry]’s anti-corruption activities? 4. What mechanisms are in place to monitor and evaluation progress of anti-corruption programming at your ministry? 5. (MOHE, MOPH, MORR) To what extent has BPR/CRMP strengthened [Ministry]’s capacity to implement anti-corruption reforms? 6. (MOPH) What role has AMANAT support with trainings, strategic communications training and complaints handling mapping played in strengthening capacity? 7. (MOE) To what extent has AMANAT’s support to the School Management Shuras strengthened MOE capacity to implement anti-corruption reforms? a. Successes of the pilot? Challenges and lessons learned? 8. Overall did AMANAT’s contributions meet your expectations? a. What contributions were most valuable? Least valuable? 9. Did AMANAT’s contributions addressed national priorities? What priority do you see most significant? 10. To what extent has [Ministry] inputted into AMANAT activity design? 11. Can you describe the nature of [Ministry]’s working relationship with AMANAT? a. Were there any problems with assistance, and can you give any suggestions on how to address these, or any challenges that arose in cooperation with the USAID/AMANAT program? EQ 2: WHAT ARE THE UNINTENDED CONSEQUENCES (POSITIVE OR NEGATIVE) OF THOSE INTERVENTIONS? HOW ARE THE CONSEQUENCES, IF ANY, AFFECTING ANTI￾CORRUPTION EFFORTS IN AFGHANISTAN? 1. Has [Ministry] detected any unintended consequences from AMANAT’s intervention, positive or negative? MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-12 a. If so, to what extent are these unintended consequences positively impacting anti￾corruption efforts? Negatively? EQ 3: HOW AND TO WHAT EXTENT DO AMANAT’S INTERVENTIONS FIT IN WITH THE OVERALL AFGHAN ANTI-CORRUPTION CONTEXT AS REGARDS POLITICAL UNCERTAINTY, LACK OF POLITICAL WILL, INSECURITY AND POST-PEACE SCENARIO. 1. Has turnover or restructuring impacted [Ministry]’s ability to work with AMANAT? a. MOMD / Presidential election 2. How has [Ministry] adapted programming and developed contingencies for the ongoing peace negotiations? a. To what extent will [Ministry]’s work with AMANAT be sustainable in a post-peace environment? 3. In your opinion, how does AMANAT’s programming fit within the broader anti-corruption landscape, as well as [Ministry]’s anti-corruption work? 4. Are there any other factors that could in the future affect process of improving the anti￾corruption policy and legal framework in Afghanistan? 5. What do you see as the most important legislative or policy innovation introduced in the past three years? a. Did the AMANAT program contribute significantly to this innovation? EQ 4: TO WHAT EXTENT ARE AMANAT'S INTERVENTIONS CONTRIBUTING TO CIVIL SOCIETY'S ABILITY TO ADVOCATE FOR REFORM? 1. To what extent has AMANAT increased [Ministry]’s coordination with anti-corruption CS? 2. Does your institution have any important non-governmental partners that have helped in improving public service delivery in Afghanistan? Can you describe this partnership? a. If you had to choose a single example or a “story” that demonstrates this area of success, what would you point to? 3. To what extent has CS involvement with anti-corruption activities increased in the past three years? a. Direct attribution to AMANAT? EQ 5: TO WHAT EXTENT ARE AMANAT’S INTERVENTIONS SUSTAINABLE FOR THE AFGHAN GOVERNMENT AND FOR CSOS? 1. Did AMANAT’s interventions contribute to systemic change that will continue beyond the life of the project? 2. How are [Ministry]’s initiatives to identify vulnerabilities to corruption sustained? 3. How is [Ministry]’s strengthened capacity to implement anti-corruption reforms sustained? 4. Overall, how would you evaluate [Ministry]’s ability to sustain results from AMANAT support following the close of such support? 5. Do you think any achievements/elements are replicable in other parts of the Afghan government? Which ones and how so? MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-13 Other 1. Is there any issue on which we have not touched that you feel should be addressed in the evaluation of AMANAT program? 2. In addition to AMANAT’s work, can you think of any other international donors working with similar anti-corruption activities? If so, are they working with your ministry or other ministries? (Y/N/DK) MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-14 ANNEX C.5 – TOOL FOR CSOS (KII) Introduction a. Name, organization b. Organization mission and grant activity c. Where do you see the organization in 5 years? EQ 1: HOW AND TO WHAT EXTENT DO AMANAT INTERVENTIONS CONTRIBUTE TO REDUCED CORRUPTION IN AFGHAN GOVERNMENT INSTITUTIONS AND IMPROVE PUBLIC SERVICE DELIVERY BY THE AFGHAN GOVERNMENT? WHAT INTERVENTIONS WORK? WHY? WHAT INTERVENTIONS DO NOT WORK? WHY? 1. What mechanisms does your organization and grant have in place to monitor and evaluate progress of its anti-corruption interventions? 2. To what extent have GIRoA anti-corruption activities and reforms changed over the past 3 years? Positive developments? Negative developments? 3. Have communications with GIRoA and public officials improved over the grant period? How so? a. Have contacts made with GIRoA and public officials continued? 4. Whether your relationship with the GIRoA involves citizen services, or policy consultation with CSOs or any new or changed relationship with GIRoA counterpart, do you expect these changes to be permanent? a. What are the factors that determine whether changes endure? EQ 2: WHAT ARE THE UNINTENDED CONSEQUENCES (POSITIVE OR NEGATIVE) OF THOSE INTERVENTIONS? HOW ARE THE CONSEQUENCES, IF ANY, AFFECTING ANTI￾CORRUPTION EFFORTS IN AFGHANISTAN? 1. Has AMANAT support contributed to any unintended consequences affecting anti￾corruption efforts? a. Generally? To your organization specifically? b. If so, to what extent are unintended consequences positively impacting anti￾corruption efforts? Negatively? EQ 3: HOW AND TO WHAT EXTENT DO AMANAT’S INTERVENTIONS FIT IN WITH THE OVERALL AFGHAN ANTI-CORRUPTION CONTEXT AS REGARDS POLITICAL UNCERTAINTY, LACK OF POLITICAL WILL, INSECURITY AND POST-PEACE SCENARIO. 1. Has government turnover or restructuring impacted your organization’s ability to work with GIRoA? a. MOMD / Presidential election 2. How has your organization adapted programming and developed contingencies for the ongoing peace negotiations? a. To what extent will your organization’s work with AMANAT be sustainable in a post-peace environment? MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-15 3. In your opinion, how does AMANAT’s programming fit within the broader anti-corruption landscape, as well as your organization’s anti-corruption work? EQ 4: TO WHAT EXTENT ARE AMANAT'S INTERVENTIONS CONTRIBUTING TO CIVIL SOCIETY'S ABILITY TO ADVOCATE FOR REFORM? 1. To what extent has AMANAT work to mobilize your organization and other CSOs strengthened the overall CS’ ability to advocate for reform? a. (if relevant) To what extent did the groundwork for the CSCC creation contribute? Shortcomings? 2. (If relevant – participant) To what extent did watchdog trainings strengthen your organization and overall CS ability to advocate for reform? a. What has been the contribution of the budget process and oversight trainings 3. (If relevant – participant) To what extent did legislative process (and related) trainings strengthen [CSO] and overall CSG capacity to analyze shortcomings in legislation? 4. (If relevant – participant) To what extent did social auditing/citizen monitoring trainings strengthen [CSO] and overall CS’ ability to advocate for reform? 5. (If relevant – participant) To what extent did strategic communications training and campaign support strengthen your organization and overall CS’ ability to advocate for reform? 6. (If relevant – grantee) Overall, how has participating in the grants program strengthened your organization’s ability to advocate for reform? a. How has proposal development and related subject matter training contributed to this capacity? 7. Has any of the assistance or training provided by AMANAT helped you to better perform your own role and functions? If yes, was this primarily in terms of: a. Management b. budgeting and planning c. policy analysis d. other technical roles and functions e. leadership and communication f. other? 8. If AMANAT advisory or technical staff worked directly with your organization, how useful was this assistance? Very useful ……. Somewhat useful ……. Only marginally useful …… Notes: EQ 5: TO WHAT EXTENT ARE AMANAT’S INTERVENTIONS SUSTAINABLE FOR THE AFGHAN GOVERNMENT AND FOR CSOS? 1. How are your organization’s strengthened capabilities to assess and improve anti￾corruption efforts sustained? 2. Overall, how would you assess your organization’s ability to sustain results from AMANAT support after the close of AMANAT support? 3. If AMANAT were to continue to another grant cycle, what sort of support would be most valuable to your organization? MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-16 Other: 1. Is there any issue on which we have not touched that you feel should be addressed in the evaluation of AMANAT project? 2. If AMANAT were to provide additional support to grantees, which areas would be most important and useful? 3. Did your organization collaborate in any anticorruption related conferences or events that were arranged by AMANAT staff? Please describe: MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-17 ANNEX C.6 – TOOL FOR CSOS (FGD) General questions: 1. Introductions a. Name, organization b. Organization mission and grant activity c. Where do you see the organization in 5 years? 2. Warm-up a. What news or stories related to anti-corruption have you seen in the past week? b. How are corruption related topics portrayed in the Afghan media? 3. Overall perceptions a. What are Afghanistan’s biggest needs to combat corruption? b. What is the role of CS in combatting corruption? c. How can CS best work with GIRoA to combat corruption? EQ 1: HOW AND TO WHAT EXTENT DO AMANAT INTERVENTIONS CONTRIBUTE TO REDUCED CORRUPTION IN AFGHAN GOVERNMENT INSTITUTIONS AND IMPROVE PUBLIC SERVICE DELIVERY BY THE AFGHAN GOVERNMENT? WHAT INTERVENTIONS WORK? WHY? WHAT INTERVENTIONS DO NOT WORK? WHY? 1. Thinking of the anti-corruption activities your organization has implemented, how can CSOs determine whether an activity contributes to corruption reduction? a. What mechanisms are in place on your specific intervention? 2. When you consider anti-corruption in activities in Afghanistan, what characteristics do successful activities share? And unsuccessful activities? 3. Are there any GIRoA activities that are particularly successful in combatting anti￾corruption? Unsuccessful? a. [Moderator – probe awareness and knowledge of AMANAT GIRoA activities] 4. Overall, how would you all characterize the working relationships between your CSOs and international donors? AMANAT specifically? a. [Moderator – probe on how AMANAT/donors could improve working relationship] EQ 2: WHAT ARE THE UNINTENDED CONSEQUENCES (POSITIVE OR NEGATIVE) OF THOSE INTERVENTIONS? HOW ARE THE CONSEQUENCES, IF ANY, AFFECTING ANTI￾CORRUPTION EFFORTS IN AFGHANISTAN? 1. In your experience, have international development interventions had unintentional consequences, both positive and negative? a. If so, explain? 2. And have there been any anti-corruption interventions that had unintentional consequences, both positive and negative? a. If so, explain b. If so, how have these consequences impacted anti-corruption efforts c. [Moderator – probe around AMANAT activities] MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-18 EQ 3: HOW AND TO WHAT EXTENT DO AMANAT’S INTERVENTIONS FIT IN WITH THE OVERALL AFGHAN ANTI-CORRUPTION CONTEXT AS REGARDS POLITICAL UNCERTAINTY, LACK OF POLITICAL WILL, INSECURITY AND POST-PEACE SCENARIO. 1. Thinking to your programs and activities, how can CSOs mitigate the impact of insecurity on programming? a. [Moderator – probe specific actions taken] 2. The Afghan government has undergone numerous changes over the past three years. What are some of these changes and how have they impacted anti-corruption programming broadly? Specific to your CSOs? 3. How do you believe the peace process and a post-peace scenario will impact the CS broadly? a. And specifically, the anti-corruption focused element of CS? b. What are some actions CSOs can take to adapt programming and develop contingencies for a post peace scenario? EQ 4: TO WHAT EXTENT ARE AMANAT'S INTERVENTIONS CONTRIBUTING TO CIVIL SOCIETY'S ABILITY TO ADVOCATE FOR REFORM? 1. Mapping exercise a. [Moderator – divide respondents, provide paper/pen and instruct to map stakeholders working on anti-corruption] b. [Discuss network linkages, probe on identifying where AMANAT has strengthened network linkages] c. What actions are needed to further strengthen the CS aspect of anti-corruption stakeholders? i. [Moderator – probe on CSCC] 2. (If relevant – participants/partners) To what extent did AMANAT trainings strengthen your CSO/CSG’s ability to advocate for anti-corruption reform? a. [Moderator – probe on budget/oversight/watchdog training, legislative process training, social audit/citizen monitoring trainings, strategic communications trainings] b. How can future programs such as AMANAT increase CSO/CSG ability to advocate for anti-corruption reform? 3. (If relevant – grantees) To what extent did the AMANAT grant program strengthen your CSO’s ability to advocate for anti-corruption reform? a. What elements of support from AMANAT do your interventions share? i. [Moderator – probe, grant proposal training, subject matter training, etc.] b. How could AMANAT improve the grants program? What would you change and why? EQ 5: TO WHAT EXTENT ARE AMANAT’S INTERVENTIONS SUSTAINABLE FOR THE AFGHAN GOVERNMENT AND FOR CSOS? 1. How can CSOs develop sustainability planning into their program design? a. [Moderator – probe practical examples] MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-19 2. How has AMANAT support helped your CSOs incorporate sustainability planning into program design? 3. What are your organizations’ plans to continue work and maintain results following the close of AMANAT support? MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-20 ANNEX C.7 – TOOL FOR GRANTEE BENEFICIARY General questions: 1. What is your profession? 2. How long have you served in this position? 3. What is your role with [Grantee]? 4. How did you gain this role? EQ 3: HOW AND TO WHAT EXTENT DO AMANAT’S INTERVENTIONS FIT IN WITH THE OVERALL AFGHAN ANTI-CORRUPTION CONTEXT AS REGARDS POLITICAL UNCERTAINTY, LACK OF POLITICAL WILL, INSECURITY AND POST-PEACE SCENARIO. 1. How does the program you participate in fit within the broader anti-corruption landscape? a. Does it play a prominent role in fighting anti-corruption in your area, or a small role compared to other activities? EQ 4: TO WHAT EXTENT ARE AMANAT'S INTERVENTIONS CONTRIBUTING TO CIVIL SOCIETY'S ABILITY TO ADVOCATE FOR REFORM? 1. What types of training or support have you received through participating in this program? 2. How have these trainings improved your ability to monitor corruption issues and advocate for reform? 3. Did this training or support meet your expectations? a. What aspects of the trainings or support would you improve? 4. Overall, how has the program aided your ability to monitor corruption issues and advocate for reform? a. Can you tell me of any examples of how you and your colleagues contributed to anti-corruption due to this program? EQ 5: TO WHAT EXTENT ARE AMANAT’S INTERVENTIONS SUSTAINABLE FOR THE AFGHAN GOVERNMENT AND FOR CSOS? 1. When the program ended/ends, have you or your colleagues continued your anti￾corruption work? a. How so? b. How is this work supported? 2. How can future versions of the program assist your ability to continue the work once the program starts? Other questions 1. In your experience, are there lessons from other projects/programs operated by other donors in Afghanistan that AMANAT can incorporate into their programming? 2. Are there any other areas you would like to bring up? MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-21 ANNEX C.8 – TOOL FOR OTHER DONOR AGENCY General questions: 1. Can you provide me at overview of your agency’s program in Afghanistan? What are the overall objectives and what are the chief sectors in which the program is engaged? Some other background: ● Does (your organization, named) operated through direct assistance by employees of the organization, contractor or grantee implementers, or other mechanism? Are technical assistance providers resident in Afghanistan, short term advisors, or both? ● Do you have one or more local grantees or contractors that carry out activities? ● If your organization provides training, is this conducted primarily in-country, or is training conducted outside of Iraq? 2. What areas, sectors, or activities does your program share with other donor agencies either objectives, program areas, or specific Afghan counterparts and stakeholder organizations? ● Are points of contact primarily with the Danish Embassy or with its implementing partners? ● Do you or your implementers also make frequent contact with contractor staff of the USAID/AMANAT program? ● Is joint attendance at public events—e.g., conferences held together with Afghan counterparts – a normal activity? How often? ● Regarding Afghan counterparts, have you or officers in your organization also held meetings jointly with local counterparts? Are these primarily GIRoA representatives or private sector or civil society organizations? 3. What were the primary purposes of meetings with other development agencies and their implementing partners? Coordination of activities? To share implementation issues? To discuss future plans and areas of anticipated engagement? To discuss the relationships of the Iraqi counterpart? Other? 4. Where joint meetings with development agencies, together with Afghan counterparts or stakeholders were held, what were the purposes of the meeting? 5. Were there any common program areas where a policy change or piece of legislation was required for progress? If yes, did you organization and USAID (for instance) coordinate for purposes of policy dialogue? 6. If there were any issues needing resolution, were these satisfactorily resolved in meetings with USAID and other development agencies? In meetings with Afghan counterparts and stakeholders? Are there any examples of resolution of issues that are illustrative of such relations? 7. Where your organization may have had a long-term advisory role, or even resident advisors MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-22 working within an Afghan government or private sector organization, were USAID staff and implementers aware of your organization’s role? 8. Were necessary communications with your organization made to coordinate each donor’s role? 9. Did you see any issues operating in Afghanistan alongside one or another USAID program, for example, where USAID was making a major investment-- human capacity, education, health, or other sector? 10. Is there any area where you see your organization and USAID/other donor agencies to have jointly contributed to a major advance or achievement? 11. What would you see as any internal or external factors that could affect long-term sustainability? Other 1. Are there any other areas you would like to bring up? MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | C-23 ANNEX C.9 – TOOL FOR AD HOC CONVERSATION CSO Team 1. Can you walk us through the grant application process, as its set-up in 2018 and 2019? a. Process reformed from 2 stage to 1 stage (from grantee) 2. How were grantees identified for soliciting applications? 3. How were grantees evaluated? 4. What transparency mechanisms were in place – announcing RFPs, announcing awardees, etc.? 5. Regarding the grant structure, can you elaborate on the following: a. Period of grant performance and why it was chosen? b. Grant budget ceiling – why it was chosen originally and why it was raised? 6. Has there been any discussion or consideration of alternative funding mechanisms, such as core funding versus activity funding? 7. Has the CSO team made any effort to coordinate AMANAT activities with other donors conducting similar activities with CSO? 8. Areas for change to engagement or grant process? PS Liaison 1. What was the process for selecting implementing partners or STTAs? 2. Has AMANAT faced challenges with any IPs/STTAs? a. What was the nature of these challenges and how were they rectified? b. How can they be mitigated in the future? 3. Can you walk us through the process for embedding technical staff within the ministries/IA directorates? 4. What mechanisms exist to assess ability for the Afghan government to sustain results during and following AMANAT support? 5. MEC cooperation MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | D-1 ANNEX D – LIST OF KEY DOCUMENTS REVIEWED The evaluation team reviewed 121 key documents, listed below. Title Type AMANAT Y4 Assignment Work Plan (AWP) Contract Asan Khedmat Annex A - Scope of Work Contract Contract - Mod1 Contract Contract - Mod3 Contract Contract - Mod5 - Fully Executed Contract Contract - Mod7 Contract Contract - Original (AID-306-H-17-00003) Contract 2020 IPDPG Overview Donor material MSI WB-ARTF TPM Completion Report (Dec 2019) Donor material TAGHIR Overview Donor material WB Internal Protocols for engagement Donor material USAID/Georgia: Georgia Business Climate Reform Final Report. Donor material (other) 201227 AMANAT Contact List Implementation monitoring AMANAT Activities Map Implementation monitoring AMANAT AMELP Final w/ COVID19 Indicators Implementation monitoring AMANAT M&E Database Implementation monitoring 1.2.2.4 CLA Progress Report for Y2 Program progress report 1.2.2.4 CLA Progress Report for Y2 (Updated) Program progress report MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | D-2 1.2.2.4 CLA Progress Report for Y3 Program progress report F.4.C.2 FY2018 Q1 QPR Program progress report F.4.C.2 FY2018 Q2 QPR Program progress report F.4.C.2 FY2018 Q3 QPR Program progress report F.4.C.2 FY2019 Q1 QPR Program progress report F.4.C.2 FY2019 Q2 QPR Program progress report F.4.C.2 FY2019 Q3 Financial Report Program progress report F.4.C.2 FY2019 Q3 QPR Program progress report F.4.C.2 FY2019 Q3 QPR Success Story Program progress report F.4.C.2 FY2020 Q1 QPR Program progress report F.4.C.2 FY2020 Q2 Success Story 1 Program progress report F.4.C.2 FY2020 Q2 Success Story 2 Program progress report F.4.C.2 FY2020 Q2 Success Story 3 Program progress report F.4.C.2 FY2020 Q2 Success Story 4 Program progress report F.4.C.2 FY2020 Q3 QPR Program progress report F.4.C.4 Annual Report 2017 - 2018 (Y1) Program progress report F.4.C.4 Annual Report 2018 - 2019 (Y2) Program progress report F.4.C.4 Annual Report 2019 - 2020 (Y3) Program progress report F.4.C.4 Annual Report 2019 - 2020 (Y3) - Success Story - Internal Audit Program progress report F.4.C.4 Annual Report 2019 - 2020 (Y3) - Success Story - Virtual Grants Program progress report F.4.C.4 Annual Report 2019 - 2020 (Y3) - Success Story Covid19 Program progress report 1.1.1.1 Institution Function Selection Report Y1 Task deliverable 1.1.1.1 iPEA Report - MOE Task deliverable MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | D-3 1.1.1.1 iPEA Report - MOHE Task deliverable 1.1.1.1 iPEA Report - MOMD Task deliverable 1.1.1.1 iPEA Report - MOPH Task deliverable 1.1.1.1 iPEA Report - MORR Task deliverable 1.1.2.2 VCA Reports for Selected Institutions - MOHE Final Task deliverable 1.1.2.2 VCA Reports for Selected Institutions - MOPH Final Task deliverable 1.1.3.1 PEA (Updated) - MOE Task deliverable 1.1.3.1 PEA (Updated) - MOHE Task deliverable 1.1.3.1 PEA (Updated) - MOMD Task deliverable 1.1.3.1 PEA (Updated) - MOPH Task deliverable 1.1.3.1 PEA (Updated) - MORR Task deliverable 1.1.3.2 VCA Reports for Selected Institutions (MOPH/ MOHE) Task deliverable 1.1.3.3 IA-CM Report - MOE Task deliverable 1.1.3.3 IA-CM Report - MOHE Task deliverable 1.1.3.3 IA-CM Report - MOMD Task deliverable 1.1.3.3 IA-CM Report - MOPH Task deliverable 1.1.3.3 IA-CM Report - MORR Task deliverable 1.1.3.3 IA-CM Report/Update - MOE Task deliverable 1.1.3.3 IA-CM Report/Update - MOHE Task deliverable 1.1.3.3 IA-CM Report/Update - MOPH Task deliverable 1.1.3.3 IA-CM Report/Update - MORR Task deliverable 1.2.1.4 iPEA (Updated) - MDD Task deliverable 1.2.1.4 iPEA (Updated) - MOE Task deliverable 1.2.1.4 iPEA (Updated) - MOHE Task deliverable MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | D-4 1.2.1.4 iPEA (Updated) - MOPH Task deliverable 1.2.1.4 iPEA (Updated) - MORR Task deliverable 1.2.2.1 BPR and CRMP Report (MOPH/MOE/MORR) Task deliverable 2.1.1.6 APS Grants Released for Y1 Task deliverable 2.1.1.2 Watchdog Training - Budget Process Agenda Task materials 2.1.1.2 Watchdog Training - Budget Process Group Activity Task materials 2.1.1.2 Watchdog Training - Budget Process PP Task materials 2.1.1.2 Watchdog Training - Budget Process Pre/Post Test Task materials 2.1.1.2 Watchdog Training - Fix Rate PP Task materials 2.1.1.3 Legislation Training - Legal Research for CSOs Task materials 2.1.1.3 Legislation Training - Legislative Drafting for CSOs Task materials 2.1.1.3 Legislation Training - Legislative Process for CSOs Task materials 2.1.1.3 Legislation Training - Working Group Activity Task materials 2.1.1.5 Communications Strategy Handbook for CSOs Task materials 2.1.2.6 Grants Workshop - 3.1 Application Development Task materials 2.1.2.6 Grants Workshop - 3.2 Grant Types Task materials 2.1.2.6 Grants Workshop - 3.3 Attachment A Budget Task materials 2.1.2.6 Grants Workshop - 3.4 Attachment B Implementation Plan Task materials 2.1.2.6 Grants Workshop - 3.5 Attachment C M&E Plan Task materials 2.1.2.6 Grants Workshop - 3.6 APS QA Task materials 2.1.2.6 Grants Workshop - 3.7 Grant Application Task materials 2.1.2.6 Grants Workshop - 4.1 Concept Development Task materials 2.1.2.6 Grants Workshop - 4.2 Agenda Task materials 2.1.2.6 Grants Workshop - 4.3 Workshop Evaluation Form Task materials MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | D-5 2.1.2.6 Grants Workshop - 4.4 Annex1 Concept Form Task materials 2.1.2.6 Grants Workshop - 4.5 Annex2 Estimated Budget Task materials 2.1.2.6 Grants Workshop - 4.6 Annex3 Pre-award Certifications Task materials Anti-Corruption Dialogue Manual Task materials Legislation Drafting and Role of CSOs Training Manual Task materials Monitoring Government Institutions and Policies Training Manual Task materials Social Audit Training Manual Task materials Watchdog Education Training Manual Task materials 1.1.1.2 VCA Planning Report for Selected Institutions Task progress report 1.1.1.4 MEC Support Planning Report Task progress report 1.1.2.4 MEC Support Progress Report (Y2) Task progress report 1.2.1.2 CRMP Implementation Progress Report (MOPH/ MOE) Task progress report 1.2.1.3 Subnational Progress Report: Selection, Planning and Implementation Task progress report 1.2.2.2 CRMP Implementation Progress Report (MOPH/ MOE/ MORR) Task progress report 1.2.2.3 Subnational Progress Report: Selection, Planning and Implementation Task progress report 1.2.3.1 BPR and CRMP Report (MOPH/ MOHE/MORR) - Update Task progress report 2.1.1.1 Anti-Corruption SCO Networking Report Task progress report 2.1.2.6 Grant Workshop Report Task progress report 2.1.2.6 Grants Awarded for Year 2 Task progress report 2.1.2.6 [Redacted] Grants Workshop Report Task progress report 2.1.3.6 APS Grants Released for Y3 Task progress report MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | D-6 2.1.3.6 Grant Proposal Design Workshop Report Task progress report Anti-Corruption Dialogue Training Report Task progress report CSO Watchdog Education Training (MOE) Report Task progress report F.4.C.2 FY2020 Q2 Annex - Grant Final Report - AALWDO Task progress report F.4.C.2 FY2020 Q2 Annex - Grant Final Report - Faryad Radio Task progress report F.4.C.2 FY2020 Q2 Annex - Grant Final Report - ORSTW Task progress report F.4.C.2 FY2020 Q2 Annex - Grant Final Report - WISE Task progress report Legislation Process Training Report Task progress report Monitoring Government Institutions and Policies Training Report Task progress report 1.1.3.3 IA-CM Report/Update - MOMD Task progress report F.4.C.2 FY2020 Q2 QPR Program progress report MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | E-1 ANNEX E – INTERVIEW ROSTER The following table contains the type of respondent interviewed for a KII, or FGD, where noted, and the date of the interview. AMANAT (IP/STTA) 25-Jan-21 AMANAT (IP/STTA) 02-Feb-21 AMANAT (IP/STTA) 01-Feb-21 AMANAT (IP/STTA) 03-Feb-21 AMANAT (IP/STTA) 02-Feb-21 AMANAT HQ Staff 04-Feb-21 AMANAT Program Staff 18-Jan-21 AMANAT Program Staff 18-Jan-21 AMANAT Program Staff 18-Jan-21 AMANAT Program Staff 18-Jan-21 AMANAT Program Staff 21-Jan-21 AMANAT Program Staff 24-Jan-21 AMANAT Program Staff 24-Jan-21 AMANAT Program Staff 01-Feb-21 AMANAT Program Staff 01-Feb-21 AMANAT Program Staff 03-Feb-21 AMANAT Program Staff 03-Feb-21 AMELA Program Staff 27-Jan-21 CSO FGD 1 28-Jan-21 CSO FGD 2 13-Feb-21 Donor (Danish Foreign Affairs) 25-Jan-21 Donor (WBG) 27-Jan-21 MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | E-2 GIRoA (Asan Khedmat) 24-Jan-21 GIRoA (MEC) 03-Feb-21 GIRoA (MEC) 04-Feb-21 GIRoA (MMD) 31-Jan-21 GIRoA (MOE) 23-Jan-21 GIRoA (MOHE) 26-Jan-21 GIRoA (MOPH) 28-Jan-21 GIRoA (MORR) 24-Jan-21 GIRoA (SAO) 27-Jan-21 Grantee (AALWDO) 20-Jan-21 Grantee (Ariana Radio and Television Network [ATN]) 21-Jan-21 Grantee (Class Vision Management Consultancy Services [CVMCS]) 19-Jan-21 Grantee (FARYAD) 20-Jan-21 Grantee (Human Rights Research and Advocacy Consortium [HRRAC]) 20-Jan-21 Grantee (ORSTW) 20-Jan-21 Grantee (WISE) 26-Jan-21 Grantee Beneficiary (AALWDO) 04-Feb-21 Grantee Beneficiary (ORSTW) 01-Feb-21 Grantee Beneficiary (ORSTW) 03-Feb-21 Grantee Beneficiary (WISE) 01-Feb-21 Grantee Beneficiary (WISE) 01-Feb-21 Grantee Beneficiary (WISE) 02-Feb-21 GIRoA (ACC mic.) 13-Feb-21 MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | F-1 ANNEX F – CHANGES TO SCOPE OF WORK The changes to the scope of work were primarily due to COVID restrictions and travel policy changes. The in-country trip for the Evaluation Team Leader and the Evaluation Deputy Team Leader had to be shortened, with those days changed to be completed remotely. Given the travel restrictions, some of the interviews had to be conducted by phone or video conference. The in￾brief and mid/exit brief meeting with USAID also had to be conducted remotely instead of in￾person. The evaluation team conducted the data analysis remotely as well. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | G-1 ANNEX G – STATEMENT OF DIFFERENCES Not applicable. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | H-1 ANNEX H – SOURCES OF INFORMATION The evaluation team utilized the sources of information identified under Annexes D and E. The team conducted a multidimensional evaluation, utilizing comprehensive program data and reporting, along with extensive primary data collection utilizing multiple methods. The team utilized over 150 external document-based data sources and approximately 50 primary data sources. MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | I-1 ANNEX I – DISCLOSURES OF CONFLICTS OF INTEREST [Redacted] MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | J-1 ANNEX J – SUMMARY OF EVALUATION TEAM MEMBERS’ QUALIFICATIONS Evaluation Team Leader – George Asatiani Senior Evaluation Expert George Asatiani has more than 25 years of experience focusing on democracy and governance, including anti-corruption, institutional development, results-based performance management, and economic development in the Middle East, former Soviet Union, and Central and South Asia. Over twenty years’ experience with the design and implementation of monitoring & evaluation and performance management of USAID contracts and cooperative agreements, including his oversight of monitoring and evaluation initiatives for the USAID’s Afghanistan Parliamentary Assistance Program and for the EU’s Increasing Afghanistan’s Capacity for Sustainable Legal Reform Program that worked closely with Afghanistan’s Ministry of Justice, the Attorney General's Office and the Supreme Court. Adept at developing quality indicators, baselines and targets, designing and administering data collection instruments, and preparing evaluation reports. As key decision-maker of three consecutive USAID-funded performance management projects in Iraq with a total budget exceeding US$65 million, Mr. Asatiani managed M&E projects that evaluated USAID funded programs with a combined value of over US$1 billion. As the Technical Director of the Performance Evaluation Division of the US-based consulting firm specializing in MEL, Mr. Asatiani served as Project Director of performance evaluations implemented in various countries in Africa, Asia, Europe, and Caribbean, and oversaw evaluation teams and the evaluation supporting teams at the corporate headquarters. Mr. Asatiani has advanced computer skills that include data development applications and statistical analysis. He is an excellent communicator, equally comfortable with instructing project staff, delivering trainings to USAID officials, and engaging with top-level host-country counterparts. Evaluation Deputy Team Leader – Alexander Moler Alexander Moler is a social scientist experienced in designing and implementing national-scale quantitative and qualitative research in insecure or conflict-affected environments, targeting hard￾to-reach populations. Mr. Moler brings over six years of experience working on USAID, US Department of State and other government-funded projects, and extensive experience in Afghanistan, including long-term in-country residency. Mr. Moler works across the research lifecycle, and has extensive experience developing project plans, methodology (including developing indicators and tools), conducting analysis and leading public-facing reporting. Mr. Moler’s project experience includes leading impact assessments, Monitoring Evaluation and Learning, program evaluations and population studies encompassing mixed research methodologies and data analysis. Mr. Moler has extensive experience working on governance related projects, highlighted by developing and implementing a multi-year tracking program of national perceptions on levels and factors of corruption, transparency and government performance, for a Ministry-plus office in Iraq’s government. Mr. Moler has also implemented evaluations in governance and anti-corruption adjacent sectors such as countering violent extremism and community security. Evaluation Team Member – is a seasoned international development evaluator and professional. brings 18 years of experience in data collection, design and facilitation of focus groups, MID-TERM PERFORMANCE EVALUATION OF AFGHANISTAN’S MEASURE FOR ACCOUNTABILITY AND TRANSPARENCY (AMANAT) CONTRACT | J-2 data analysis, and report writing. has worked with a variety of donor organizations, including USAID, on design, implementation, and evaluation of development projects. She is well￾versed in gender and anticorruption issues in Afghanistan and is connected to an extensive development, civil society, and public sector professional networks. Evaluation Team Member – brings over 25 years of experience in development in Afghanistan, including serving as the interim Executive Director of the leading anti-corruption agency in Afghanistan - Monitoring and Evaluation Committee on anti-corruption (MEC), whose practices included Vulnerability to Corruption Assessment (VCA), monitoring and evaluation of corruption level and anti-corruption activities in various ministries and independent government agencies and communication and coordination with other anti-corruption institutions including HOO, TA in Afghanistan. served as the Governance Pillar for the Afghanistan National Development Strategy, which supported and monitored the development process of the first national anti-corruption strategy of Afghanistan in 2007. He served as the Program Director for the Afghanistan Justice Sector Reform working with justice institutions, including the Ministry of Justice, the Supreme Court, and the Attorney General’s office, to improve their capacity to perform their constitutionally mandated services focusing on improvement of mechanisms to identify possible corruption and institute anti-corruption measures across institutions. has been engaged in evaluation of more than 20 major national programs funded by the international community, the UN or the government of Afghanistan, such as the UNDP’s Afghanistan Sub-national Governance Program (ASGP – I and II). He is an experienced evaluator, familiar with quotative and qualitative methodologies design and deployment.