This publication was produced at the request of the United States Agency for International Development. It was prepared independently by the Indonesia Monitoring Evaluation and Learning Platform (MEL-P), managed by Panagora Group. JOINT MIDTERM PERFORMANCE EVALUATION OF USAID/ASEAN PROSPECT, IGNITE, AND API ACTIVITIES FINAL REPORT MARCH 2022 PHOTO CREDITS: ASEAN SECRETARIAT (ASEC) JOINT MIDTERM PERFORMANCE EVALUATION OF USAID/ASEAN PROSPECT, IGNITE, AND API ACTIVITIES FINAL REPORT March 2022 Evaluation Mechanism Number: GS-00F-210DA/72049720M00001 Indonesia Monitoring Evaluation and Learning Platform (MEL-P) Cover photo by ASEAN Secretariat Authors Francis Ogojo Okello, Ph.D., Team Leader Saut Sagala, PhD Ruth Elisabeth, PhD Dian Anshar, MA Dian Rachmawati, BA FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | i ABSTRACT Background. This report presents key findings of a joint mid-term evaluation of three Association of Southeast Asian Nations (ASEAN) USAID-funded activities. ASEAN is the primary multilateral institution for regional cooperation, facilitating political-security, economic, and socio-cultural integration in the Southeast Asia region. The U.S. Government is a long-term partner to ASEAN’s aims to achieve sustainable and inclusive growth, and rules-based systems necessary for a prosperous Southeast Asia. Purpose, Questions, and Methods. This evaluation assessed the implementation effectiveness of three USAID/ASEAN projects—PROSPECT, IGNITE, and API, and their results at midterm. Evaluation questions examined project strategies and approaches, theoretical models, progress towards expected results, challenges, and USAID’s support. Evaluation methods included desk review, key informant interviews, and secondary data analysis. Findings. PROSPECT. PROSPECT made considerable progress in implementing planned activities at mid￾term. Significant progress has been made to increase ASEAN’s capacity to address non-traditional security threats, including trafficking in persons, countering violent extremism, wildlife trafficking, and disaster management; and expanding opportunities for women and vulnerable groups including gender mainstreaming, Women Peace and Security, and CSO engagement. USAID has effectively supported PROSPECT implementation with ASEAN regional bodies, but more is needed to strengthen project bilateral engagement with ASEAN member states (AMS). However, PROSPECT’s performance in promoting responsive and transparent government, including judicial cooperation, anti-corruption, and promoting human rights, is sub-optimal. The project also faces challenges from its broad scope and the multiple needs of ASEAN. API. Project performance shows mixed results. Under the technical support for ASEAN Economic Community (AEC) Policies for ASEAN Single Window (ASW) and Digital Data Governance (DDG), API has identified the critical needs of seven out of the planned 10 ASEAN member states. The best project achievement is its technical support to the Philippines Bureau of Customs. Delay in project endorsement by ASEAN, bureaucratic challenges within the AMS, and travel restrictions due to COVID-19 have limited project performance. IGNITE. IGNITE has mainly supported ASEAN’s priorities in trade facilitation and the digital economy. Accomplished activities include technical support for ASW back-end systems design, designing a study on regional and global best practices for reducing bottlenecks to low-value shipments, and facilitating workshops for harmonizing standards across ASEAN member states. IGNITE also supported ASEAN’s digital economy workstream on regulatory reform, cyber-security, privacy regulations, and expanding broadband access. Under ASEAN’s shared priorities, IGNITE supported the science prize and STEM webinar for women and Women Entrepreneurs Amplifying Ventures and Economies (WEAVE). Conclusion. The joint mid-term evaluation findings provide valuable insights into the performance of USAID/ASEAN projects at mid-term. These projects have collectively contributed to strengthening systems, regulations, and human capacity of the ASEAN regional bodies and the AMS. The project designs are relevant to ASEAN’s needs and effectively address those needs. Despite their successes in the mid-term, the projects face multiple implementation challenges related to the strength of their engagement with ASEAN, the lack of capacity by ASEAN to enforce policy ii | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION implementation by the AMS, budget constraints, and the local AMS capacity limitations and bureaucracies. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | iii TABLE OF CONTENTS ABSTRACT ...............................................................................................................................I LIST OF FIGURES ..................................................................................................................V LIST OF TABLES....................................................................................................................V ACRONYMS...........................................................................................................................VI EXECUTIVE SUMMARY.......................................................................................................IX EVALUATION PURPOSE AND EVALUATION QUESTIONS....................................................................IX DESIGN, METHODS, AND LIMITATIONS......................................................................................................IX KEY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS............................................................IX EVALUATION PURPOSE, OBJECTIVES, AND EVALUATION QUESTIONS...............1 EVALUATION PURPOSE AND OBJECTIVES.................................................................................................... 1 EVALUATION QUESTIONS................................................................................................................................... 1 BACKGROUND.......................................................................................................................4 EVALUATION METHODS AND LIMITATIONS..............................................................10 DOCUMENT REVIEW...........................................................................................................................................10 PRIMARY DATA.......................................................................................................................................................10 SAMPLING.................................................................................................................................................................11 DATA PREPARATION...........................................................................................................................................11 DATA ANALYSIS.....................................................................................................................................................11 ETHICAL CONSIDERATIONS ............................................................................................................................11 LIMITATIONS ...........................................................................................................................................................12 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS ..........................................13 PROSPECT PROJECT .............................................................................................................................................13 API PROJECT ............................................................................................................................................................40 IGNITE PROJECT.....................................................................................................................................................60 OVERALL CONCLUSIONS .................................................................................................79 CONCLUSIONS ......................................................................................................................................................79 ANNEXES...............................................................................................................................82 ANNEX I: TIMELINE...............................................................................................................................................83 ANNEX II: EVALUATION STATEMENT OF WORK....................................................................................84 iv | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION ANNEX III: EVALUATION DESIGN MATRIX.................................................................................................94 ANNEX IV: DATA COLLECTION AND ANALYSIS TOOLS.................................................................. 112 ANNEX V: SOURCES OF INFORMATION.................................................................................................. 125 FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | v LIST OF FIGURES Figure 1. Map of ASEAN Member States.............................................................................................................. 4 Figure 2. USAID/ASEAN Results Framework...................................................................................................... 5 Figure 3. Implementation Strategies of PROSPECT to Implement Activities............................................20 Figure 4. Policy Continuum: Most Support and Results are at Stage Building Coalition of Stakeholders and Advancing ASEAN Regional Policy ...............................................................................................................21 Figure 5. PROSPECT ToC and Inputs.................................................................................................................28 Figure 6. ASEAN Visions, Priorities, and Sectoral Bodies Work Plans.......................................................29 Figure 7. Procedures of API Endorsement Process .........................................................................................50 Figure 8. Graphical Representation of API Project Activities........................................................................51 Figure 9. API Result Framework ...........................................................................................................................54 Figure 10. IGNITE Assistance Under the Trade Facilitation Workstream.................................................61 Figure 11. IGNITE Assistance under Digital Economy Workstream...........................................................62 Figure 12. The Needs of ASEAN in Shared Priorities and IGNITE's Interventions.................................64 Figure 13. The Most Effective Interventions Under Trade Facilitation Workstream..............................65 Figure 14. The Most Effective Interventions Under Digital Economy Workstream................................66 Figure 15. The Most Effective Initiatives Under STI.........................................................................................67 Figure 16. Challenging and Potentially Effective Interventions.......................................................................69 Figure 17. Impacts of ASW to Consumers, Firms, Exporters, and Workers............................................69 Figure 18. IGNITE Results Framework ...............................................................................................................71 LIST OF TABLES Table 1. Summary of API Sub-Areas and Key Activities.................................................................................... 8 Table 2. PROSPECT Intervention to Increase ASEAN Capacity to Address Non-Traditional Security Threats........................................................................................................................................................................15 Table 3. Interventions to Promote Responsive and Transparent Governance.........................................16 Table 4. PROSPECT’s Interventions to Promote Human Rights and Expand Opportunities for Women and Other Vulnerable Groups..............................................................................................................18 Table 5. Most Successful Interventions at Midterm .........................................................................................22 Table 6. Least Successful Interventions at Midterm.........................................................................................23 Table 7. Results in Increasing Capacity to Address Non-Traditional Security ..........................................30 Table 8. Results in Strengthening ASEAN Capacity to Respond to Emerging Humanitarian Crises....31 Table 9. Results in Increasing ASEAN Capacity to Promote Responsive and Transparent Governance .......................................................................................................................................................................................32 Table 10. Results in Increasing Civil Society Participation in ASEAN..........................................................33 Table 11. Results in Expanding Rights and Opportunities for Women, Youth, and Other Vulnerable and Marginalized Groups........................................................................................................................................34 Table 12. Performance of API in Core Project Indicators..............................................................................44 Table 13. The capacity of Each AMS to Implement DDG ..............................................................................47 Table 14. Impacts of ASW on Trade Costs.......................................................................................................70 vi | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION ACRONYMS ACDD ASEAN Customs Declaration Document ACW ASEAN Committee on Women ACWC ASEAN Commission on the Promotion and Protection of the Rights of Women and Children ACTIP ASEAN Convention Against Trafficking in Persons ADII ASEAN Digital Integration Index AEC ASEAN Economic Community AEQ API Activity Evaluation Question AFMAM ASEAN Foundation Model ASEAN Meeting AGMSF ASEAN Gender Mainstreaming Strategic Framework AHA ASEAN Humanitarian Assistance AHRD ASEAN Human Rights Declaration AIC ASEAN Impact Challenge AICHR ASEAN Intergovernmental Committee on Human Rights AIMD ASEAN Integration Monitoring Directorate AIPA ASEAN Inter-Parliamentary Assembly ALVS ASEAN low-value shipment AMMTC ASEAN Ministerial Meeting on Transnational Crime AMS ASEAN member state API ASEAN-USAID ASEAN Policy Implementation APSC ASEAN Political-Security Community ARIA Asia Reassurance Initiative Act ATIGA ASEAN Trade in Goods Agreement ASCC ASEAN Socio-Cultural Community ASEAN Association of Southeast Asian Nations ASEAN-ISIS ASEAN Institutes of Strategic and International Studies ASEC ASEAN Secretariat ASW ASEAN Single Window AWSC ASEAN-wide Self Certification AYSJC ASEAN Youth Social Journalism Contest AYVC ASEAN Youth Video Contest BOC Bureau of Customs CACJ Council of ASEAN Chief Justices CERITA Community Empowerment for Raising Inclusivity & Trust through Technology Application CLA collaborating, learning, and adapting CO certificate of origin COR Contracting Officer’s Representative FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | vii COSTI Committee on Science, Technology, and Innovation COVID-19 coronavirus disease 2019 CSO civil society organization CVE Countering Violent Extremism DCCP Digital Connectivity and Cybersecurity Partnership DCF Development Cooperation Framework DDG digital data governance DIFAP Digital Integration Framework Action Plan DO Development Objective e-CO electronic commerce EDGE Enhancing Growth and Development through Energy EG economic growth EQ evaluation question EV electric vehicle EYSD Education Youth and Sports Division FOE Freedom of Opinion and Expression FY fiscal year GDP Gross Domestic Product GEQ General Evaluation Question HR human resources HRWG Human Rights Working Group ICT information and communications technology IGNITE ASEAN-USAID Inclusive Growth in ASEAN Through Innovation, Trade, and E-Commerce IP Implementing partner IPS The U.S. Indo-Pacific Strategy ITAN Infrastructure Transaction and Assistance Network J/TIP Office to Monitor and Combat Trafficking in Persons KII key informant interview KPK Corruption Eradication Commission MEL Monitoring, Evaluation, and Learning MEL-P Monitoring, Evaluation, and Learning Platform MRA Mutual Recognition Arrangement MSME micro, small, and medium enterprises NSW National Single Window PCPMD Programme Cooperation and Project Management Division PCRVE Prevent and Counter Radicalization and Violent Extremism PEGD Poverty Eradication & Gender Division PEQ PROSPECT Activity Evaluation Question viii | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION PoA Plan of Action PPR Pandemic Preparedness and Response PROSPECT ASEAN-USAID Partnership for Regional Optimization within the Political￾Security and Socio-Cultural Communities RoL Rule of Law RPoA Regional Plan of Action SEA-PAC Southeast Asia Parties Against Corruption SME small and medium enterprises SOMTC Senior Official Meeting on Transnational Crime SOMY Senior Official Meeting on Youth STEM science, technology, engineering, and mathematics STI science, technology, and innovation TIP trafficking in persons ToC Theory of Change ToR Terms of Reference USAID United States Agency for International Development USG United States Government USO Universal Service Obligation WEAVE Women Entrepreneurs Amplifying Ventures and Economies W-GDP Women’s Global Development and Prosperity Initiative WTO World Trade Organization FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | ix EXECUTIVE SUMMARY EVALUATION PURPOSE AND EVALUATION QUESTIONS EVALUATION PURPOSE, OBJECTIVES, AND QUESTIONS USAID/ASEAN developed this performance evaluation to assess the effectiveness of three USAID/ ASEAN activities: 1. ASEAN-USAID Partnership for Regional Optimization within the Political-Security and Socio-Cultural Communities (PROSPECT) 2. ASEAN-USAID Inclusive Growth in ASEAN Through Innovation, Trade, and E-Commerce (IGNITE) 3. ASEAN Policy Implementation (API) USAID/ASEAN stipulated three specific objectives for this evaluation, to: 1. Review ASEAN activity performance, focusing on project design management and implementation 2. Identify successes and opportunities 3. Determine key lessons learned that can be used to improve program management and inform future programming design The USAID grouped the evaluation questions into four categories: 1. General Evaluation Questions (GEQs) 2. PROSPECT Activity Evaluation Questions (PEQs) 3. API Activity Questions (AEQs) 4. IGNITE Activity Evaluation Questions (IEQs) DESIGN, METHODS, AND LIMITATIONS METHODS The evaluation team conducted this joint midterm evaluation using a concurrent mixed-methods design. Specific methods included a desk review, key informant interviews (KIIs), and secondary data analysis. Evaluation participants in the KII included USAID/ASEAN Contracting Officer’s Representatives (CORs), implementing partner (IP) staff, technical staff, and ASEAN sectoral bodies. The evaluation team combined these methods to examine the appropriateness of activities design, the effectiveness of implementation strategies, assessment of progress toward expected results, and analysis of implementation and results enabling and limiting factors. KEY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS ASEAN-USAID PARTNERSHIP FOR REGIONAL OPTIMIZATION WITHIN THE POLITICAL￾SECURITY AND SOCIO-CULTURAL COMMUNITIES (PROSPECT) PEQ1. HOW HAS PROSPECT ADJUSTED ITS STRATEGY AND APPROACH IN SUPPORTING ASEAN WITH THE CURRENT SET OF ASEAN PARTNERS? WHAT ARE THE SPECIFIC INSTANCES IN WHICH THESE ADJUSTMENTS HAVE BEEN MADE, AND WHY? Key findings PROSPECT has made considerable progress in implementing its planned activities at midterm. The project has undertaken multiple initiatives in workstream I and achieved partial implementation x | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION progress at midterm in workstream II and III. Furthermore, evaluation findings revealed a need for much more support from USAID to support PROSPECT implementation, particularly establishing bilateral networks and linkages with ASEAN member states during start-up and implementation. Conclusions & Recommendations PROSPECT has mainly applied and adapted its strategies and approaches in line with ASEAN principles, practices, and needs. PROSPECT generally consults with ASEAN counterparts to obtain endorsement PROSPECT interventions in workstream I have shown significant progress and brought the most successful results. Recommendations PROSPECT should continue strengthening alignment with ASEAN’s agenda, ensuring buy-in from ASEAN counterparts, and selecting key priority areas with stronger regional/cross-border dimensions. Adjust and adapt its design to accommodate offline and online activities (hybrid) to keep moving forward with the implementation activities. PEQ2. WHAT SUPPORT HAS USAID PROVIDED TO PROSPECT IN ADMINISTRATIVE AND DIPLOMATIC ENGAGEMENT TO SUPPORT PROJECT IMPLEMENTATION AND ASEAN IN DEVELOPING INSTITUTIONAL CAPACITY AND HUMAN CAPITAL TO BETTER ADVANCE SHARED PRIORITIES? Key findings USAID has provided administrative and diplomatic support to PROSPECT in the following ways: (i) Contributing to and participating in the development and approval of PROSPECT work plans, (ii) Helping facilitate relationships with ASEAN high-level officials and relevant agencies within the U.S. Government in Washington, (iii)Securing funding from other potential donors, (iv) Facilitating communication and support with ASEAN’s Secretariat. Conclusions & Recommendations USAID has effectively supported PROSPECT with overall administrative and diplomatic engagement; however, more support is needed to increase the effectiveness of PROSPECT’s support, particularly at country levels. Some additional actions that USAID/ASEAN can take to enhance support to PROSPECT include (i) Further strengthening the relationship between PROSPECT and USAID missions in the AMS, and (ii) Providing additional funding and budget flexibility to accommodate emerging and shifting needs of ASEAN and their regional bodies. PEQ3. HOW EFFECTIVE IS PROSPECT’S THEORY MODEL IN GUIDING THE IMPLEMENTATION OF PROJECT ACTIVITIES TOWARD ACHIEVING EXPECTED RESULTS (DEVELOPMENT HYPOTHESIS, THEORY OF CHANGE, RESULTS FRAMEWORK, OR LOGIC MODEL)? WHY OR WHY NOT? Key findings The TOC effectively guided the implementation of activities as it referred to key action lines in the Blueprints that served as ASEAN’s roadmaps to achieve its Community Vision. PROSPECT’s activities have resulted in establishing a regional policy implementation framework, policy toolkits and guidelines, multi-sectoral platform and coordination mechanism, women’s and CSO’s participation in the ASEAN process are consistent with the TOC and required in the ASEAN community-building process. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | xi Conclusions & Recommendations The ToC is effective in guiding the implementation of interventions at the regional level. Aligning ToC with ASEAN policy framework guidance is key to ensuring that ASEAN’s needs are addressed and met. The ToC and logic model should need to be dynamic and consider other critical factors in ASEAN that might affect achieving results, e.g., the internal political situation in ASEAN member states and pandemics. PEQ4. WHAT CHALLENGES HAS PROSPECT FACED WITH IMPLEMENTING AN ASSISTANCE PROGRAM THAT SPANS SUCH A BROAD SPECTRUM OF ISSUES? Key findings PROSPECT’s broad design creates opportunities for PROSPECT to engage in different areas which resonate with ASEAN’s agenda on cross-sectoral cooperation, enabling PROSPECT to streamline cross-cutting issues across three pillars of ASEAN. However, disadvantages include the need for more time, effort, and resources to bring together various stakeholders to reach a consensus in implementing each activity. Conclusions & Recommendations Despite its challenges, the broad-spectrum programs benefit PROSPECT and ASEAN more than specific projects focusing on particular thematic issues. PROSPECT should enhance other forms of support, for example, exchanging best-practice programs among ASEAN member states and providing dedicated resources for the initiated cross-sectoral collaboration and coordination initiatives in ASEAN, such as in the context of the Bali Work Plan. ASEAN POLICY IMPLEMENTATION (API) AEQ1. WHAT CHALLENGES HAS OR IS API FACING IN ACHIEVING THE PROJECT’S INTENDED OBJECTIVES OR VISION? Key findings API has had several achievements at the midterm of project implementation under three intermediate results (IRs) and three priority areas: ● API has successfully identified the concrete needs of AMS based on the National Consultant Report; seven AMS reports have been published, two of them are still in the draft stage. ● API has achieved several results in implementing direct technical assistance to member states. API has tried to explore ways to be aligned with USAID missions to ease project implementation, specifically on a national level. Conclusions & Recommendations In the implementation of its planned activities, the resultant progress has been slow, with several planned tasks still pending implementation. USAID should ensure funding is disbursed appropriately, as scheduled, to maintain API work plan and target consistently. API should strengthen its coordination with targeted AMS to achieve formal agreement on technical assistance from API. AEQ2. IN WHAT WAYS DID ASEAN’S ENDORSEMENT PROCESS IMPACT THE PACE OF PROJECT IMPLEMENTATION? WHAT INTERVENTIONS COULD OR COULD NOT BE UNDERTAKEN WHILE AWAITING ENDORSEMENT? Key findings API’s endorsement process impacted implementation progress considerably. The API project faced a delay in getting ASEAN’s endorsement, due to the long process of approval and bureaucracy, which xii | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION impacted the targeted timeline and actual implementation of planned activities. Coupled with budget constraints, the work plan adjustment presented a challenge to API’s ability to ensure the timeliness of project implementation. Conclusions & Recommendations The endorsement delivered two main benefits because it increased 1) the adoption of API’s support by the AMS and 2) regional collaboration on ASEAN issues. Providing and allocating sufficient time to build relationships with key ASEAN stakeholders as part of the work plan might be essential, in the form of introducing or socializing the scope, target, and implementation activities. AEQ3. HOW SHOULD USAID/ASEAN ADJUST API’S THEORY MODEL (DEVELOPMENT HYPOTHESIS, THEORY OF CHANGE, RESULTS FRAMEWORK, OR LOGIC MODEL), AND WHY? Key findings API ToC has successfully demonstrated its effectiveness on project implementation. The results framework portrayed targeted goals that were reflected in the form of the project’s intermediate results (IR). However, some critical factors still need to be considered and additional indicators also need to be reckoned. Conclusions & Recommendations API ToC and its results framework are aligned and have been effective in guiding implementation. The ToC draws from ASEAN policy priorities, including AEC, ASW, and DDG. Nevertheless, the challenges that go beyond the manageable capacity of API should be articulated in the API critical assumptions, and strategies to mitigate them should be defined in the project design. API should devise a work plan to mitigate and adapt the challenges beyond internal factors that might affect achieving results. AEQ4. WHAT LESSONS HAVE BEEN LEARNED ABOUT PROGRAM ELEMENTS (INTERVENTIONS OR SET OF INTERVENTIONS) THAT ARE MOST EFFECTIVE IN API DESIGN, AND WHICH PROGRAM ELEMENTS ARE LEAST IMPACTFUL? Key findings One of the success stories is the support API provided to the Philippines Bureau of Customs (BOC) to implement ACDD and NSW. API wanted to strengthen the country-level capacity to implement regional policies, but the project performance in this area is weak. API has also not yet undertaken activities to bridge the gaps between the private and public sector’s knowledge on digital transformation, as the capacity of government officials is limited. Conclusions & Recommendations Overall, API strategies and initiatives have been responsive to ASEAN needs. API applies a flexible approach to offer technical support to AMS for the ASW. Even though the technical support to ASEAN and AMS is effective, at midterm, it has not yet reached scale. API should expand its operation beyond ASW, such as supporting other trade facilitation initiatives for AMS at the national level. ASEAN-USAID INCLUSIVE GROWTH IN ASEAN THROUGH INNOVATION, TRADE, AND E￾COMMERCE (IGNITE) IEQ1. IN WHAT WAYS IS USAID/ASEAN ASSISTANCE HELPING NARROW THE DEVELOPMENT GAP IN ASEAN’S ECONOMIC GROWTH (EG) SECTOR? Key findings FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | xiii IGNITE assistance showed good progress and contribution to narrowing the development gap in the ASEAN by supporting increased trade and favorable investment climate. IGNITE assistance met the demand of ASEAN in the areas of trade facilitation, digital economy, and shared priorities IGNITE drafted the concept of Digital Data Governance Strategic Priority 4 (policies and regulations). To avoid overlapping assistance, IGNITE transitioned this task to API. Conclusions & Recommendations IGNITE assists in the high-demand area of ASEAN: ASW, standards harmonization, and digital economy. USAID/IGNITE should continue to support the ASW, workshop or training to harmonize standards, Digital Integration Framework Action Plan (DIFAP), and ASEAN Digital Integration Index (ADII) ALVS, USO, 5G, and ASCN. IEQ2. WHICH TYPE OF USAID/ASEAN ASSISTANCE (INTERVENTIONS OR SET OF INTERVENTIONS) ARE MOST EFFECTIVE IN MEETING ASEAN’S NEEDS AND ACHIEVING DELIVERABLES IN AREAS OF SHARED PRIORITY (PROVIDE EXAMPLES), AND WHY? Key findings Interventions under trade facilitation showed significant progress and substantially reduced costs and time to trade. The most effective interventions under the trade facilitation workstream are support for 1) ASW, 2) ASEAN’s low-value shipment study, and 3) workshops and training on traditional medicines and health supplements. Three activities have been identified in this evaluation as challenging but potentially effective, including support for 1) a Mutual Recognition Arrangement (MRA) for STI professionals, 2) a Gender Mainstreaming Support for the AEC, and 3) a workshop on electric vehicles. Conclusions & Recommendations Interventions under trade facilitation showed significant progress and results to reduce costs and time to trade, particularly for the ASW, and support in harmonizing standards. Intervention under the digital economy also showed progress, particularly for DIFAP and ADII. Even with a limited budget, interventions on the STI workstream and cross-cutting objectives (STI, gender, and MSME) showed good results. USAID/IGNITE should provide more funding and analysis to determine the high-impact activities. IEQ3. HOW EFFECTIVE IS IGNITE’S THEORY MODEL IN GUIDING THE IMPLEMENTATION OF PROJECT ACTIVITIES TOWARDS ACHIEVING EXPECTED RESULTS? Key findings The IGNITE Theory of Change (ToC) is effective and relevant in guiding implementation. However, several conditions need to be anticipated, such as the condition where certain ASEAN policies, laws, regulations, standards, and agreements should be closely consulted and endorsed by AMS and the gap in capacities and capabilities among AMS. Conclusions & Recommendations Adjustment to the results framework regarding financial integration activities is needed. The financial integration activities should be included in the digital economy workstream, but in the results framework's design, it is included in the trade facilitation workstream. Update any indicators that may be needed and relevant to assess the performance of interventions in the different stages. IEQ4. IN WHAT WAYS HAS USAID/ASEAN CONTRIBUTED TO INCLUSIVE GROWTH (PROVIDE EXAMPLES)? xiv | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION Key findings IGNITE interventions on ASW, digital economy, and WEAVE have contributed to inclusive growth and narrowing of the development gap. The launch of the ASW in all 10 AMS was a major accomplishment recognized by ASEAN. Support for the ASEAN Framework for USO 2.0 guides policymakers to align USO policies to focus on supply-side connectivity and demand-side solutions to increase inclusive growth. The COVID-19 pandemic has positive and negative impacts on IGNITE results. Besides Covid-19, IGNITE also faced other challenges in its implementation Conclusions & Recommendations Interventions on ASW, digital economy, and WEAVE have supported inclusive growth in ASEAN. IGNITE has two additional needs for support from USAID: 1) Increased budget, funding is the biggest challenge experienced by IGNITE compared to other Dialogue Partners 2) Prioritizing support for high-impact projects, participants in this evaluation also suggested that USAID place more emphasis on supporting high-impact projects by financing long-term projects. USAID should support IGNITE by providing adequate funding and prioritizing high-impact interventions such as support for ASW. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 1 EVALUATION PURPOSE, OBJECTIVES, AND EVALUATION QUESTIONS EVALUATION PURPOSE AND OBJECTIVES The purpose of the evaluation was to assess the implementation effectiveness of three USAID/ASEAN activities at midterm. Evaluated activities include PROSPECT, IGNITE, and API. The scope of work for assessing specific objectives of the evaluation was to: 1. Review performance of the USAID/ASEAN activities with a focus on project design management, and implementation 2. Identify successes and opportunities 3. Identify key lessons learned that can be used to improve program management and inform future programming design The MEL-P Activity conducted this joint midterm evaluation from October 2021 to January 2022 at the request of USAID/ASEAN. Consistent with the USAID Evaluation Policy definition, this evaluation is classified as a performance evaluation. USAID/ASEAN and the implementing partners (IPs) of the evaluated activities are the primary audiences of this evaluation report. The secondary audience, who may receive this report through USAID/ASEAN, is ASEAN and its respective secretariats, the AMS, other ASEAN donors, and USAID IPs working in ASEAN countries. The wider public will also have access to this report online when uploaded to USAID’s Development Experience Clearinghouse website. EVALUATION QUESTIONS This section lists the evaluation questions, which were obtained from the USAID/ASEAN Statement of Work for this midterm evaluation. The evaluation team provided descriptions under each evaluation question to demonstrate its understanding of the focus of each question. Overall, the evaluation aimed to answer four general questions and four specific questions per activity evaluated. Because the general questions derive from the activity-specific questions, the team transformed general questions into conclusions based on the activity-specific findings. Therefore, the general questions are addressed in the conclusions chapter of this report. This approach mitigates the potential for repetitive descriptions of findings between the general questions and activity-specific questions. GENERAL EVALUATION QUESTIONS (GEQS) 1. GEQ1: How effective are the USAID/ASEAN activities in achieving their expected results? GEQ1 examines the progress made by the three USAID/ASEAN activities toward achieving their expected results. It will analyze the extent to which the activities design, management, and implementation support progress toward expected results. This question will also explore what USAID/ASEAN can do to strengthen the effectiveness of these activities to support ASEAN Community Vision on three pillars. Political-Security Community, Economic Community, and Social￾Cultural Community. 2. GEQ2: What main challenges affect project implementation, and how can they be mitigated? This question investigates the main challenges experienced by the USAID/ASEAN activities if any. These may include limitations related to activity design and implementation. The question will also establish how USAID could help in addressing those challenges. 3. GEQ3: What was learned (from information or data) through project implementation that can be used to refine or change our activities’ theory models (development hypotheses, 2 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION theory of change, results frameworks, or logic models)? In what ways should USG assistance to ASEAN be more targeted or focused? The primary purpose of this evaluation question is to identify, based on solid evidence, critical lessons from activity implementation that can be used to refine implementation approaches. The evaluation team will also assess the effectiveness of the theories of change of the three activities to guide implementation strategies. Findings from this question will support the advancement of policy implementation across ASEAN member countries and establish measures for USAID/ASEAN assistance. 4. GEQ4: What limitations to USAID assistance have been observed, and how should USAID/ASEAN and/or USG players address them? This question examines perspectives about USAID’s assistance to ASEAN at the higher level with the view of identifying ways to improve USAID assistance approaches. PROSPECT EVALUATION QUESTIONS (PEQS) 1. PEQ1: How has PROSPECT adjusted its strategy and approach in supporting ASEAN with the current set of ASEAN partners? What are the specific instances in which these adjustments have been made, and why? This evaluation question examines whether and how PROSPECT adapted its strategies to fit the needs of ASEAN partners. It also explores the effectiveness of PROSPECT strategies based on the evidence of the extent to which PROSPECT interventions are progressing toward expected results. Also, this question examines the reasons for the observed progress or lack of progress toward expected results. 2. PEQ2: What support has USAID provided to PROSPECT in administrative and diplomatic engagement to support project implementation and ASEAN in developing institutional capacity and human capital to better advance shared priorities? The primary focus of this question is to examine the extent to which USAID has supported PROSPECT in creating a positive environment for implementing the activity. This question will also examine the relevance and effectiveness of USAID’s assistance to meet the implementation needs of the activity. 3. PEQ3: How effective is PROSPECT’s theory model in guiding the implementation of project activities toward achieving expected results (development hypothesis, theory of change, results framework, or logic model)? Why or why not? PEQ3 explores the appropriateness of PROSPECT’s theory of change to the implementation needs of the activity and the extent to which the implementation strategies are aligned with the activity’s theoretical model. 4. PEQ4: What challenges has PROSPECT faced in implementing an assistance program that spans such a broad spectrum of issues? This question will examine the benefits and limitations of the PROSPECT design and implementation approach. The findings will also establish which type of assistance should be prioritized and the criteria to use for prioritizing assistance decisions. API EVALUATION QUESTIONS (AEQS) 1. AEQ1: What are the main successes of the API activity, and what challenges is the activity facing in achieving its intended objectives or vision? This question will investigate the areas where API has been most and least successful at the midterm of its implementation. Also, this question will establish the reasons for the successes and challenges faced, including the potential impact of the COVID-19 pandemic on the API implementation. This question will also glean the key lessons learned to support implementation adaptations. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 3 2. AEQ2: How did ASEAN’s endorsement process of API impact the pace of project implementation? What interventions could or could not be undertaken while awaiting endorsement? Specific emphasis in this question will be placed on identifying whether and how ASEAN’s endorsement process impacted the activity implementation. This question will examine how API support has contributed to improving the endorsement process and managing effective communication between AMS under ASEAN targeted policy, and lessons for accelerating the endorsement process. 3. AEQ3: How should USAID/ASEAN adjust API’s theory model (development hypothesis, theory of change, results framework, or logic model), and why? This evaluation question will assess the relevance and effectiveness of API’s Theory of Change to guide implementation strategies by examining the extent to which the theoretical model accommodates ASEAN policy needs. Potential improvements to the theoretical model/framework will be established. 4. AEQ4: What lessons have been learned about the effectiveness of API program elements? This question will explore learnings from the specific elements/components of the activity to develop improvement strategies. It will also explore the extent to which API is progressing toward building capacity for policy development and implementation across member states. IGNITE EVALUATION QUESTIONS 1. IEQ1: How is USAID/ASEAN assistance helping to narrow the development gap in ASEAN's EG sector? This question examines how IGNITE interventions or set of interventions are progressing toward meeting ASEAN's needs, particularly under the digital group and Trade Facilitation Group interventions, and the reasons for the observed results. 2. IEQ2: Which type of USAID/ASEAN assistance (interventions or set of interventions) is most effective in meeting ASEAN's needs and achieving deliverables in areas of shared priority (provide examples), and why? The main emphasis of this evaluation question is to examine the performance (specific results) of IGNITE interventions in areas of shared priority, e.g., science, technology, and innovation (STI), gender, and micro, small, and medium enterprises (MSME). This question also explores how the coronavirus disease 2019 (COVID-19) pandemic affected progress toward achieving expected results. 3. IEQ3: How effective is IGNITE’s theory model in guiding the implementation of project activities toward achieving expected results (development hypothesis, theory of change, results framework, or logic model)? Why or why not? The main emphasis of this evaluation question is to assess the relevance and effectiveness of IGNITE’s theory of change to guide implementation strategies by examining the extent to which the theoretical model accommodates ASEAN policy needs. Potential improvements to the theoretical model/framework will be established. 4. IEQ4: How has IGNITE contributed to achieving inclusive growth priorities (provide examples)? This question assesses the contributions of IGNITE toward inclusive growth. It also examines what USAID/ASEAN should do to improve and sustain its support for implementing IGNITE interventions. 4 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION BACKGROUND CONTEXT ASEAN was founded in 1967 as Southeast Asia’s primary multilateral institution for regional cooperation. It was established to serve as the means for facilitating political security, economic, and socio-cultural integration. In addition to spurring economic development and social progress, ASEAN seeks to provide a mechanism for peaceful resolution of differences and to promote regional stability. ASEAN membership covers 10 countries: 1) Brunei Darussalam, 2) Cambodia, 3) Indonesia, 4) Laos, 5) Malaysia, 6) Myanmar, 7) the Philippines, 8) Singapore, 9) Thailand, and 10) Vietnam. The ASEAN region has nearly 650 million people and a GDP (as of 2016) of over $2.8 trillion (ASEAN, 2020). Over the past 50 years, ASEAN has made significant advances in security and stability among ASEAN member states (AMS), allowing the region to focus on economic development and other areas of transnational cooperation. Although ASEAN has become a key regional institution for rule￾making and fostering peace and stability in Southeast Asia, it has no supranational authority over its members, functioning instead as a framework for collaboration and cooperation. The USG is a long-term strategic ASEAN Dialogue Partner, supporting ASEAN as a key multilateral institution under the IPS. The overall goal of USAID assistance to ASEAN is to build the capacity of ASEAN and AMS to contribute to sustainable and inclusive growth and encourage rules-based systems to support a prosperous Southeast Asia. This goal aligns with the goals of the IPS in enhancing shared prosperity, championing good governance, and ensuring a peaceful and secure regional order. Figure 1. Map of ASEAN Member States Source: https://www.dreamstime.com/royalty-free-stock-image-asean-map-countries-location-flags-image40294386 FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 5 USAID/ASEAN DEVELOPMENT HYPOTHESIS The USAID/ASEAN Development Cooperation Framework (DCF), presented in Figure 2, has three Development Objectives (DOs): • DO1: Regional and global challenges faced by ASEAN and its member states effectively addressed • DO2: ASEAN Capacity to further regional economic integration and sustainable economic growth increased • DO3: Rules-based architecture to uphold human dignity and the RoL advanced by ASEAN and its member states The USAID/ASEAN results framework (Figure 2) shows these DOs and their associated intermediate results. The development hypothesis of USAID/ASEAN assistance posits that: If Asia’s regional institutions are better prepared to promote sustainable and inclusive growth (DO1), vulnerable populations are more able to address transnational risks (DO2), and development assistance in Asia is increasingly effective, then Asia’s growth will be more sustainable and inclusive. Figure 2. USAID/ASEAN Results Framework ASSUMPTIONS USAID/ASEAN identifies four critical assumptions that could affect this development hypothesis: 1. The Asia-Pacific region will experience relatively robust, but uneven, economic growth. China will remain a dominant economic power in the Asia-Pacific. 6 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION 2. The region remains free of major armed conflict, and internal conflicts do not threaten macro-stability at the country or sub-regional level. 3. The frequency and, potentially, the intensity of climate-related events will continue to increase, affecting Asia’s susceptibility to natural disasters and climate change. 4. Regional Development Mission for Asia’s major development initiatives will continue to receive high levels of diplomatic support and provide opportunities to take a “whole of government” approach to select regional development issues. BACKGROUND ON EVALUATED ACTIVITIES PROSPECT PROSPECT is a five-year $11,329,555 activity, USAID Award Number 72049718C00002, implemented by Development Associates International. It is part of the USG’s overall program designed to provide comprehensive support to all three ASEAN communities. The activity implementation started in August 2018, ending in August 2023. PROSPECT supports ASEAN’s objective of building a more inclusive, rules-based, and prosperous Southeast Asia. The activity aims to achieve this objective by increasing ASEAN’s capacity to resolve regional and global challenges and advancing an effective, rules-based cooperation architecture to uphold human dignity and the RoL among governments in the region. PROSPECT supports targeted ASEAN institutions, sectoral bodies, Entities Associated with ASEAN and ASEC to strengthen their effectiveness and ability to implement international policy standards and agreements. The activity builds capacity through technical assistance, training, workshops, and other resources for targeted ASEAN institutions, including ASEC and other ASEAN institutions. Capacity strengthening of these bodies and ASEAN as a whole will equip them to implement multilateral policy standards and agreements effectively. PROSPECT THEORY OF CHANGE AND EXPECTED RESULTS PROSPECT has a theory of change comprised of two causal pathways. These two theories are: 1. IF ASEAN addresses transnational challenges related to human security AND strengthens effective response to humanitarian crises, particularly natural disasters THEN ASEAN capacity to resolve regional and global challenges will be strengthened. 2. IF ASEAN strengthens good governance and rules of law through engagement with judicial, legislative, and executive branches AND increases civil society participation in ASEAN AND expands rights and opportunities for women, youth, and other vulnerable and marginalized groups THEN ASEAN and AMS will advance effective regional integration to enhance the RoL and uphold human rights. The activity is implemented under three primary components: 1) building ASEAN and AMS capacity to resolve regional and global challenges, 2) supporting ASEAN and AMS to advance effective regional integration to enhance the RoL and uphold human rights, and 3) human rights and opportunities for marginalized and/or vulnerable groups. These components primarily contribute to DO1 and DO3 of the USAID/ASEAN DCF. Under DO1: ASEAN and its member states more capably and frequently contribute to resolving regional and global challenges, PROSPECT contributes to the following two intermediate results (IRs) of the cooperation framework: FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 7 1. IR 1.1: ASEAN takes united, collaborative, and proactive approaches to address maritime and security issues 2. IR 1.2: More robust responses to emerging humanitarian crisis enabled Under DO3: ASEAN and its member states advance an effective, rules-based architecture to uphold human dignity and the RoL, PROSPECT contributes to three IRs: 1. IR 3.1: ASEAN strengthens civil society engagement and influence on critical issues, including with state institutions 2. IR 3.2: ASEAN expands rights and opportunities for women, youth, and other vulnerable people 3. IR 3.3: ASEAN and AMS integration advances through addressing governance gaps API API is a two-phase activity of USAID/ASEAN. The first phase is a four-year $9 million technical assistance, USAID Award Number 7200AA18D00011, implemented by International Business Initiative. The performance period for Phase 1 is 2019 to 2023. The activity aims to strengthen ASEAN centrality as a leading IPS partner by supporting member states in implementing ASEAN policies under the AEC. The main target of API Phase 1 is to undertake national-level interventions in the AMS to accelerate the implementation of key ASEAN initiatives that support USG’s and ASEAN’s goals. IPS, USAID/ASEAN DCF, ASEAN Community Vision 2025 guide the implementation of Phase 1 of API. API is also aligned with USAID missions in the ASEAN region. API Phase I is a four-year activity that focuses on the implementation of selected ASEAN policies under the ASEAN Economic Community (AEC), moreover, this phase should provide USAID with lessons learned that will feed and nourish into Phase II. Phase II might be cover and address the implementation of ASEAN instruments within all three ASEAN communities (Political-Security, Economic, and Socio-Cultural). API collaborates with IGNITE and PROSPECT to achieve common results and support the achievement of the DCF objectives. The last graphic in Annex 5 shows the future of work API focus areas and their relation to PROSPECT and IGNITE. Despite ASEAN’s interest to ensure that each member state implements ASEAN policies, it lacks the formal authority to enforce implementation within the member states. API technical assistance aims to address this gap by strengthening each member state's capacity to implement ASEAN policies. USAID/ASEAN expects API Phase 1 to achieve three key results: 1. USAID contribution to the IPS reinforced. 2. ASEAN centrality in the U.S. Indo-Pacific vision strengthened. 3. Implementation of ASEAN policies at the member-state level advanced. Four overarching parameters guide the implementation of API, the: 1) IPS, 2) USAID/ASEAN DCF, 3) ASEAN Community Vision 2025, and 4) strategic alignment with USAID missions in the ASEAN region. USAID/ASEAN expects results of API to contribute to the IPS vision of enhancing shared prosperity and USAID’s approach to advancing this vision by unlocking private enterprise-led economic growth. API may also contribute to the efforts of IPS and other USG initiatives in the future when the opportunity may arise, the scope or areas including (but not limited to) (1) the Indo-Pacific 8 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION Transparency Initiative, Enhancing Growth and Development through Energy (EDGE), (2) the Infrastructure Transaction and Assistance Network (ITAN), (3) the Digital Connectivity and Cybersecurity Partnership (DCCP), and (4) the Women’s Global Development and Prosperity Initiative and legislation such as the 2018 Asia Reassurance Initiative Act (ARIA). API THEORY OF CHANGE AND EXPECTED RESULTS/OBJECTIVES The API theory of change states that: • IF API continually identifies key opportunities, refines approaches to those opportunities through research and consultation, THEN API can implement policy support interventions that can improve ASEAN priorities such as the expanded implementation of the ASEAN Single Window (ASW) and Digital Data Governance (DDG), among others. The theory of change is the basis for the API results framework (Annex 5). Two objectives drive the API phase 1 design and implementation to: 1. Identify concrete needs and opportunities for supporting the implementation of targeted ASEAN policies at the member state level. 2. Provide concrete technical assistance to member states to strengthen their implementation of targeted ASEAN policies. Table 1 summarizes API technical activities for these two objectives by sub-area. Table 1. Summary of API Sub-Areas and Key Activities API SUB-AREAS KEY ACTIVITIES ASW Mapping the status of implementation of AMS for the ASW Mapping for prioritizing and selecting needs and opportunities for AMS for operationalizing the ASW An indicative action map for AMS to enhance the operation of NSWs in supporting the ASW Live Operation’s implementation Increased the abilities of selected AMS (Cambodia, Lao People’s Democratic Republic (PDR), Myanmar, and the Philippines) to do the live operation of exchange of the intra-ASEAN certificate of origin (ASEAN Trade in Goods Agreement (ATIGA) Form D), Common Message Exchange Header, and the ASEAN Customs Declaration Document (ACDD). This includes increased use of the ACDD as advance information to support the Risk Management System by these selected AMS Increased public and business stakeholders’ awareness of the benefits of NSW establishment and the ASW live operation in Burma, Cambodia, and Laos ASEAN Framework on Digital Data Governance (DDG) Mapping the current status of AMS to the ASEAN Framework of DDG Mapping for prioritizing and selecting needs and opportunities for AMS with the ASEAN Framework on DDG A study on current implementation status and needs of AMS on digital data governance, especially concerning the ASW policy at the national level; mainly on cross-border data flow mechanism, data protection, and digital signature FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 9 IGNITE IGNITE is a five-year $10,496,880 activity implemented by Nathan Associates under USAID Award Number 72049718C00004. The activity, whose implementation started in 2018, was designed to promote a well-integrated and inclusive AEC. IGNITE aims to strengthen the enabling environment for cross-border trade to benefit consumers and workers in ASEAN and the United States. The three focus areas of the activity are 1) trade facilitation, 2) digital economy, and 3) science, technology, and innovation. The activity aims to reduce trade costs, spur ecommerce, and increase productivity to deepen ASEAN economic integration and foster more inclusive economic growth. IGNITE works with the private sector and civil society organizations (CSOs) to increase their engagement with ASEAN Sectoral Bodies to ensure that ASEAN policies have broad political support and are market-friendly. The activity addresses capacity limitations through technical assistance to ASEAN Sectoral Bodies focused on market-based approaches, supporting MSME development while addressing gender issues. IGNITE THEORY OF CHANGE AND OBJECTIVES/EXPECTED RESULTS IGNITE theory of change posits that IF ASEAN institutions are able to mobilize sufficient political will and capacity to reduce the time and cost of trade, particularly in key growth sectors (e.g., digital services and science, technology, and innovation), THEN AMS will benefit from accelerated and inclusive growth benefitting AMS consumers and firms as well as US exporters and workers. USAID/ASEAN expects IGNITE to achieve results under three key objectives as shown in Annex 5. 10 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION EVALUATION METHODS AND LIMITATIONS This midterm evaluation was conducted using a concurrent mixed-methods design. Specific methods used include documents review, primary qualitative data collection using key informant interviews (KIIs), and secondary analysis of monitoring data and data from other related institutions (e.g., trade commission of respective countries). Evaluation design matrixes linking each of these approaches with respective evaluation questions are provided in Annex 3. DOCUMENT REVIEW The evaluation team reviewed several documents obtained from USAID and the implementing partners (IPs) to gain background and understanding of the activities design, implementation approaches inform the evaluation design and data collection priorities, and establish the implementation progress and results to date. Reviewed documents include activity descriptions, routine performance monitoring reports, databases, and studies and assessments conducted by the IPs. The document review will establish evidence related to the activity design, implementation approaches, and progress toward expected results, challenges and lessons learned midway through the Activities’ implementation. PRIMARY DATA Primary data were collected through: 1) KIIs with representatives from USAID/ASEAN, government officials, IPs, ASEAN sectoral bodies; and 2) focus group discussions with government officials from ASEAN member countries. KEY INFORMANT INTERVIEWS KIIs were used to gather information on specific themes as illustrated in the evaluation design matrixes in Annex 3. Interviews were conducted using semi-structured interview guides, attached to Annex 4. Participants in the KII included representatives from USAID/ASEAN, government officials, IPs, ASEC, ASEAN sectoral bodies, Entity Associated with ASEAN, and CSOs. Annex 6 provides a detailed list of the institutions from which the evaluation team recruited KII participants, organized by activity under evaluation. SECONDARY DATA ANALYSIS The evaluation team conducted a secondary analysis of data collected routinely by IPs to assess the progress of the activities on their key performance indicators. These datasets include project documents (project reports, AMEL Plan, publications), relevant ASEAN documents (ASEAN Vision, ASEAN Blueprints, ASEAN Workplans), ASW Trader Survey 2020, Women in STEM Webinar Survey 2021, AMS National Consultant Reports, Quarterly Regional Trend Analysis, Summary of Workshop Discussion/Knowledge Sharing Section, ASEAN US Mission, and AMS National Government website. The findings are obtained from the mixed method, combining secondary analysis with the document review and KIIs findings, thus providing a more comprehensive understanding of the performance of USAID/ASEAN activities. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 11 SAMPLING The sampling method applied in this study is a non-probability sample; informants are selected based on non-random criteria, and not every individual has a chance of being included. The team selected evaluation institutions and participants purposely based on their involvement with the USAID/ASEAN activities. Working in consultation with the CORs of the three evaluated activities and the respective IPs, the evaluation team established a comprehensive list of institutions to include in the sample. These institutions included ASEAN bodies/secretariats, beneficiary institutions, government entities, and USAID technical offices funding and managing USAID/ASEAN activities. Once the list of institutions was established, the team selected participants based on their roles and interactions with, or involvement in, implementing USAID/ASEAN activities. The complete list of selected institutions is presented in Annex 6. DATA PREPARATION QUALITATIVE DATA Preparation of qualitative data for analysis was done in two stages: 1. During KIIs, the evaluation team audio-recorded interviews and took notes. The notes captured the most important responses to each question asked of the KII participants and formed the summary sheet or completed the key findings summarization matrix (Annex 4). 2. The evaluation team transcribed all audio recordings from the interviews to capture the findings in greater detail. The transcripts were structured based on the evaluation questions, sub-questions, and pre-identified themes from the data collection tools. SECONDARY QUANTITATIVE DATA The team obtained secondary quantitative data from the project AMEL reports on key performance indicators. The key performance indicators showing target numbers and the percentage achieved from each input of the project are compared and used to support the qualitative data on project results gathered from the KIIs. Quantitative secondary data was analyzed using a statistic descriptive approach to show the trends, baseline, and results. The data are from annual monitoring documents. DATA ANALYSIS The team conducted a detailed thematic analysis of the transcripts manually, using a standard template, which was structured by evaluation questions and the detailed questions from the KII guide. Then, conducted the analysis with specific emphasis on filling gaps in the findings from the desk review and secondary analysis, combining KII responses with data from secondary sources and analyses, and comparing findings across countries and participant institutions to identify similarities and differences in the opinions obtained. ETHICAL CONSIDERATIONS This midterm evaluation collected non-sensitive data. For this reason, a review by an institutional review board was not needed. However, evaluation team members followed professional and ethical procedures during data collection, ensuring the honesty and integrity of the evaluation. The team applied appropriate strategies to protect the identities of evaluation participants. Before conducting each interview, the evaluation team obtained the participant’s consent to be interviewed and briefed 12 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION them on the evaluation and the potential use of the findings. Data collectors used phones or VoIP applications to conduct the interviews. Therefore, the participant chose an appropriate location with reasonable privacy to take the interview without the influence of the evaluation team member. The evaluation team used a central password-protected Google Docs repository to store all transcripts, thus preventing unauthorized access to transcripts by non-evaluation team members. Evaluation team members used personal computers to record and transcribe the interviews, which were password-protected, thus preventing unauthorized access. LIMITATIONS 1. This evaluation was conducted by an in-country team based in Indonesia, working remotely with a team leader based in the U.S. The differences in time zones and remote work impacted efficiency in coordination among team members. 2. Travel restrictions due to COVID-19 limited in-person data collection efforts. Several of the KIIs were conducted online, which, in some cases were affected by poor internet connectivity. 3. The process of mobilizing and scheduling key informants to participate in this evaluation was challenging. Some did not respond to the emails and phone call requests to participate, some responded late, and several confirmed interviews had to be rescheduled, which impacted the timely completion of data collection 4. The timeframe to complete this evaluation was tight, considering that it encompassed three USAID/ASEAN activities and was multi-country. Travel restrictions and scheduling challenges compounded this challenge further. 5. Due to the limitation of secondary data, this evaluation conducted secondary data analyses based on the latest data (Q3 2021). Analysis with more updated data will provide more accurate results. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 13 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS PROSPECT PROJECT FINDINGS PEQ 1: HOW HAS PROSPECT ADJUSTED ITS STRATEGY AND APPROACH IN SUPPORTING ASEAN WITH THE CURRENT SET OF ASEAN PARTNERS? WHAT ARE THE SPECIFIC INSTANCES IN WHICH THESE ADJUSTMENTS HAVE BEEN MADE AND WHY? PROSPECT INTERVENTIONS PROSPECT undertakes a broad range of interventions to support the ASEAN agenda in the ASEAN political and security community (APSC) and the Socio-Cultural Community (ASCC) pillars. This support falls under three target areas and workstreams: 1. Addressing transnational challenges and non-traditional security threats 2. Promoting transparent and effective governance 3. Elevating the rights and opportunities for women and other vulnerable groups In its interventions, PROSPECT provides technical assistance, administrative and logistical support, and grants to ASEAN counterparts including ASEAN Sectoral Bodies and Organs, Entities Associated with ASEAN, National Government Agencies, and Civil Society Organizations. PROSPECT support assists ASEAN in developing policy instruments and tools, capacity building and awareness-raising programs, consultations and policy dialogue, and research and assessments. PROSPECT’s key interventions within the three target areas or workstream fall under multiple focus areas. Addressing Transnational Crimes and Non-Traditional Security Threats TIP. In TIP, PROSPECT has supported ASEAN through the ASEAN Commission on the Promotion and Protection of the Rights of Women and Children (ACWC) in implementing and improving cross-sectoral coordination of the ASEAN Convention Against Trafficking in Persons, Especially Women and Children (ACTIP), and its ASEAN Plan of Action Against Trafficking in Persons, Especially Women and Children (APA). PROSPECT provided technical assistance for the finalization and launch of the ACWC Regional Guidelines and Procedures to Address the Needs of Victims of Trafficking in Persons, and its companion Practitioners’ Model Implementation Toolkit (2019) developed based on the ACTIP and APA. The toolkits are translated into six ASEAN national languages: I) 1ndonesian, 2) Khmer, 3) Lao, 4) Myanmar, 5) Thai, and 6) Vietnamese, for use among national practitioners and other key stakeholders involved in counter-trafficking in persons in ASEAN member states. At the national level, PROSPECT convened the Practitioners’ Workshop on the Implementation of ACWC Regional Guidelines in Thailand. In collaboration with the UN-ACT and the Nexus Institute, PROSPECT developed modules for the Regional Curriculum on Victim Identification, Protection, Assistance, and Reintegration and adapted the curriculum for national-level use in a series of trainings in Thailand for frontline officials, labor inspectors, social workers, and health service providers. 14 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION In 2019, PROSPECT conducted an assessment on opportunities within ASEAN to address trafficking and forced labor in the region’s fishing industry. This assessment resulted in a Synthesis Report on Opportunities within ASEAN for the Advancement of a Regional Multisectoral Strategy to Address Human Trafficking and Forced Labor in the Fishing Industry. Countering violent extremism (CVE). In supporting ASEAN’s CVE effort, including ASEAN multi￾sectoral coordination, PROSPECT provided technical assistance for the SOMTC Working Group on Counterterrorism. PROSPECT assisted in drafting the SOMTC multi-sectoral work plan to address CVE in ASEAN, the Bali Work Plan to Prevent and Counter the Rise of Radicalization and Violent Extremism 2019–2025 (Bali Work Plan). PROSPECT also supported the consultation process in developing the Bali Work Plan. It convened two multi-sectoral consultation meetings involving 14 ASEAN Sectoral Bodies across the pillars of ASEAN and civil society organizations. After the adoption and launch of the Bali Work Plan, PROSPECT continued supporting ASEAN efforts to coordinate and manage the implementation of the work plan by convening the first Multi￾Task Force Meeting among relevant ASEAN Sectoral Bodies and ASEAN’s Secretariat to formalize implementation mechanisms of the Work Plan. Wildlife trafficking. PROSPECT work responds to the threat to the region from being the source of the world's greatest marketplaces for wildlife and timber. PROSPECT provided technical assistance to the ASEAN Working Group for the Convention on International Trade in Endangered Species of Wild Fauna and Flora and Wildlife Enforcement. The working group drafted ASEAN’s Guidelines for Detecting and Preventing Wildlife Trafficking. The guidelines have been endorsed by ASEAN’s senior official on forestry and will be used as a common regional resource for coordinating counter wildlife-trafficking efforts in ASEAN. To review and finalize the draft guidelines, PROSPECT supported convening two ASEAN multi-sectoral regional workshops for AMS representatives. Also, PROSPECT supported the development of ASEAN’s Regional Plan of Action (RPoA) on the Convention on International Trade in Endangered Species of Wild Flora and Fauna (CITES) and Wildlife Enforcement (CITES-WE RPoA) 2021–2025. PROSPECT began by convening the consultation meeting involving relevant stakeholders to map steps for developing the new RPoA and subsequently supported the drafting and finalization of the RPoA. Disaster management. PROSPECT has several interventions on disaster management, which aim at increasing ASEAN’s collective capacity to prepare for and respond to natural and non-natural disasters (health) in the region. Regarding natural disasters, PROSPECT conducted intensive English language training courses for the ASEAN Coordinating Centre for Humanitarian Assistance on Disaster Management (AHA Centre), and a capacity needs assessment on humanitarian logistics and supply management among ASEAN National Disaster Management Organization (NDMO). PROSPECT prepared a summary report of the assessment on Emergency Supply Chain/Logistics Management Readiness based on the data collected from interviews with ASEAN member states NDMOs. In health, PROSPECT provided a technical expert to support the development of a multi-sectoral ASEAN Pandemic Preparedness and Response (PPR) Framework, which was endorsed by the ASEAN Senior Officials Meeting on Health Development (SOMHD). PROSPECT also collaborated with the SOMHD to develop ASEAN Public Health Emergency Coordination System (APHECS) to support a multi-sectoral, institutional response to emerging outbreaks. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 15 Responsive and Transparent Governance PROSPECT has mainly worked with ASEAN judiciary and civil society organizations in the region to promote good governance, RoL, and compliance with ASEAN conventions and agreements. PROSPECT has also supported efforts to strengthen civil society networks and cultivate multi￾sectoral platforms for engaging with ASEAN officials on critical issues, as follows: Judicial cooperation. In the judiciary, PROSPECT provided legal expertise for drafting the Council of ASEAN Chief Justices (CACJ) Working Group on Judicial Education and Training Work Plan 2020–2025. The Work Plan outlines priority areas for training, comprising corporate law, cross￾border criminal law, civil procedural law, and leadership and management law. It also guides how to manage and implement regional training programs for judges effectively. PROSPECT also supported a high-level regional dialogue between the CACJ and AICHR on the rights of accused persons in criminal cases held in Malaysia. Anti-corruption for a rule-based ASEAN. PROSPECT worked with the KPK Indonesia’s Corruption Eradication Commission and a member of the Southeast Asia Parties Against Corruption (SEA-PAC) to convene the SEA-PAC Regional Workshop on Private Sector Corruption, Legal Frameworks, Enforcement, and Prevention. In collaboration with other donors, PROSPECT supported the participation of the SEA-PAC representatives from eight member states to share good practices and experiences related to combating corruption in the private sector. Strengthening civil society organizations’ networks. PROSPECT started the initiative to establish a formal relationship between ASEAN’s Secretariat and ASEAN Institutes for Strategic and International Studies (ASEAN ISIS), called ASEAN-ISIS Network. PROSPECT carried out an assessment on Partnership for Knowledge: Collaboration between ASEAN’s Secretariat and ASEAN￾ISIS and provided a technical expert to develop Terms of Reference (ToR) for the relationship. TRAFFICKING IN PERSONS (TIP) • Supported ACWC Regional Guidelines to Address the Needs of TIP Victims & Practitioners Workshop • Supported development of Practitioners’ Model Toolkits and translation into six AMS languages • Developed Curriculum on Victim Identification, Protection, Assistance and Reintegration • Developed Synthesis Report on Trafficking and Forced Labor in the Fishing Industry COUNTER VIOLENT EXTREMISM •Supported development of Bali Work Plan to Prevent and Counter Radicalism and Violent Extremism, 2019 •Supported ASEAN Cross-Sectoral and Cross-Pillar Meetings, and CSO Consultations on Bali Work Plan •Supported the first Multi-Task Force Meeting to formalize implementation mechanisms of the Work Plan WILDLIFE TRAFFICKING • Developed ASEAN Guidelines for Detecting and Preventing Wildlife Trafficking and and its multi-sectoral regional workshops • Develop ASEAN Regional Plan of Action (RPoA) on the Convention on CITES and Wildlife Enforcement 2021-2025 DISASTER MANAGEMENT •Natural Disaster: •Conducted capacity needs assessment of ASEAN on humanitarian logistics in 2020 • Supported the English language training on disaster management for ASEAN National Disaster Management Authorities • Health: • Supported Multi￾sectoral ASEAN PPR Framework •ASEAN Public Health Emergency Coordination System (APHECS) to support a multi-sectoral, institutional response to emerging outbreaks Table 2. PROSPECT Intervention to Increase ASEAN Capacity to Address Non-Traditional Security Threats 16 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION ASEAN’s Secretariat and ASEAN-ISIS consulted on the ToR, and the ToR was then returned to PROSPECT for further review and follow-up. PROSPECT provided grants to Habibie Centre to implement the Community Empowerment for Raising Inclusivity and Trust through Technology Application (CERITA) program involving youth representatives from all 10 AMS. The program becomes a forum for storytelling and dialogue among youth with different backgrounds and identities to share experiences and discuss critical social issues, such as discrimination and preventing violent extremisms, to build awareness and strengthen their shared ASEAN identity. As a follow-up to the CERITA program, PROSPECT provided funding for the organization of Talking ASEAN, a public event involving wider stakeholders not limited to youth, such as government, think tanks, CSOs, and media representatives to discuss strategies for tackling the violent extremism and developing regional community networks on preventing violent extremism. PROSPECT also provided grants to a CSO working on ASEAN advocacy, the Human Rights Working Group (HRWG), to organize a regional policy dialogue on implementing the ASEAN Consensus on the Protection and Promotion of the Rights of Migrant Workers. PROSPECT subsequently supported the follow-up policy dialogue, attended by civil society organizations and government representatives, in seven member states: 1) Cambodia, 2) Indonesia, 3) Malaysia, 4) Myanmar, 5) the Philippines, 6) Thailand, and 7) Vietnam. Table 3. Interventions to Promote Responsive and Transparent Governance Promoting human rights and expanding opportunities for women and other vulnerable groups Human rights. PROSPECT supported the convening of the AICHR Forum on Access to Justice and consultation on Freedom of Opinion and Expression (FOE) in ASEAN to promote Article 23 of the ASEAN Human Rights Declaration (AHRD) on FOE. PROSPECT also supported the AICHR Youth Debate on Human Rights to provide an opportunity for youth in ASEAN to learn about the AHRD and engage in the discourse on human rights issues in the region. GOOD GOVERNANCE AND RULE OF LAW •Judicial cooperation • Supported development of the CACJ Working Group on Judicial Education and Training Work Plan (CAC WG-JET) 2019–2025 •Convened AICHR-CACJ High-Level ASEAN Human Rights Dialogue: The Rights of Accused Persons in Criminal Cases •Anti-corruption: Rule-based ASEAN •Convened SEA-PAC Regional Workshop on Private Sector Corruption, Legal Frameworks, Enforcement,and Prevention CIVIL SOCIETY ORGANIZATION NETWORKS •ASEAN-ISIS Network •Conducted an assessment report on partnership between ASEAN-ISIS Network and the development of the TOR for the basis of the relationship •CERITA and Talking ASEAN • Supported implementation of CERITA for youth reps of 10 AMS • Supported implementation of Talking ASEAN on strategies for tackling violent extremism • Migrant Workers • Supported the regional policy dialogue on implementinh the ASEAN Consensus Migrant Workers and the follow-up policy dialogue in seven member states through HRWG FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 17 Gender mainstreaming. To operationalize the ASEAN Declaration on the Gender-Responsive Implementation of ASEAN Community Vision 2025, PROSPECT supported the ACWC and ACW joint effort to establish the ASEAN Gender Mainstreaming Strategic Framework (AGMSF). In developing the AGMSF, PROSPECT, in close consultation with ACW, ACWC, and ASEAN Secretariat, supported with logistics the meetings of the ACW-ACWC Ad-hoc Working Group on Gender Mainstreaming. PROSPECT also assisted ASEAN with data collection, such as two online surveys involving more than 2,500 respondents from government officials and external stakeholders from across AMS. The draft AGMSF was endorsed by the ASEAN Ministerial Meeting on Women. Women, Peace, and Security (WPS). As part of the realization of the Joint Statement on Promoting Women, Peace, and Security in ASEAN of 2017, PROSPECT, in collaboration with UN Women, supported the development and launch of the ASEAN Regional Study on WPS. PROSPECT began by convening the First Regional Symposium on Implementing the WPS Agenda in ASEAN, conducted in Phnom Penh. Symposium participants discussed opportunities and challenges, shared good practices, and identified potential focus areas for a regional scoping study on WPS. Also, PROSPECT supported the first meeting of the ASEAN WPS Advisory Group to finalize the study and discuss the next steps for regional action. PROSPECT support for this intervention included logistics and technical expertise. Youth engagement and raising awareness about ASEAN. In expanding youth engagement to contribute to ASEAN’s process, PROSPECT supported the ASEAN Foundation and Education Youth and Sports Division (EYSD) of ASEC’s key activities related to youth. These activities include the ASEAN Youth Video Contest (AYVC), ASEAN Youth Social Journalism Contest (AYSJC), ASEAN Youth Entrepreneurship Program, and ASEAN Foundation Model ASEAN Meeting Training of Trainers (AFMAM ToT). Support from PROSPECT included technical expertise and logistical support. The annual AYVC was conducted by the ASEAN Foundation and EYSD of ASEC to support and promote awareness about ASEAN through people-to-people interaction and collaboration with ASEAN stakeholders. The AYVC was designed for youth interested in learning new videomaking techniques and skills to effectively employ media in raising awareness and advocating for positive change in their communities. In collaboration with Impact Hub of Kuala Lumpur and ASEAN Foundation, PROSPECT hosted the ASEAN Impact Challenge (AIC) Regional Finals for youth social entrepreneurs held in Bangkok, which was officially endorsed by the Senior Official Meeting on Youth (SOMY). The annual AIC aims to strengthen innovative entrepreneurship by providing an opportunity for young innovators of ASEAN to tap into business expertise and pitch their sustainable business ideas. PROSPECT also continued supporting the ASEAN Foundation Model ASEAN Meeting Training of Trainers (AFMAM ToT), which has been supported by the USG since 2015, technically and logistically. AFMAM ToT has now become the flagship program of the ASEAN Foundation. PROSPECT worked with the ASEAN Foundation to further strengthen the sustainability of the AFMAM program, among other programs, by updating the AFMAM Guidebook. PROSPECT also contributed to funding the participation of youth representatives across 10 AMS in the AFMAM ToT. 18 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION Table 4. PROSPECT’s Interventions to Promote Human Rights and Expand Opportunities for Women and Other Vulnerable Groups PROSPECT IMPLEMENTATION STRATEGIES This section describes strategies and approaches PROSPECT has applied to implement interventions in ASEAN. PROSPECT designed its initial work plans based on the U.S.-ASEAN Strategic Partnership, drawing on the ASEAN Vision, ASEAN Blueprints, and other major documents such as ASEAN Declarations, Conventions, Leaders Statements, and Plan of Actions; ASEAN five-year work plans; and the U.S. strategy in the region. Most PROSPECT assistance is provided based on sectoral work plans, decisions, or priorities, and some are in response to the demand-driven request of sectoral bodies. Given the sensitive nature of the political and security cooperation issues, demand￾driven requests are common in the APSC Pillar compared with the ASCC Pillar. Consultation and consensus. Evaluation findings reveal that PROSPECT strategies are grounded in ASEAN’s principle of consultation in its work with various ASEAN sectoral bodies. According to the ASEAN Charter, consultation and consensus are key principles in ASEAN’s operations. As a program with many stakeholders from ASEAN and the USG, PROSPECT implementation involves ongoing consultative processes with relevant stakeholders, influencing continuous adaptations of PROSPECT strategies and activities. PROSPECT prepares draft concept notes and ToR in reference to ASEAN documents and in consultation with ASEAN counterparts to implement specific activities. However, particularly with the AICHR, PROSPECT does not participate in developing supported activities. The concept note and program are developed directly by the body before requesting PROSPECT’s support with implementation. On the other hand, PROSPECT shares its work plans with USAID, the U.S. State Department, and other agencies within the USG to ensure shared goals and priorities of USG and ASEAN are addressed. For example, PROSPECT seeks inputs from the Office to Monitor and Combat Trafficking in Persons (J/TIP) for anti-trafficking, Counter-Terrorism Office for CVE, and Labor Office for migrant worker activities. Using this approach, PROSPECT developed the following deliverables: • Bali Work Plan, in collaboration with the SOMTC Working Group on Counter-Terrorism HUMAN RIGHTS • Supported convening of the AICHR Forum on Access to Justice • Supported AICHR consultation on Freedom of Opinion and Expression (FOE) • Supported AICHR Youth Debate on Human Rights, “Advancing Partnership for Sustainability” GENDER MAINSTREAMING • Supported development of ASEAN Gender Mainstreaming Strategic Framework 2021-2025 • Supported meetings of the ACWC-ACW Ad￾Hoc Working Group, data collection and survey WPS •Developed and launched ASEAN Regional Study on WPS •Convened the Regional Symposium on Implementing Women, Peace and Security Agenda in ASEAN •Supported meetings of the ASEAN WPS Advisory Group (in collaboration with UN Women) YOUTH • Supported the ASEAN Youth Video Contest Training Workshop ASEAN Social Youth Journalism Contest • Supported the ASEAN Youth Entrepreneurship Program: ASEAN Impact Challenge Regional Finals • Supported the ASEAN Foundation Model ASEAN Meeting Training of Trainers (AFMAM ToT) FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 19 • TIP Guidelines and Toolkits, with the ACWC • Gender mainstreaming the WPS, with ACWC and ASEAN Committee on Women (ACW) • Youth engagement, with ASEAN Foundation PROSPECT also: • Works through ASEAN’s Secretariat, which serves as gatekeepers, and ASEAN sectoral bodies as the main stakeholders • Maintains constant communication with ASEAN’s Secretariat or sectoral entities throughout the activity's design and implementation to track progress and identify and address challenges • Keeps in close communication with ASEAN counterparts to address requests, or to anticipate and mitigate challenges that might arise • Leverages the opportunity with ASEAN realistically, as ASEAN had issues that are outside PROSPECT’s control, such as ASEAN bureaucracy and member states’ situation • Collaborates with other ASEAN Dialogue Partners and donors, such as with the UN Women, USAID Wildlife Asia, and International Labour Organization (ILO) to help fill gaps in assistance The track 1.5 approach. PROSPECT uses “track 1.5” to support ASEAN with other relevant agenda items, such as civil society involvement, which are typically not endorsed or covered by an ASEAN sectoral body. Track 1.5 is a strategy that uses semi-official platforms that include a mix of ASEAN or AMS government officials and CSO representatives to sit around the same table and engage. PROSPECT works directly with regional and national CSOs on such interventions. It provides a platform for policy dialogue on critical issues, such as implementing the ASEAN Consensus on Migrant Workers, involving representatives from CSOs, ASEAN sectoral bodies, and AMS representatives. PROSPECT reports the activity to ASEAN’s Secretariat or relevant sectoral bodies for their information or needed follow-up. Bilateral engagement. PROSPECT also undertakes bilateral engagements with individual member states or champions within particular sectoral bodies to support the implementation of regional interventions at the national/local level. PROSPECT has applied this strategy, for instance, in a series of provincial Multi-Disciplinary Team (MDT) Trainings on TIP Victim Identification held in Thailand for local frontline officials, labor inspectors, social workers, and health service providers. The training is jointly supported by other donors, such as DATIP, United Nations Action for Cooperation against Trafficking in Persons, USAID Thailand CTIP, and Work Vision. 20 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION Figure 3. Implementation Strategies of PROSPECT to Implement Activities STRATEGY ADAPTATIONS TO IMPROVE PROSPECT PERFORMANCE In general, PROSPECT remains committed to its work plan, and some adjustments are undertaken along the way to accommodate sectoral body circumstances and requests or when the situation changes. Most of the adaptations PROSPECT undertakes aim to accommodate circumstances or requests from ASEAN counterparts and are mostly related to the administrative and technical aspects of the support. Examples include requests to incorporate inputs in the draft documents, timelines of activities, dates for ASEAN product launch, and provision of proposed experts. In gender mainstreaming, PROSPECT had to postpone the launch of the ASEAN Gender Mainstreaming Strategic Framework to the next quarter, as some AMS have requested further revisions. In its approach to implementing CVE, PROSPECT shifted the institutional entry points from the ASEC to the chair of SOMTC of CT following the advice of the SCD of ASEC on the SOMTC preference to work. PROSPECT mostly accommodates adjustments and requests from ASEAN counterparts, as it is given the flexibility to work with ASEAN counterparts, as long as it is underneath the umbrella approval of the USG and ASEAN. In midterms, the finding reveals that major adaptions undertaken by PROSPECT are primarily to address the COVID-19 impacts. Starting in Q1 2020, PROSPECT has shifted all activities to online forms. (This will be discussed in the other section). ASEAN Follow-up of ASEAN work plans and decisions – demand driven request Consultation with both ASEAN and USG sides Working through ASEC or Sectoral Bodies Collaboration with DPs and Donors Track 1.5 (CSOs and Governments) to support civil society Bilateral engagement for implementation at the national level Committed yet flexible, and realistic FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 21 PROGRESS TOWARD EXPECTED RESULTS ACCORDING TO POLICY CONTINUUM Figure 4. Policy Continuum: Most Support and Results are at Stage Building Coalition of Stakeholders and Advancing ASEAN Regional Policy From the perspective of the policy continuum, most of the support provided by PROSPECT is on stage building, collation of stakeholders, and advancing ASEAN regional policy. PROSPECT is also at the early stage of implementing regional policy into practice at the AMS national level. At midterm, ASEAN sectoral bodies have improved their engagement and cross-sectoral collaboration with other ASEAN bodies across sectors and pillars, civil society organizations, and other regional actors in addressing challenges in each of the focus areas. This improvement is evidenced by the participation of the stakeholders in, among others, the regional cross-sectoral and multi-sectoral consultations and dialogues in developing TIP Guidelines and TIP, Bali Work Plan of CVE, Guidelines for Detecting and Preventing Wildlife Trafficking, Gender Mainstreaming Framework, WPS study, youth events, and migrant worker recommendations. ASEAN regional policies have progressed to the stage that ASEAN has adopted the following policies: • In CVE, the Workplan to Prevent and Counter Radicalization and Violent Extremism (PCRVE) 2019–2025 was adopted by the ASEAN Ministerial Meeting on Transnational Crime (AMMTC) in 2019. • In TIP, ASEAN Regional Guidelines and Procedures to Address the Needs of Victims of Trafficking in Persons, and the companion regional Model Implementation Toolkit, were adopted by ASEAN; also completed a research study on forced labor and TIP in the region's fishing industry to inform recommendations for guiding regional policy measures and new program initiatives. • In wildlife trafficking, ASEAN Guidelines for Detecting and Preventing Wildlife Trafficking were adopted by ASEAN Ministers on Agriculture and Forestry in October 2019. 22 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION • In gender mainstreaming, the ASEAN Gender Mainstreaming Strategic Framework of 2021–2025 was adopted by ASEAN. • In WPS, Regional Study on WPS, the basis for developing the Regional Plan of Action for the WPS in ASEAN, was adopted and launched by ASEAN. Support to bring policy into practice has been mostly initiated or explored in 2020 to follow up on ASEAN regional policy adoptions. For instance, in CVE, PROSPECT began supporting the development of a Training-of-Trainers curriculum on countering disinformation and establishing the Multisectoral Task Force to prevent and counter-terrorism. In gender mainstreaming, PROSPECT will start exploring the next steps for operationalizing the ASEAN Gender Mainstreaming Strategic Framework. In some areas, such as TIP, PROSPECT has developed some training modules on TIP identification and conducted training in one member state. In less political areas, such as youth engagement, capacity-building programs have been implemented earlier than other priority areas. MOST AND LEAST SUCCESSFUL/EFFECTIVE INTERVENTIONS AT MIDTERM Table 5. Most Successful Interventions at Midterm The findings reveal that PROSPECT’s workstream 1 has achieved much more success than the two other workstreams, in terms of deliverables and outcome. Specifically, PROSPECT support of CVE, TIP, wildlife trafficking, and health under workstream I; and gender mainstreaming and WPS under workstream II are regarded as the most successful interventions. The KIIs base the qualitative indicators of success on: • The adoption of the policy product by ASEAN at high political levels • Their significance and ability to push policy initiatives forward from high political commitment into practices • Number of sectoral bodies across pillars and sectors involved • The involvement of key stakeholders particularly civil society organization participation • Buy-in and satisfaction of the beneficiary CVE •Adopted and launched by ASEAN • Framework implemented for Manila Declaration and Plan of Action established •Multi-sectoral platform and coordination •Involved 19 sectoral bodies, the biggest in ASEAN • Participatory: CSOs consulted • Beneficiary satisfaction assured TIP • Guidelines and toolkits adopted by ASEAN • Realization of ACTIP and APA • Logical progression of support from policy framework to capacity building • Cross-sectoral work • Participatory approach • CSO consulted GENDER •WPS adopted and launched by ASEAN • Policy baseline for Joint Statement on WPS 2017 by HOS/G • Cross-sectoral collaboration • Participatory approach; CSO consulted WILDLIFE TRAFFICKING • Guidelines adopted by ASEAN • Crosscutting and cross-sectoral work FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 23 According to KIIs, the Bali Work Plan is the first cross pillar and multi-sectoral work plan on counter violent extremism in ASEAN adopted by the AMMTC. In TIP, the ACWC Regional Guidelines and Procedures to Address the Needs of Victims of TIP were officially endorsed at the ASEAN Senior Officials Meeting on Social Welfare Development. In terms of its significance, the Bali WP serves as the implementation framework for the ASEAN leader’s political commitment in the Manila Declaration to Counter the Rise of Radicalization and Violent Extremism of 2017 and the ASEAN Plan of Action to Prevent and Counter the Rise of Radicalization and Violent Extremism (ASEAN PoA PCRVE). Through the interventions, ASEAN established a multi-sectoral platform and coordination between the involved bodies. In TIP, PROSPECT provides comprehensive support with a logical progression in the support as it started from the development of the guidelines, its companion toolkits, training modules, and the organization of a series of training for local practitioners. The interventions also bring together representatives from different sectoral bodies, AMS representatives, practitioners of human rights, CSOs, and local practitioners and front liners officers to work on TIP victims. KII indicate the activities as the real implementation of the ACTIP. In terms of the number of sectoral bodies involved, the Bali Work Plan was able to bring together around 20 sectoral bodies, the largest in ASEAN, and is therefore seen as the most comprehensive work plan in ASEAN. Bali Work Plan took a strong participatory approach, as it was developed through a series of consultations, not only with government representatives but also with CSOs in the region. It is a new development in ASEAN, particularly SOMTC, where CSOs are given space and consulted in their security-related policy-making process. In CVE, TIP, gender mainstreaming, WPS, wildlife trafficking, and health intervention, PROSPECT has provided greater coordination and partnership between the sectoral bodies. PROSPECT contributes to the establishment of regular platforms for dialogue and coordination between the member states in the sectoral bodies. LESS SUCCESSFUL INTERVENTIONS Table 6. Least Successful Interventions at Midterm Some less successful intervention is mostly under the workstream II that includes the legislative cooperation with the ASEAN Inter-Parliamentary Assembly (AIPA), the anti-corruption with the KPK, human rights with the AICHR, including in the establishment of the formal relationship between the ASEAN-ISIS Network. HUMAN RIGHTS • Budget issues due to COVID￾19: No activities with AICHR since December 2019 RULE OF LAW: ANTI-CORRUPTION • Halted due to Indonesian internal politics related to the Corruption Eradication Commission (KPK) LEGISLATIVE COOPERATION • Unmatched priority issues with AIPA 24 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION Cooperation with the AIPA is unable to move forward due to the unmatched priority areas between the AIPA and PROSPECT Statement of Work. The activity proposed related to pedophilia is not covered by the PROSPECT, and neither is it is the focus area of ASEAN in general. Support to the anti-corruption through the Indonesian KPK has been halted due to the Indonesian internal politics factor, which is beyond the PROSPECT’s control. With human rights, in 2019 PROSPECT provided significant support to AICHR such as through AICHR-CACJ high-level dialogue, youth debate on human rights, AICHR consultation on freedom of expression in 2019. However, no further interventions have been carried out since the COVID-19 pandemic in late December. Some KIIs suggested that the AICHR prioritized face-to-face activities for certain sensitive human rights issues. IMPACT OF COVID-19 AND PROSPECT COPING STRATEGIES Impact Since the COVID-19 outbreak, PROSPECT started to work under modified operational arrangements as a result of the “working from home” and travel restrictions policy. Due to the pandemic, most activities and events of PROSPECT were slowed down or delayed. Some activities are suspended until further notice, this includes ASEAN Youth Video Contest, Talking ASEAN, support for CACJ, English training for the AHA Centre. Initially, COVID-19 was expected to last for a few months. However, since the pandemic has lasted for years, many activities were eventually canceled and the PROSPECT budget was reduced and refocused on the pandemic response. Because of COVID's travel restrictions, PROSPECT conducts remaining activities virtually. The WPS and gender studies, for instance, were done online with minimal face-to-face data collection. Despite the limitations, this does not affect the quality and deliverable of the studies. The online platform has, however, limited PROSPECT opportunities for networking and rapport building, particularly with new representatives joining different sectoral bodies. PROSPECT (including USAID) has lost the opportunity to engage with ASEAN counterparts in informal settings (such as during coffee break, lunch, and dinner of events) to build and strengthen relationships and receive direct feedback and inputs from ASEAN counterparts. Coping strategies PROSPECT began implementing activities under modified operational arrangements in Q1 2020 as a result of the COVID-19 pandemic's "working from home" policy. PROSPECT continues to implement feasible activities with ASEAN following the operational measures • No organization of in-person meetings and shift to virtual events • No project-related travel, domestically nor internationally • PROSPECT team working from home • PROSPECT will not invite nor fund a consultant or event participant to travel internationally • To use WhatsApp, Zoom, and Microsoft Teams among the team while continuing the official communication on project-related matters by email Following the budget cut, PROSPECT continues implementation activities with ASEAN counterparts based on the following criteria: 1. Maintaining relationships with ASEAN counterparts by continuing the existing commitments. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 25 2. Not making new commitments for new activities as it will have a budget implication unless additional money is available STRATEGIES FOR IMPROVING PROSPECT PERFORMANCE This section provides strategies and approaches that PROSPECT can undertake to improve performance for more effective and better results with ASEAN. Institutional entry points ASEAN is a complex institution that consists of different sectoral bodies and organs with different roles, functions, and mandates. Based on ASEAN institutional arrangements in place, each division of ASEAN’s Secretariat remains the most effective entry point for PROSPECT assistance for particular sectors of the ASEAN Socio-Cultural Pillar Community (ASCC) Pillar. In the ASCC Pillar, ASEAN Secretariat divisions have greater flexibility in performing their functions and therefore play a critical role as the gatekeepers and are able to assist in pushing the agenda in the sectoral bodies. Unlike the ASCC, in the ASEAN Political Security Community (APSC) Pillar, which is more political, ASEAN Secretariat divisions, particularly Security and Human Rights Divisions, perform a more passive role as a facilitator of sectoral bodies and AMS and Dialogue Partners. ASEC mostly waits and acts based on instructions and guidance from sectoral bodies as the main stakeholders. In this context, building and maintaining working relationships directly with sectoral bodies, either the lead shepherd, focal points, or champion instead of with the ASEC, will be more effective. Regional and domestic issues Selecting specific themes that contain solid regional dimensions or values such as trafficking in persons, migration, and terrorism which affect all 10 AMS as common regional challenges, will be ASEAN’s top priority rather than issues that affect several AMS or domestic of AMS. With ASEAN’s non-interference principle in place, it is challenging to take up activities that address domestic issues. For politically sensitive local issues such as human rights, KII suggest an approach of sharing best practices between AMS in addressing their domestic challenges. Some AMS has initiated this approach, for instance, the AICHR Indonesia initiated the Universal Periodic Review on Human Rights (UPR) model of ASEAN used as a platform to share Indonesia’s experience and best practices in implementing its obligations under the UPR framework. Alignment and buy-in In terms of the concept design for proposed activities, clear and strong justification with indicators for ASEAN consideration will also help PROSPECT obtain sectoral bodies’ endorsement and move forward with the implementation. The proposed concept notes are advised to have strong regional coverage, contribute to ASEAN blueprints, and align with ASEAN sectoral bodies mandate (five-year work plan). PEQ 2: WHAT SUPPORT HAS USAID PROVIDED TO PROSPECT WITH RESPECT TO ADMINISTRATIVE, DIPLOMATIC AND ENGAGEMENT TO SUPPORT PROJECT IMPLEMENTATION AND ASEAN IN DEVELOPING INSTITUTIONAL CAPACITY AND HUMAN CAPITAL TO BETTER ADVANCE SHARED PRIORITIES? HOW USAID/ASEAN HAS SUPPORTED PROSPECT WITH DIPLOMATIC AND ADMINISTRATIVE ENGAGEMENT AND THE EFFECTIVENESS OF THAT SUPPORT 26 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION In administrative support, evaluation participants stated that USAID contributes and participates in developing PROSPECT's work plans, technical implementation, administrative, and logistical aspects. USAID reviews and approves all PROSPECT work plans and reports. PROSPECT works with several stakeholders from ASEAN and the USG. In developing work plans, USAID/ASEAN facilitates linkages between PROSPECT relevant agencies within the USG in Washington, such as the J/TIP for anti￾trafficking, the Bureau of Counter-Terrorism for CVE work plans, and the U.S. Mission to ASEAN in Jakarta. USAID/ASEAN assists PROSPECT with its communication strategy, including reviewing and clearing communication products such as social media, factsheets, program highlights, and press releases. USAID also supports PROSPECT in developing proposals to secure funding opportunities from other potential donors. USAID meets with interagency partners to leverage existing funds and coordinates with the regional program office to link to the formal budget request process. In diplomatic support, USAID facilitates communication and meetings with PCPMD of ASEC and high-ranking officials of ASEAN. USAID has helped facilitate meetings with the deputy secretary￾general of ASEC to secure office space for PROSPECT in the ASEC building. ADDITIONAL ACTIONS THAT USAID/ASEAN CAN TAKE TO SUPPORT PROSPECT Some KII suggested additional actions that USAID/ASEAN could take to support project implementation. These ideas include the following: Increasing support for the project from USAID missions. Several thought that USAID/ASEAN could facilitate more support from the USAID mission in each ASEAN member state for project activities in the respective member state. Support ideas identified by key informants include reviewing and providing input to project documents, guiding project staff in the implementation of project interventions, facilitating local linkages, and participating actively in events organized by the project, e.g., the forthcoming WPS activity to be held in Cambodia in 2022. Funding flexibility. KII from the ASEAN Foundation expressed the need for USAID to be flexible in their funding to support Brunei and Singapore. Despite the non-eligible status of both countries to receive donor support, some AMS are concerned about this decision by donors. Also, ASEAN Foundation key informants raised their expectation for flexibility to use USAID/ASEAN funds to hire dedicated staff for the ASEAN Foundation to help with project day-to-day operations. Additional funding. KII from SOMTC stated that they expect more funding allocation from USAID to support the implementation of the Bali Work Plan. Similar to AF, SOMTC recommended that USAID/ASEAN provide funding for the SOMTC lead shepherd on counter-terrorism with a dedicated staff to monitor the implementation of the Bali Work Plan. Advancing PROSPECT work through the U.S. Ambassador to ASEAN. Some KII also suggested that the U.S. Ambassador to ASEAN could help with promoting PROSPECT’s agenda in ASEAN, particularly obtaining ASEAN endorsement of PROSPECT proposals and interventions and supporting PROSPECT with communication and networking with ASEAN to convene official meetings. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 27 PEQ3. HOW EFFECTIVE IS PROSPECT THEORY MODEL IN GUIDING THE IMPLEMENTATION OF PROJECT ACTIVITIES TOWARD ACHIEVING EXPECTED RESULTS (DEVELOPMENT HYPOTHESIS, THEORY OF CHANGE, RESULTS FRAMEWORK, OR LOGIC MODEL)? WHY OR WHY NOT? PERSPECTIVES ABOUT PROSPECT THEORY OF CHANGE Relevance to the project design and ASEAN needs Findings from the desk review and KIIs established that PROSPECT developed its Theory of Change (ToC) in alignment with the ASEAN Community Vision 2025 and its two companion Blueprints, the APSC and the ASEAN ASCC. PROSPECT's ToC is high level and echoes the purpose of the ASEAN Community Vision 2025, which is also USAID’s development goal: “A region of lasting peace, security, and stability, sustained economic growth, shared prosperity and social progress.” This goal is built on three community pillars: 1) political-security, 2) economic, and 3) socio-cultural. In 2015, ASEAN adopted the APSC, AEC, and ASCC Blueprints as the roadmap for achieving the communities outlining the ASEAN priority agenda, guidance on the implementation, resources, and review. USAID/ASEAN designed PROSPECT to support the development of two community pillars of ASEAN—the APSC and ASCC communities—to provide comprehensive support to all three ASEAN communities. For this purpose, USAID has broken down the high-level purpose into two specific Development Objectives (DO 1 and 2) or PROSPECT goals following some core elements and objectives in the APSC and ASCC Blueprint. PROSPECT interventions are based on ToC that follow ASEAN’s logical structure. The ToC hold that: • IF ASEAN addresses transnational challenges related to human security AND strengthens effective response to humanitarian crises, particularly natural disasters, THEN ASEAN capacity to resolve regional and global challenges will be strengthened. • IF ASEAN strengthens good governance and rules of law through engagement with judicial, legislative, and executive branches AND increases civil society participation in ASEAN AND expands rights and opportunities for women, youth, and other vulnerable and marginalized groups THEN ASEAN and AMS will advance effective regional integration to enhance the RoL and uphold human rights. To achieve maximum impact for each DO, PROSPECT focus its interventions on three main workstreams, including increased ASEAN capacity: 1. Address non-traditional security threats: Designing and implementing interventions to combat human and wildlife trafficking, counter violent extremism and improve response to emerging humanitarian emergencies, particularly natural disasters; aiming at Development Objective 1. 2. Promote responsive and transparent governance: Strengthening judicial, legislative, and executive body cooperation and networks to promote good governance, RoL, and compliance with ASEAN conventions and agreements; strengthening civil society networks and cultivating multi-sectoral platforms for engagement with ASEAN officials on critical issues; aiming at Development Objective 2. 3. Advocate for human rights and opportunities for marginalized and/or vulnerable groups: Enhancing ASEAN’s institutional capacity to promote human rights and to provide 28 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION opportunities for underserved populations, including engaging youth and women and supporting institutional mechanisms to ensure their meaningful participation in ASEAN processes and decision-making, aiming at Development Objective 2. Figures 5 and 6 describe the relationship between PROSPECT’s ToC and the ASEAN Vision and Blueprints. Figure 5. PROSPECT ToC and Inputs ASEAN DEVELOPMENT COOPERATION GOAL: ASEAN and its member states contribute to sustainable and inclusive growth and encourage rules-based systems to support a peaceful, secure, and prosperous Southeast Asia through partnership with USAID DO 1 ASEAN and its member states contribute more to resolving regional and global challenges WORKSTREAM 1: Increased ASEAN capacity to address non-traditional security threats PROSPECT INPUTS: 1. Provide technical support for improved cross-sectoral coordination of the ACTIP and APA 2. Support ASEAN’s efforts to CVE 3. Build technical capacity to address non-traditional security threats 4. Strengthen ASEAN's capacity to respond to emerging humanitarian crises DO 3 ASEAN and its member states advance an effective, rules-based architecture to uphold human dignity and the rule of law WORKSTREAM II: Increased ASEAN capacity to promote responsive and transparent governance PROSPECT INPUTS: 5. Strengthen linkages between ASEC, CACJ, and AIPA to promote rule of law 6. Support transforming ASEAN into a rules-based organization 7. Cultivate and strengthen civil society networks for effective engagement with ASEAN on critical issues WORKSTREAM III: Increased ASEAN capacity to advocate for the human rights of marginalized and/or vulnerable groups PROSPECT INPUTS: 8. Support AICHR and other actors to promote, protect, and mainstream human rights 9. Engage youth to contribute to ASEAN processes and strengthen shared ASEAN identity 10. Expand the rights and opportunities of women and other vulnerable/marginalized groups FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 29 Figure 6. ASEAN Visions, Priorities, and Sectoral Bodies Work Plans The findings suggest that the ToCs effectively guide the implementation of interventions to achieve PROSPECT results. The key to the ToC's effectiveness is its close alignment with ASEAN’s policy framework and the consultations and endorsement process applied by ASEAN. This process ensures that PROSPECT interventions respond to ASEAN objectives and needs. PROSPECT developed a result framework comprising ten inputs and projects that relate to key implementation frameworks of ASEAN sectoral bodies and entities, including their five-year work plans. To implement a particular activity under each PROSPECT work plan, ASEAN requires PROSPECT to develop draft concept notes and Terms of References (ToR) in consultation with sectoral bodies, who also must approve the ToR before PROSPECT implements them is able to move forward with implementation. Some sectoral bodies under the APSC apply a demand-driven request approach. By this approach, APSC submits their proposals (concept notes and ToR) to PROSPECT to request specific support for their activities. This demand-driven approach has helped PROSPECT ensure that program support effectively meets the real needs of ASEAN sectoral bodies, which has enhanced results achievement. PROSPECT also supports the implementation of ongoing activities and programs of ASEAN bodies/organs, such as youth programs with the ASEAN Foundation. Such support is in line with the provisions of PROSPECT ToC. PROSPECT’s ability to rapidly respond to demand-driven requests from ASEC and ASEAN sectoral bodies was identified by key informants as a major strength of PROSPECT support. Tables 7 through 11 summarize ASEAN frameworks, PROSPECT support, and performance results at midterm across the different frameworks. ASEAN COMMUNITY VISION: A region of lasting peace, security and stability, sustained economic growth, shared prosperity, and social progress APSC OBJECTIVES - Rule-based ASEAN, - A resilient community with enhanced capacity to respond to global challenges ASEAN PRIORITIES Strengthen ability to deal with non￾traditional challenges and crisis situations Sectoral Bodies/Entities Work Plan and Decisions ASEAN PRIORITIES: Strengthen democracy, good governance, RoL, human rights and fundamental freedoms as well as combat corruption Sectoral Bodies/Entities Work Plan and Decisions ASCC OBJECTIVES Community that engages and benefits the people, inclusive, sustainable, resilient, and dynamic ASEAN PRIORITIES Engage stakeholders; promote access for ASEAN peoples, women, children, youths, migrant workers,, and vulnerable and marginalized groups Sectoral Bodies/Entities Work Plan and Decisions 30 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION Table 7. Results in Increasing Capacity to Address Non-Traditional Security DO1: ASEAN AND ITS MEMBER STATES MORE CAPABLY AND FREQUENTLY CONTRIBUTE TO RESOLVING REGIONAL AND GLOBAL CHALLENGES WS 1: INCREASE CAPACITY TO ADDRESS NON-TRADITIONAL SECURITY ASEAN IMPLEMENTATION FRAMEWORKS PROSPECT INPUTS ACTIVITIES UNDERTAKEN RESULTS • ACTIP and APA • Bohol TIP Work Plan • ACWC Work Plan • SOMTC Work Plan • Plan of Action to Implement the ASEAN-U.S. Strategic Partnership • Provide technical support for improved cross-sectoral coordination on implementing the ACTIP and APA • Launched the ACWC Regional Guidelines on TIP • Provided consultant expert for developing Practitioners’ Model Toolkits • Translated the Toolkits into six AMS languages • Initiated the ACTIP Implementing Partners Coordination Meetings • Initiated a regional study for a common approach on TIP in the region’s fishing industry • State-specific policy toolkits for capacity building and resources established • Multi-sectoral platform and coordination established • Engagement with CSOs, Indicator #8: 39 percent achievement (2019–2021 YTD) • Programmatic coordination and collaboration across regional development partners • Manila Declaration • ASEAN PoA PCRVE • ASEAN PoA Combating Transnational Crimes (2016– 2025) • PoA to implement the ASEAN-U.S. Strategic Partnership • Support ASEAN’s efforts to counter violent extremism, including supporting multi-sectoral coordination • Supported the development of the Bali Work Plan through a multisectoral process • Provided consultant expert for developing Bali Work Plan • Convened ASEAN Cross-Sectoral and Cross-Pillar Meetings and Consultations • Regional implementation framework established, Indicator #7: 35 percent achievement (2019–2021 YTD) • Multi-sectoral platform and coordination established. • Engagement with CSOs • ASEAN PoA Combating Transnational Crimes • PoA to Implement the ASEAN-U.S. Partnership • Provide technical capacity building to ASEAN to address non-traditional security threats, such as cybercrime, wildlife trafficking • Provided technical expert for developing ASEAN Guidelines to Detect and Prevent Illegal Trafficking • Policy toolkits adopted • Multi-sectoral coordination, Indicator #5: 98 percent achievement (2019–2021YTD) FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 31 Table 8. Results in Strengthening ASEAN Capacity to Respond to Emerging Humanitarian Crises DO1: ASEAN AND ITS MEMBER STATES MORE CAPABLY AND FREQUENTLY CONTRIBUTE TO RESOLVING REGIONAL AND GLOBAL CHALLENGES WS 1: STRENGTHEN ASEAN CAPACITY TO RESPOND TO EMERGING HUMANITARIAN CRISES ASEAN IMPLEMENTATION FRAMEWORKS PROSPECT INPUTS ACTIVITIES UNDERTAKEN RESULTS • Plan of Action to Implement the ASEAN-U.S. Strategic Partnership • ASEAN Agreement on Disaster Management and Emergency Response (AADMER) Work Program 2021– 2025 • Strengthen ASEAN capacity to respond to emerging humanitarian crises • Conducted language course for AHA Centre officials to enhance English skills for coordinating and collaborating during times of disasters • Conducted capacity needs assessment of ASEAN on humanitarian logistics in 2020 • Supported development of multi-sectoral ASEAN Pandemic Preparedness and Response (PPR) Framework • Enhanced capacity and increased awareness • Study assessment established • Multisectoral framework and coordination for addressing emerging pandemic threats and crises. 32 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION Table 9. Results in Increasing ASEAN Capacity to Promote Responsive and Transparent Governance DO3: ASEAN AND ITS MEMBER STATES ADVANCE AND EFFECTIVE, RULES-BASED ARCHITECTURE TO UPHOLD HUMAN DIGNITY AND THE ROL WS 2: INCREASE ASEAN CAPACITY TO PROMOTE RESPONSIVE AND TRANSPARENT GOVERNANCE ASEAN IMPLEMENTATION FRAMEWORKS PROSPECT INPUTS ACTIVITIES UNDERTAKEN RESULTS • APSC Blueprint (Paragraph A.2.4, A.2.6) • CACJ Work Plan • Build capacity of and strengthen linkages between ASEAN’s Secretariat, CACJ, and AIPA to promote RoL in ASEAN • Worked closely with the KPK to organize workshop on Private Sector Corruption, bringing together 29 SEA-PAC representatives from eight AMS • Provided legal expertise for drafting CACJ WG￾JET Work Plan (2020– 2025), which serves to operationalize the CACJ WG-JET Strategic Plan (2018–2025) • Worked closely with the KPK to organize workshop on Private Sector Corruption, bringing together 29 SEA-PAC representatives from eight AMS • Work Plan adopted, which lays out priority areas for training and provides guidance for effectively managing and implementing regional training programs for judges • Shared best practices and experiences related to combating corruption in the private sector, including strategies • APSC Blueprint (Para 7.A 1.2) • SLOM decision • Support the process of transforming ASEAN into a rules-based organization, including improving implementation of agreed-upon ASEAN instruments • Supported the participation of SLOM representatives at the first-ever ASEAN High￾level Policy Dialogue on Greater Cross-Pillar Collaboration in ASEAN on Free Trade Agreements • ASEAN participation in the cross-pillar platform for the ASEAN Economic and Socio￾Cultural Communities, exchanges views on matters relating to labor and environment in the context of free trade agreements FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 33 Table 10. Results in Increasing Civil Society Participation in ASEAN DO3: ASEAN AND ITS MEMBER STATES ADVANCE AND EFFECTIVE, RULES-BASED ARCHITECTURE TO UPHOLD HUMAN DIGNITY AND THE ROL WS 2: INCREASE ASEAN CAPACITY TO PROMOTE RESPONSIVE AND TRANSPARENT GOVERNANCE— INCREASE CIVIL SOCIETY PARTICIPATION IN ASEAN ASEAN IMPLEMENTATION FRAMEWORKS PROSPECT INPUTS ACTIVITIES UNDERTAKEN RESULTS • APSC Blueprint • ASCC Blueprint • Cultivate and strengthen civil society networks to support existing platforms and create new ones for effective engagement with ASEAN on critical issues • Support formal linkages between ASEAN-ISIS Network and Asian Secretariat (ASEC) Completed an assessment report on between ASEAN ISIS • Engaged a technical consultant to support ASEC in crafting the TOR on ASEAN-ISIS Network partnership • Conducted 3 National Policy Dialogues (NPDs) to promote engagement between state and non-state stakeholders on migrant workers’ rights and protection and to develop strategies on the implementation of the ASEAN Consensus. (HRWG) • Trained youth from all 10 AMS on storytelling on tolerance, and moderation within their communities (CERITA) • Undertook a series of Talking ASEAN events to bring together government officials and civil society stakeholders VE prevention. • Endorsement to support the formalization of partnership received • Networking among CSOs in the region • Strategies of engagement and recommendations established • Networking among CSOs established • Potential collaboration halted 34 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION Challenges with applying the ToC The main challenge with PROSPECT ToC is the difficulty and lack of mandate to support and follow up on country-level policy implementation. ASEAN lacks an appropriate mechanism for supporting and monitoring the implementation and performance of PROSPECT-supported regional policies by each AMS. The absence of a country-level monitoring system within ASEAN limits understanding of the fulfillment of respective state-level commitments within each sector. ASEAN works at the regional level when developing regional policies for the ASEAN Vision. However, PROSPECT ToC does not provide clear and firm guidelines for operationalizing ASEAN’s agenda at the state level. This gap is strongly related to ASEAN Way principles of non-interference and voluntary adoption and implementation of policies at the member-state level. ASEAN has established some legally binding agreements to mitigate this gap, such as the ACTIP. However, as the Table 11. Results in Expanding Rights and Opportunities for Women, Youth, and Other Vulnerable and Marginalized Groups DO3: ASEAN AND ITS MEMBER STATES ADVANCE AN EFFECTIVE, RULES-BASED ARCHITECTURE TO UPHOLD HUMAN DIGNITY AND THE ROL WS 3: EXPAND HUMAN RIGHTS AND OPPORTUNITIES FOR MARGINALIZED AND VULNERABLE GROUPS ASEAN IMPLEMENTATION FRAMEWORKS PROSPECT INPUTS ACTIVITIES UNDERTAKEN RESULTS • APSC Blueprint • ASCC Blueprint • AICHR Work Plans • ACWC Work Plans • Support the AICHR and other regional actors to promote, protect, and mainstream human rights issues in ASEAN • Engage youth to contribute more often to ASEAN processes and strengthen shared ASEAN identity • Expand the rights and opportunities of women and other vulnerable and marginalized groups • AICHR Forum on Access to Justice • AICHR Consultation on FOE • AICHR Youth Debate on Human Rights • AUN-HRPE Launch and Regional Workshop • AICHR Forum on Access to Justice • Supported development of the ASEAN Gender Mainstreaming Strategic Framework 2021–2025 • Regional Symposium on Implementing Women, Peace, and Security Agenda in ASEAN • Developed and launched ASEAN Regional Study on WPS • ASEAN Youth Video Contest Training Workshop • ASEAN Youth Entrepreneurship Program • ASEAN Foundation Model ASEAN Meeting Training of Trainers (AFMAM ToT) • Recommendations on FOE developed and submitted • ASEAN alumni network established, with contributions from alumni • CSO engagement and participation • Promotion of human rights and women’s rights in ASEAN FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 35 ACTIP has no reporting and monitoring system, it is impossible at the moment to measure the Convention's implementation progress and results. PROSPECT also: • Supports follow-up actions with AMS through bilateral agreements with some member states to close the gap. • Collaborates with focal points and champions from sectoral organizations who are actively involved and willing to coordinate, such as in the implementation of the TIP Training in Thailand. • Collaborates through the ASEAN Champions Meeting, which brings together ASEAN champions in the region and each USAID mission in ASEAN member states to carry out national activities that support USAID's ASEAN/PROSPECT goals. • Has two ToCs, which presents a broad scope for the project. Greater resource investments are needed to implement PROSPECT's broad scope effectively. PROSPECT theory of change lessons and improvement recommendations Regional versus country/state-level support. The ToC provides adequate guidance for implementing interventions at the regional level; however, it does not provide the same level of guidance for policy implementation within the member states. To address this issue, PROSPECT should continue creating and strengthening its relationship with the champions from AMS working in the sectoral bodies and using the Champions Meeting as an avenue for assisting AMS to fulfill regional commitments within its respective countries. Alignment of the ToC with ASEAN priorities and needs. The alignment between PROSPECT ToC and ASEAN policy frameworks is the most important factor for defining the relevance and effectiveness of the ToC relative to the needs of ASEAN. That close linkage establishes common grounds between PROSPECT and ASEAN for guaranteeing mutually beneficial consultative and implementation processes and engagements with ASEAN sectoral/entities bodies. Other critical factors to consider The key assumptions of PROSPECT—ASEAN commitment, ASEAN partners commitments, and USG commitment—are still valid and consistent with the current ASEAN dynamics. Other critical factors for PROSPECT and USAID/ASEAN to consider in their design and implementation include the specific effects of the internal political situation in the AMS on the project, e.g., in Indonesia in the case of the project with the KPK, the coup in Myanmar, and the COVID-19 pandemic. Both factors have substantially impacted PROSPECT's initiatives. The pandemic, for example, has been the primary driver of changes in PROSPECT's strategy and interventions in ASEAN. ADDITIONAL INDICATORS FOR MEASURING PROSPECT RESULTS Apart from the existing indicators, some KIIs suggested additional indicators to measure PROSPECT results adequately. These include the following: 1. Number of sectoral bodies across sectors and pillars involved in PROSPECT￾supported activities Justification. PROSPECT has successfully provided multi-sectoral collaboration platforms for various ASEAN stakeholders for different priority issues; still, it is not yet included as a contributing factor to the KPIs. With the ASEAN Community's top priority to streamline cross-sectoral and multi-sectoral cooperation in three ASEAN pillars, PROSPECT should 36 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION add this indicator to help it assesses its contribution to ASEAN community integration efforts. 2. Number of male participants in gender mainstreaming activities Justification. According to KIIs from the gender sector, the present indicators primarily measure the number of female participants in gender activities, but not males. PROSPECT should therefore measure male participation in PROSPECT-sponsored activities for gender mainstreaming. The consideration is that gender equality is not just a women’s issue but everyone's issue and therefore should fully engage women and men alike. 3. Number of supported entities incorporating gender mainstreaming in their work plans and documents Justification. Some KIIs requested PROSPECT not only to focus on indicators that measure people-related results but also on documents and work plans of ASEAN. They suggested PROSPECT include the number of supported entities integrating gender mainstreaming in their work plans or policy documents as part of the effort to assist gender mainstreaming throughout ASEAN's sectors and pillars. Evaluation participants also recommended that more qualitative indicators be included in the MEL plan instead of having only indicators whose unit of measurement is a number. For instance, the number of participants joining the capacity-building program might not necessarily reflect the effectiveness of the activity; sometimes the higher the number, the more likely it is ineffective. The capacity-building program will be more effective if it is attended by concerned officials and parties who hold strategic and influential positions. PEQ 4. WHAT CHALLENGES HAS PROSPECT FACED WITH IMPLEMENTING AN ASSISTANCE PROGRAM THAT SPANS SUCH A BROAD SPECTRUM OF ISSUES? BENEFITS OF THE BROAD-SPECTRUM APPROACH OF PROSPECT The broad design of PROSPECT creates several opportunities for PROSPECT to engage in different areas of the APSC and ASCC pillars of ASEAN cooperation. These two pillars involve engaging with multiple ASEAN sectoral bodies and entities. PROSPECT also has the flexibility to shift its support quickly from one area to another as the need arises or undertake new initiatives that emerge. The current approach resonates with ASEAN’s agenda on cross-sectoral cooperation. It gives PROSPECT a better understanding of the linkages between ASEAN bodies and the cross-cutting priorities and issues that PROSPECT can support across the pillars of ASEAN cooperation. PROSPECT IMPLEMENTATION CHALLENGES Evaluation participants identified three key challenges that have been encountered in implementing PROSPECT, arising mainly from the project’s wide scope: 1. Reaching consensus is time-consuming. PROSPECT works with a wide range of stakeholders from several sectors and backgrounds, including government and non￾government institutions that have different priorities and expectations. Bringing together these various ASEAN stakeholders and getting them to reach consensus takes much more effort and time. As an example, in developing the Bali Work Plan, PROSPECT had to facilitate and accommodate different ideas and perspectives on CVE from 19 sectoral bodies across sectors and CSOs in the region. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 37 2. Needs sometimes outmatch resources. The shifting needs of ASEAN and sectoral bodies require PROSPECT to be constantly flexible in its resource allocation, including financial and human resources (HR) to meet current and emerging needs. Resource limitations pose the risk of poor performance, which could make implementation difficult and affect PROSPECT’s performance and results if needs outmatch resources. PROSPECT has mitigated HR limitations by hiring consultants to fill HR gaps. However, budgetary constraints dictate the extent to which PROSPECT can rely on consultants to fill PROSPECT’s HR gaps. 3. The review process for PROSPECT’s draft work plans, including draft TORs, concept notes, and policy products, is cumbersome. Because PROSPECT supports multi-sectoral and cross-sectoral policy development, the process is often time-consuming, as it involves multi-stakeholders and multi-stage review and endorsement by ASEAN. With the wide range of PROSPECT activities, the volume of documents subject to review and approval is large and takes longer to complete, affecting the implementation timelines. TYPE OF ASSISTANCE THAT USAID SHOULD PRIORITIZE AND THE CRITERIA FOR PRIORITIZING ASSISTANCE Evaluation findings show that the prioritization of assistance is highly dependent on the timelines and policy continuum of each support area. Most KIIs suggested that PROSPECT continue the current types of support that target strengthening technical expertise, administrative and logistical support, and overall capacity building. Participants noted that USAID should expand its support following ASEAN needs according to the policy continuum, as follows: • Regional policy framework: to support the development of political statements, declarations, conventions, and regional/sectoral bodies work plans • Practical toolkits, Guidelines: to support the development of practical guides to help implement the new policy/decision • Capacity-building program: to strengthen the implementation of new policies. This should involve developing regional training modules, training, and facilitation of learning trips and awareness-raising programs • Shared learning and exchange of best practices: to provide a platform for AMS to learn about each other’s work and results and carry out joint initiatives that are based on regional policies • Coordination: to support cross-sectoral coordination (such as by providing dedicated staff to monitor implementation of the Bali Work Plan) CONCLUSIONS The midterm performance evaluation of PROSPECT investigated the project’s strategy and implementation adjustments, how USAID has supported project implementation, the effectiveness of the PROSPECT theory model, and challenges experienced mid-way through project implementation. Based on the findings, the evaluation team concludes that: PROSPECT’S STRATEGY AND IMPLEMENTATION ADJUSTMENT The implementation of PROSPECT is progressing well despite the challenges arising from the project breadth, the diverse needs of ASEAN, and limitations to working directly with member states. PROSPECT has implemented several interventions at mid-point, adjusting its strategy to accommodate the multiple needs of ASEAN and its sectoral bodies. USAID’s design of PROSPECT and the project’s strategic approach closely respond to the needs and 38 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION priorities of ASEAN. The main emphasis of PROSPECT support is ASEAN regional bodies, including AMS, and CSOs. PROSPECT’s performance at the member state level is still limited due to the lack of a mechanism to follow through at the country level and budget constraints. The interventions that PROSPECT has implemented are achieving expected results. The technical assistance provided by PROSPECT is comprehensive and, at midterm, it has contributed to the achievement of key results in building coalitions of stakeholders and advancing ASEAN policy. In midterms, major adaptions undertaken by PROSPECT are primarily to address the COVID-19 impacts rather than issues around the relationship between PROSPECT and ASEAN Counterparts. USAID SUPPORT USAID’s support has been instrumental to the implementation of PROSPECT in the following ways: • Contributing and participating in developing PROSPECT work plans • Helping facilitate relationships with ASEAN high-rank officials and relevant agencies within US Government in Washington, and • Assisting in securing funding from other potential donors Despite the support received to date, PROSPECT needs additional help from USAID, particularly to enhance PROSPECT’s engagement with USAID missions in the AMS and the U.S. ASEAN Ambassadors’ office, additional funding, and budget flexibility to accommodate emerging and shifting needs of ASEAN and their regional bodies. PROSPECT THEORY OF CHANGE The PROSPECT ToC is relevant and effective in guiding the project design and implementation. However, its wide breadth could pose a risk to implementation success unless a commensurate level of resources is provided. The ToC presents multiple critical dimensions of support to ASEAN, including elements of human security, humanitarian crises, capacity strengthening, good governance and rules of law, civil society participation, rights and opportunities for women, youth, and other vulnerable and marginalized groups and regional integration. To achieve the expected results in this diverse, theoretical scope calls for a wide range of technical experts, larger financial commitment from USAID, and sound coordination mechanisms across ASEAN entities and the AMS. Findings from this evaluation also show that PROSPECT ToC and logic model does not consider critical factors in ASEAN that might affect the achievement of results, e.g., the internal political situation in ASEAN member states and pandemics. THE BROAD-SPECTRUM APPROACH OF PROSPECT The broad design of PROSPECT offers ample opportunities for PROSPECT to be involved and undertake more strategic and comprehensive interventions across community pillars of ASEAN. PROSPECT also has a window of opportunity to expand and enhance other forms of support following each policy continuum of ASEAN in different focus areas and thematic issues with other key sectoral bodies. This could be in the form of capacity building programs, exchange best practices, dedicated resources for cross-sectoral coordination of ASEAN. However, this could only be exercised effectively if PROSPECT is equipped with sufficient human resources and budget. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 39 RECOMMENDATIONS PROSPECT’s strategy and implementation adjustment • PROSPECT should continue strengthening its alignment with ASEAN’s agenda, particularly sectoral bodies’ 5-year work plan and annual priority program and activities. ASEAN sectoral bodies and organs work in reference to their adopted plans and decisions, and in this regard, PROSPECT is expected to put ASEAN’s decision as a priority and support the realization and implementation of those work plans and decisions • PROSPECT should continue strengthening buy-in from ASEAN counterparts and focusing on priority areas with stronger regional/cross-border dimensions. PROSPECT should maximize its close relationship and engagement with ASEAN counterpart to ensure a stronger participatory approach from the design and implementation of programs and activities. • Given that COVID-19 has been the major contributor of PROSPECT’s strategy adjustments, PROSPECT should adjust and adapt its design to accommodate offline and online activities (hybrid) to keep moving forward with the implementation activities. USAID support • USAID/ASEAN should provide more support to PROSPECT in creating a strong and supportive relationship with USAID missions located at the AMS level and foster collaboration with the ASEAN Champions Meeting to help PROSPECT in implementing the regional policy/decisions at the country level. PROSPECT Theory of Change • The ToC and logic model should be revised to take into consideration other critical factors in ASEAN that might affect achieving results, e.g., the internal political situation in ASEAN member states and pandemics The broad-spectrum approach of PROSPECT • PROSPECT should enhance other forms of support such as exchanging best practices among ASEAN member states and providing dedicated resources for cross-sectoral collaboration and coordination initiatives in ASEAN, such as in the context of the Bali Work Plan. • To that end, USAID should consider allocating more funding and resources for PROSPECT to ensure PROSPECT’s needs for more extensive and comprehensive support to ASEAN are met. 40 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION API PROJECT FINDINGS AEQ 1: WHAT ARE THE MAIN SUCCESSES OF THE API ACTIVITY? AND WHAT CHALLENGES IS THE ACTIVITY FACING IN ACHIEVING ITS INTENDED OBJECTIVES OR VISION? PROGRESS IN IMPLEMENTING PLANNED TASKS Identifying needs and opportunities. A primary task of API is conducting research and consultations to identify needs and opportunities for supporting the implementation of targeted policies by AMS. This task is premised on the understanding that each member state has distinct characteristics and specific needs. API’s research studies provide an overview of the gaps and needs that the project aims to address and the technical assistance that API provides for AMS. API has selected four target AMS for initial research and consultations: 1) Cambodia, 2) Laos, 3) Myanmar, and 4) the Philippines. API selected these countries because they joined the ASW1 most recently and thus needed support in preparing their trade exchange documents. Cambodia API completed two mapping studies in Cambodia, including mapping the country’s status on ASW and ASEAN Framework of Digital Data Governance and identifying needs and opportunities. The mapping exercise established that Cambodia has limited ICT infrastructure support for e-commerce capacity and HR, technical, and funding resources. However, the country started implementing initiatives for streamlining customs, investment, and trade procedures. The Cambodian government started its NSW platform in 2017, led by the Department of Customs and Excise in the Ministry of Economy and Finance. Five other ministries, including the Council of Development of Cambodia, Ministry of Health, Ministry of Commerce, and Ministry of Industry, Science, Technology, and Innovation use the platform. The Cambodian government also published the NSW Blueprint in 2014, supported by the World Bank to develop the NSW. API established that Cambodia believes in its capacity to implement some of ASEAN’s priorities based on the capacity built from previous support from the World Bank. Despite this confidence from the Cambodian government, existing legal frameworks in the country are weak and have hindered ASW development. The poor ICT infrastructure, technical issues, funding, and low HR capacity impede the growth of Cambodia’s e-commerce sector. For the Digital Data Government (DDG), the Cambodian government published its ICT Master Plan in 2014. Despite this notable progress, awareness of these initiatives is low among stakeholders. API plans to provide technical support to Cambodia to increase awareness of NSWs and ASWs, but as of this midterm evaluation, no support has been provided. Key informants from ASEC noted that Cambodia has passed end-to-end testing to conduct the live exchange of the ACDD2 on its own and had started the exchange of the ACDD through ASW at the end of 2020. However, the implementation/process was still not completely effective, as 1 The ASEAN Single Window (ASW) is a regional initiative that connects and integrates the NSWs of AMS, thereby allowing for electronic exchange of data between the AMS. 2 The ACDD is an electronic document to facilitate the exchange of export declaration information between exchange￾ready AMS. It contains a specific set of TradeNet export permit data, which will be sent to the Customs authority in the importing exchange-ready AMS for the purpose of supplementing risk management by the Customs authority in the importing country. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 41 Cambodia still needs to improve several areas, such as increasing and enhancing human resources for ASW and DDG development; improving the infrastructure; and finding a strategy to prevent ICT failures. The slow development of digital technology also affects awareness-raising efforts, which is also one of the gaps that API could fill. Laos API conducted research in Laos to identify the current status of the Lao Government in implementing ASW and DDG policy areas. Critical needs about NSW status development are used for devising and providing specific technical assistance in addressing the main gap in Laos. However, the findings report on the baseline study is yet to be produced. Preliminary findings show that, unlike other AMS that started early, at the time of this midterm evaluation, the Laos government had implemented NSW activities only for a short time (two years, 2019–2021). Like Cambodia, Laos faces infrastructure limitations related to ICT, which hinder operationalization of the electronic commerce (e-CO) system developed and operated by the Ministry of Industry and Commerce to link and exchange information with the Laos NSW. As a recent entrant to the ASW, as of 2021, Laos had not yet joined the exchange of ACDD. Other gaps identified include shortage and weak HR capacity, and low public- and private-sector awareness of ASW and DDG. API planned to support the Laos government in increasing 1) its capacity to operate live exchange of ATIGA Form D3 , Common Message Exchange Header, and ACDD, and 2) public and business stakeholders’ awareness of the benefits of establishing and operating the NSW and ASW. However, at the time of this midterm evaluation, none of these activities had been conducted because API has not yet entered into a cooperation agreement with the Laos national government. The need to address identified barriers is immediate, particularly upgrading and integrating the Laos NSW e￾commerce system and addressing HR limitations. Myanmar Myanmar has three phases for developing its NSW: • Phase 1: Establish the necessary basic functionality within the certificate of origin (CO) issuing authority of the Ministry of Commerce e-CO system to be able to send out the e￾ATIGA Form D as well as the CO receiving authority of the Customs Department to be able to receive the incoming e-ATIGA Form D and generating the RES. • Phase 2: Expand the architecture of the system development to be able to exchange ACDDs, and e-SPS Certificate through the ASW live environment. • Phase 3: Evolve the process to develop a fully functional NSW. API has planned and listed three support areas for Myanmar: 1) mapping the current status of the country for the ASW and ASEAN Framework for DDG, 2) mapping the current status of the ASW and ASEAN Framework on DDG for prioritizing and selecting needs and opportunities for AMS, and 3) increasing public and business stakeholder awareness of the benefits of NSW establishment and the ASW live operation of DDG. At midterm, API had not undertaken these activities except a desk review, whose draft report was under review by API at the time of this midterm evaluation. API paused implementation in Myanmar because of the current suspension of USG activities due to political instability and zone restriction resulting from the military coup and presidential election issues. 3 An ATIGA Form D issued electronically by the issuing agency and sent electronically to the importing Customs through the secure government and ASW networks in accordance with the Protocol for the Legal Framework. Certificate of Origin issued under ATIGA to provide preferential duty rates for goods imported into an AMS from another AMS. 42 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION Despite joining the ASW live operation in 2019 and completing end-to-end tests in the same year, the API desk review found that Myanmar is still in NSW implementation Phase 1, making it eligible to receive only ATIGA Form D. The main gaps for improving NSW system development, based on the desk review, are similar to those of other countries that recently joined ASW, and include technical capacity and ICT Infrastructure limitations. API desk review findings show Myanmar’s NSW system is limited to less than 50 percent of its information and technology infrastructure capacity needs. Despite this limitation, KIIs from ASEC stated that Myanmar started conducting live exchange of ACDD on its own through ASW at the end of 2020. Other activities that API included in its plan but had not started at this midterm evaluation include increasing Myanmar’s ability to perform live exchange of ATIGA Form D operations, Common Message Exchange Header, and ACDD. The Philippines API has achieved greater success in its work in the Philippines compared with the other three countries. The project completed three of four planned tasks, including 1) mapping the current status of the Philippines for the ASW and ASEAN Framework of Digital Data Governance, 2) prioritizing and selecting needs and opportunities for AMS under ASW and DDG areas, and 3) increasing the country’s ability to perform live exchange of ATIGA Form D, Common Message Exchange Header, and ACDD. At the midterm evaluation, work was still in progress for the fourth task—increasing public and business stakeholders’ awareness of the benefits of NSW establishment and the ASW live operation. Identified needs relate to the several hurdles that the Philippines has faced in developing its NSW system. These hurdles, dating as far back as 2005, include challenges arising from leadership changes, procurement impediments, and technical issues. The NSW operations and its accompanying platform (TradeNet) need improvements in the procedures for processing applications, permits, licenses, and clearance for import and export goods. Through formal direct agreement, the engagement between API and BOC Philippines has been nurtured by both parties, as USAID COR familiarized API Project into USAID Mission in the meeting, the support has also been there to invigorate the assistance. Bureaucratic challenges and sparse coordination within government agencies have influenced several member states to develop trade systems independently, and the Philippines is no exception. The Philippines adjusted the platform goals and processes to be relevant to the agencies’ status and integrated 70 government agencies involved in cross-border transactions through the Philippines NSW platform. Delays in the procurement process have impacted the solicitation of technical resources for the ASW platform development due to the limited number of bidders responding to procurement requests. To address this limitation, API assisted the Philippines Bureau of Customs (BOC) in revising and simplifying the technical requirements of their Terms of Reference (ToR). The revised ToR attracted qualified bidders, and a winning bid was selected. API’s work to increase awareness of public and business stakeholders about NSW and to strengthen the system capacity is still in progress. Recently, API started developing outreach and communication plans to increase business awareness of the benefits of NSWs and the ASW in the ASEAN region. API supported BOC in making Technical Reference Guidelines to raise public and business stakeholders’ awareness of the benefits of NSW establishment and the ASW live operation. API has completed a baseline study in the Philippines to measure the current status, priority, needs, and opportunities of ASW and ASEAN Framework on DDG. The direct support provided by API to the Philippines to develop the ASW Platform helped Philippines BOC to achieve its target to FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 43 exchange the ACDD documents following the successful completion of the end-to-end testing of the system in 2021. However, several shortcomings remain. For example, the trade system still requires modernization and system-level difficulties preventing small and medium enterprises (SME) from accessing the portal persist. Refining API approaches to opportunities based on evidence. API provides technical support to some country-specific projects at the request of the respective member states. For example, in the Philippines, the Bureau of Customs requested API’s support in developing tools and procurement of ICT services for the exchange of ACDD portals. API also provided technical assistance for language translation of ASEAN’s certification handbook on trade and customs within the AMS. Initially, the handbook was written in English, which was less useful in member states whose customs officials and businesses do most of their work in their local language. To mitigate this limitation, API translated the handbook into six languages (Malay, Khmer, Indonesian, Lao, Burmese, Thai) and published it in five or six languages. API has also initiated the procurement of vendor services to translate the ASEAN-Wide Self-Certification (AWSC) Guidebook into six local languages aforementioned for seven countries because Brunei Darussalam and Malaysia use Malay as requested by AMS. API’s implementation approach involves conducting weekly check-in meetings between the national researchers and the API team. API has facilitated four learning and knowledge transfer sessions between 2020 and 2021. These sessions were conducted with national researchers after completing each research and were attended by ASEAN stakeholders. ASEAN member states have benefited from these knowledge-sharing sessions by learning about the status and results of API-supported activities across the AMS. API implemented a bottom-up approach, which begins by providing direct support to some of the AMS and then connecting the country-level system with the ASW to allow the exchange of electronic documents. MAIN RESULTS ACHIEVED AT MIDTERM Performance on core project indicators. Table 12 summarizes the midterm performance of API on project indicators. These indicators were selected to measure API performance in its support for USAID/ASEAN initiatives under the United States’ IPS, the USAID/ASEAN DCF, the ASEAN Community Vision 2025, and the strategic alignment with USAID missions in the ASEAN region. Overall, of the 13 indicators in the API MEL plan, performance has been achieved and surpassed in just one indicator, partial results have been achieved on eight indicators, and no results have been realized on three indicators. One indicator pertaining to intra-regional trade share has no target, making it difficult to assess progress at midterm. The indicator “Number of World Trade Organization (WTO) trade facilitation agreement provisions supported by USG assistance” is measured under two domains: standard indicator under foreign assistance and the IPS. Similarly, the indicator “Number of technical assistance potential packages approved by USAID” is listed twice—under IR1 and IR3, but reported performance in the two result areas is different. 44 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION Table 12. Performance of API in Core Project Indicators TYPE/SOURCE OF INDICATORS INDICATORS MIDTERM INDICATOR RESULTS/STATUS IPS indicators Number of WTO trade facilitation agreement provisions supported by USG assistance (API Goal 1; also IPS: O1.2) In progress. This indicator is measured in four targeted AMS: 1) Cambodia, 2) Laos, 3) Myanmar, and 4) the Philippines. API has achieved planned results in the Philippines but has yet to achieve results in the other target countries. Number of commerce laws and regulations simplified and implemented in accordance with international standards as a result of USG assistance (API Goal 3) Not achieved. The indicator targeted six AMS: 1) Cambodia, 2) Indonesia, 3) Laos, 4) Myanmar, 5) the Philippines, and 6) Vietnam. API has not achieved the expected results in any of these target countries. API custom indicators International technical standards adopted (API Goal 2) In progress. API targets providing support to seven AMS for international technical standards: 1) Cambodia, 2) Indonesia, 3) Laos, 4) Myanmar, 5) the Philippines, 6) Thailand, and 7) Vietnam. At midterm, API had provided support to only Laos in 2019. Number of technical assistance potential packages approved by USAID (API I.A) In progress. The potential technical assistance package targets two priority areas, ASW and DDG. The indicator was set to measure performance in 11 technical assistance packages that API would deliver to seven AMS. At the midterm, API had achieved results only in the Philippines for NSW in 2021. Number of action research reports finalized by API (API 1.1.1) In progress. This indicator measures API performance on 11 action research reports to be conducted in seven AMS. Action research reports are classified into two priority areas, ASW and DDG. At midterm, API has achieved four action research reports completed in 2021 for Cambodia and the Philippines. In 2022, API aims to complete five action studies in Indonesia, Laos (two reports), Myanmar, and Vietnam. Number of the kinds of certificates that are being exchanged as a result of USG support—ASW new documentation by member state added to data exchange system (API 2.B) In progress. This indicator targeted four AMS: 1) Cambodia, 2) Laos, 3) Myanmar, and 4) the Philippines. API has supported the Philippines in developing its portal, which is now functional. Cambodia and Myanmar already exchanged ACDD documents on their own, without API support. Number of approved concrete technical assistance activities completed (API 2.1.1) In progress. API targeted providing 15 technical assistance activities to conduct baseline studies in seven AMS: 1) Cambodia, 2) Indonesia, 3) Laos, 4) Myanmar, 5) the Philippines, 6) Thailand, and 7) Vietnam. At the midterm, API had provided technical assistance to only one AMS, the Philippines, in 2021, but plans to continue providing further technical assistance to other AMS in 2022. Number of technical assistance potential packages approved by USAID (API 3.A) Not achieved. This indicator is under ASEAN USAID Mission Engage (IR3). API targeted providing technical support related to this indicator in six AMS: 1) Cambodia, 2) Indonesia, 3) Laos, 4) Myanmar, 5) the Philippines, and 6) Vietnam in 2019 and two AMS in 2021, Laos and the Philippines, but no technical assistance potential packages were approved by USAID Contracting Officer Representatives (CORs), FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 45 TYPE/SOURCE OF INDICATORS INDICATORS MIDTERM INDICATOR RESULTS/STATUS Number of indicative action maps finalized by API (API 3.1.1) In progress. API has finalized five of six planned indicative action maps for Cambodia, Indonesia, Laos, the Philippines, and Vietnam. The map for Myanmar had not been finished at the time of this midterm evaluation, and it is not expected to be completed because of the prevailing political situation in Myanmar. Placeholder outcome indicators TBD objective outcome indicator: Intra-regional Trade share (API Goal 4) API has not set a target for this indicator, though the measurements have shown the value of intra-regional trade share in four targeted AMS. Those AMS scored 2.48 in total, with the details 0.44 for Myanmar, 0,32 for Cambodia, 0.25 for Lao, and 1.47 for the Philippines. Intermediate Result 1 TBD Outcome Indicator: Interest and formal agreement by AMS to accept API's assistance (API I.B) In progress. This indicator was targeted at four AMS, but API achieved performance results in the Philippines only. In FY 21, a formal agreement has been signed with the national government of the Philippines to help the exchange of ACDD per targeted timelines of ASEAN (Nov 2020); Agreed to work on the Business Processing Analysis (BPA), ACDD Operations Portal, and Routing Platform, Public Awareness Campaign; FY 22 Formal approval to work on the Technical Reference Guidelines (Oct 2020). Intermediate Result 2 TBD Outcome Indicator: Compliance with ASEAN/WTO/World Customs Organization agreements (API 2.C) Not achieved. The indicator targeted four AMS: 1) Cambodia, 2) Laos, 3) Myanmar, and 4) the Philippines. No results had been achieved at midterm. Two standard foreign assistance indicators Person-hours of USG-supported training completed in trade and investment (API 2.A; also F indicator 2.1) Achieved. The indicator targeted delivering 70 person-hours of training. API exceeded the target and achieved 87.5 person-hours of training in total. This indicator was set and achieved in 2021 in the Philippines. No results had been achieved in the other targeted AMS, Cambodia, Laos, and Myanmar until present because API has not signed a cooperation agreement with the national government of these countries to offer technical assistance. Number of WTO trade facilitation agreement provisions supported by USG assistance (API Goal 1; also IPS: O1.2) In Progress. This indicator is measured in four targeted AMS: 1) Cambodia, 2) Laos, 3) the Philippines, and 4) Myanmar. API has achieved planned results in one country, the Philippines, but is yet to achieve results in the other target countries. API performance on policy implementation support API has provided technical support to enhance ASEAN policy implementation. Under three priority areas, including ASEAN Economic Community (AEC), ASW, and ASEAN Framework on Digital Data Governance (DDG), API has devised some activities to elevate country-level policy implementation. The first year's umbrella activity focused on research and identified the concrete needs and opportunities for supporting the implementation of targeted ASEAN policies at the member-state level. For AEC, API conducts a Regional Trend Analysis, produced quarterly. The other ASEAN policies (ASW and DDG) are being adopted by the AMS, and API has supported these ASEAN policies (ASW and DDG) in several ASEAN member states: under ASW areas, API targeted four AMS to be supported, including Cambodia, Myanmar, Laos, and the Philippines; meanwhile under DDG areas, API set targets on seven AMS: 1) Cambodia, 2) Indonesia, 3) Laos, 4) Myanmar, 5) the Philippines 6) Thailand, and 7) Vietnam. AEC policy support. The ASEAN Economic Community’s (AEC) goal is to achieve economic integration in the region. It envisions making ASEAN a single market and production base, a highly 46 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION competitive region with equitable economic development, and fully integrated into the global economy. This pillar has published AEC Blueprint 2025, complemented by the 2025 Consolidated Strategic Action Plan (CSAP) to guide stakeholders on the action items that will be implemented in pursuit of the ASEAN economic integration agenda (2016 to 2025). API has conducted initial research and mapping and produced a Regional Trends Analysis Report quarterly in support of the AEC policy areas. These reports track the status of AEC implementation and are external communication tools with USAID Missions and other stakeholders. API has mapped the current status of implementation of key AEC commitments and conducted an analysis to produce the regional trends4 of ASEAN Economic Development in the ASEAN region. API’s COR helped API to disseminate the trends analysis results by sending Quarterly Regional Trends Analysis reports to USAID/ASEAN internally and USAID’s bilateral missions in the ASEAN region. The project aims to achieve in-depth research at the national level and prioritize technical assistance to AMS, adjusting their needs as necessary. The Regional Trend Analysis helped API in summarizing research on ongoing developments, including political and policy developments, relevant studies, and emerging areas of needs, priorities, and policy developments for USG, ASEAN, and its Member States. This report has a different focus each quarter and incorporates API actions to respond to the trends. The first quarter (May 2020) focused on identifying and mapping areas under AEC Development such as NSW, DDG, and COVID-19 challenges and stimulus in AMS. In Q2 (August 2020), API focused on how political developments in ASEAN during COVID-19 and its impact on ASW and DDG. It also included data on budget allocation and spending comparison for health in ASEAN. In Q3 (February 2021), the focus will be on ASEAN Recovery Framework, and digital economy and data policies in Regional Comprehensive Economic Partnership. Based on and aligned with ASEAN trends, API tried to accelerate the implementation of key ASEAN policy agreements at the national level to advance economic prosperity and rules-based governance in the Indo-Pacific region. ASW policy support. API reviewed the implementation status of the ASW live operation in AMS, particularly the implementation of the intra-ASEAN certificate of origin (ATIGA Form D), Common Message Exchange Header, the exchange of ACDD, and the exchange of e-Phytosanitary (e-Phyto) certificate5. Results from the initial mapping of AMS readiness for document exchange showed that several AMS have not completed ratifying the agreements for ASW implementation and policy adoption in their countries. However, the findings showed that the readiness of AMS to implement the ASW live operation varies across the AMS. The COVID-19 situation contributed to influencing ASEAN to maximize the ASW platform and digital technology use. However, as shown in the description of API’s country-specific progress results, countries that joined the ASW live operation recently need substantial support to build their capacity to implement the ASW regional policy. As noted earlier, among the four target countries, Cambodia, Laos, Myanmar, and the Philippines, API made significant progress in strengthening the Philippines’ NSW to connect to the ASW. A key informant noted that: 4 Each Regional Trend Analysis has different content in which mostly on economic development recent update, such as economic situation during pandemic etc. Each report usually consists of 3-4 sub section, but sub-section 1 is always about ASEAN Economic Development 5 E-Phytosanitary (e-Phyto) certificate is part of e-SPS certificate that authenticated, secure and preferably non-repudiated, electronic transmission and receipt of certification data, including the certifying statement, from a competent authority in the exporting country to a competent authority in the importing country FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 47 “The Philippines was able to successfully connect the NSW to the ASW during the ACDD testing as the purpose of what we’re doing and we’re really trying to get to the point that we interface with ASW and can successfully exchange documents” - Researcher/ Consultant in The Philippines API focused its work on assisting the Philippines BOC to elevate its NSW systems. It conducted Business Process Analysis (BPA) for the electronic exchange of the ACDD and developed technical specifications and implementation activities of the ACDD Management System. However, there is room for API to support the Philippines in increasing the country’s exchange of certificates in the ASW trade platform. DDG policy support. Under the ASEAN Framework on Digital Data Governance, API has conducted several activities. These activities include mapping of the current status, priorities, needs, and opportunities for DDG in the AMS, especially concerning the ASW policy at the national level, cross-border data flow mechanism, data protection, and digital signature. API findings show that the status of each AMS varies in the development of data privacy laws, infrastructure, and technology, especially Cambodia, Lao PDR, Myanmar, and Vietnam, which lack infrastructure and technology support. Table 13 shows the status of each country on its readiness to implement DDG. Table 13. Capacity of Each AMS to Implement DDG STATUS COUNTRY LOW INTERMEDIATE ADVANCED Brunei Darussalam √ Cambodia √ Indonesia √ Lao PDR √ Myanmar √ Malaysia √ The Philippines √ Singapore √ Thailand √ Vietnam √ The limited capacity in several AMS will slow implementation of the ASEAN Framework on DDG and DIFAP 2019–2025. Implementing the ASEAN Framework on DDG is complex and involves multiple stakeholders. The framework has four major implementation areas, including (1) data life cycle and ecosystem (2) cross-border data flows (3) digitalization and emerging technologies, and (4) legal regulatory and policy provisions; therefore, its implementation is multi-sectoral. For example, the leading authority for the ASEAN Framework on DDG is the ASEAN Digital Senior Officials’ Meeting (ADGSOM), which operates under the ASEAN Digital Ministers’ Meeting (ADGMIN). Effective data flow mechanisms are essential in instances of border closures, digital adoption and transformation have accelerated due to the COVID-19 pandemic. However, the different levels of economic development in ASEAN impact the ability of each AMS to release stimulus packages and policy measures, and as the urgency to prevent an economic downturn in the pandemic era, ASEAN has agreed to keep their market open. API has conducted in-depth research on DDG not only in the seven targeted AMS, comprised of Cambodia, Indonesia, Laos, Myanmar, Thailand, the Philippines, and Vietnam (even though Laos, Myanmar, and Thailand reports are still in the draft stage) but also in the other three AMS, which include Brunei Darussalam, Malaysia, and Singapore. As those three countries were categorized before their DDG development among other AMS, they were used by API as information resources 48 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION and best practices in DDG implementation or systems to share with the other AMS for capacity building as stock-taking policy references in DDG development. API faces several challenges in the implementation support of DDG, including budgetary limitations, technical capacity, and infrastructure limitations within the AMS; coordination with the countries; and the absence of a local DDG researcher in Myanmar. API implementation progress has, therefore, been slow at the midterm of project implementation. HOW USAID/ASEAN HAS SUPPORTED API PROJECT IMPLEMENTATION USAID/ASEAN supports API implementation in coordination, communication, and funding and provides input to and feedback on technical products. Coordination. USAID holds regular meetings with the API IP team. This includes also how information has been passed to the national researchers for check-backs and giving approval for implementing project activities. Through this scheme, API could update and upgrade the project implementation progress through feedback from USAID. Furthermore, API and USAID could continually convene regular meetings to oversee and review activities as well as Incorporate feedback loops with key stakeholders. Communication. The USAID COR contributes to bridging communication gaps between API and USAID missions in AMS. This communication support has helped API to foster smooth relations and coordination with the missions. USAID also supports API with events-related communication with stakeholders, particularly by the USAID communication officer in supporting API pre-events planning or post-event communications. For example, the USAID Communication Division working with the API Communication and Learning Officer on communication strategy through social media toolkit for a virtual environment, such as social media platform development through the USAID website for a press release or highlight event. USAID has helped API to establish strong working relations with ASEAN, ASEC, and some AMS, e.g., the USAID COR introduced API projects to the USAID missions in Cambodia and the Philippines. In terms of communication, USAID was involved in before, during, and after promotions of events or activities. However, during the transition from initial planning to project execution, there was a time gap. USAID used this time gap to bridge communication with U.S. missions, particularly while waiting for endorsement approval from ASEAN by giving backing up in the form of covering the needs or support to further API activities. Funding. The disbursement of funds for Fiscal Year (FY) 2022 was delayed due to USAID’s internal budget constraints. Delayed funding hindered the implementation of short- and long-term technical assistance activities for the AMS, most of which were postponed. USAID/ASEAN asked API to scale back activities until the funding for FY 2022 was disbursed. Furthermore, strengthening API performance by ensuring that funding is disbursed right on time is crucial to help API maintain its work plan and target consistently. IMPACT OF COVID-19 ON API IMPLEMENTATION AND COPING STRATEGIES The COVID-19 pandemic has constrained some API activities. Activities that require international travel are most affected, such as field research and site visits to identify needs and opportunities for supporting AMS with ASW and DDG priority areas. The pandemic also has limited API’s ability to travel and build relationships for the work, specifically in the initial stage. In the introduction phase of FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 49 API activities to ASEAN and AMS, most did not know the exact role of API and how it works, and it's hard for API to physically reach them with COVID-19 pandemic restriction. In anticipating the potential impact of COVID-19, API has devised some coping strategies, comprised of (1) shifting to online work to maintain their engagement with the key stakeholder, (2) hiring local nationals as consultants in some AMS to take advantage of their local networks and knowledge, (3) scaling down the scope of several activities, particularly on communication activities, and (4) continuing to have virtual meetings with representatives from ASEAN’s Secretariat and IGNITE to discuss new developments concerning DDG. API’s national consultants could initiate consultations with governments and the private sector, thus maintaining progress in project implementation. OTHER IMPLEMENTATION CHALLENGES EXPERIENCED BY API AND COPING STRATEGIES Besides COVID-19 limitations, the main challenges that have impacted API’s implementation communication and coordination issues, are project endorsement delays, internal problems within respective countries, and budget disbursement delays. Another challenge that API faced is the lack of clarity about stakeholders' involvement and contribution to API’s work, e.g., ASEAN’s Secretariat. ASEC Trade Facilitation Division is unsure of their API because they work at the regional level, while API addresses issues and provides direct support at the country level. Furthermore, API encountered challenges arising from the internal AMS problems as described previously in the country-specific findings. In summary, Cambodia has a weak legal framework and local capacity. Laos has poor ICT infrastructure, limiting its participation in the ACDD exchange. The Philippines had difficulties with procurement services, and Myanmar had undergone a military coup. AEQ 2: HOW DID ASEAN’S ENDORSEMENT PROCESS OF API IMPACT THE PACE OF PROJECT IMPLEMENTATION? WHAT INTERVENTIONS COULD OR COULD NOT BE UNDERTAKEN WHILE AWAITING THE ENDORSEMENT PROCESS? EFFECTIVENESS OF API’S THEORY MODEL? API EXPERIENCES REGARDING THE PROJECT ENDORSEMENT BY ASEAN ASEAN’s endorsement of API was critical to implementing project activities. API’s planned activities are all developed to support ASEAN’s priority needs. For this reason, endorsement by ASEAN was an important initial step for gaining acceptance of the project across ASEAN bodies and the AMS. API is targeting AMS that needs API’s assistance in implementing ASEAN’s policies and priorities based on the reports conducted by the national researchers. However, there is a striking challenge because each AMS doesn’t automatically implement ASEAN’s policies and priorities; they will implement it only if it aligns and supports their national priorities and interests. For example, the Philippines asked for technical assistance from API because ASEAN’s policies and priorities aligned with the country’s BOC’s priorities. This is why the endorsement of API by ASEAN is highly crucial for implementing project activities; ASEAN needs to introduce and promote API to the AMS by emphasizing that API can help AMS in implementing ASEAN’s policies and priorities that are in line with their national priorities and interests. Before receiving the endorsement from ASEAN, API found it difficult to communicate and introduce the project at the ASEAN level, especially with all the restriction policies related to the COVID-19 pandemic. 50 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION Endorsement procedures, the effectiveness of the process, and lessons learned The approach to obtaining API’s endorsement was multi-pronged. At the initial stages, the API team reached out directly to ASEAN regional bodies and targeted AMS to introduce the project. This approach was unsuccessful in helping API connect with ASEAN regional offices and the AMS. USAID/ASEAN then got involved and put together project introduction paperwork for obtaining the official endorsement of API by ASEAN. Concurrently, API applied a parallel track by enlisting the support of the former ASEAN Deputy Secretary-General ASEAN and the help of the ASEC Trade Facilitation Division. API explained the project to the former Deputy Secretary-General, discussed priorities, and got his assistance with connecting API to ASEAN. Based on this communication with the former Deputy Secretary-General, ASEAN invited API to a Technical Working Group (TWG) meeting to update them about API work and report their regional level progress. After this meeting, formal endorsement procedures began and ASEAN officially endorsed API in July 2020, but this endorsement did not automatically imply country-level approval of API. API still had to follow and meet specific requirements of the target countries. At the time of this midterm evaluation, API had communicated directly with four member states to establish working relationships. Specific countries that have formalized their engagement with API include the Philippines. Pending countries include Cambodia, Laos, and Myanmar. Figure 7. Procedures of API Endorsement Process API’s endorsement process drew some important lessons for the project. First, that endorsement by ASEAN does not necessarily grant API authority to work at the country level, but improves API’s ability to engage with the AMS to gain state-level authorization to implement activities. Second, the process that works in one member state may not necessarily work in another. API’s success in engaging with the Philippines was because API’s tasks were aligned with and supported the BOC priorities. In Laos, the cooperation with ASEAN is under the Ministry of Foreign Affairs; therefore, working with the foreign affairs ministry was critical for obtaining permission to implement API activities. In Cambodia, having an ASEAN endorsement before engaging with the country was more important. Benefits of API endorsement The endorsement delivered two main benefits. It increased the adoption of API support by the AMS and increased regional collaboration on ASEAN issues. AMS adoption of API. ASEAN’s endorsement helped to create trust of API by the AMS and their acceptance of the project. This was especially the case for countries such as Cambodia and Laos, which were hesitant to provide project access to information or their recent progress regarding FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 51 ASEAN issues. With ASEAN’s endorsement and country-level adoption of the project, API effectively started implementing activities in selected countries from 1 July 2021. Collaboration. API’s endorsement increased regional collaboration on ASEAN priorities. API is designed to help the AMS implement ASEAN priorities and issues, thus ASEAN must support API and vice-versa. The ASEAN Secretariat Trade Facilitation Division noted that API worked directly with the respective member states for country-level endorsement, which again increased collaboration between API and the AMS. Also, the engagement between API and the former Directorate Secretary-General of the ASEAN Economic Community Departments helped connect API with the officials from ASEAN’s Secretariat and stakeholders. Challenges related to the delay in API endorsement The main challenges from the delay in API endorsement included a time lag between the project’s inception and the actual implementation of planned activities. The delayed endorsement by ASEAN also delayed the adoption of the project in the AMS. Finally, API had to make adjustments to their scope to deliver on some work plan priorities within tight timeframes. Coupled with budget constraints, work plan adjustments represented a challenge to API’s ability to ensure the timeliness of project deliverables. AEQ 3: HOW SHOULD USAID/ASEAN ADJUST API’S THEORY MODEL (DEVELOPMENT HYPOTHESIS, THEORY OF CHANGE, RESULTS FRAMEWORK, OR LOGIC MODEL), WHY? PERSPECTIVES ABOUT API THEORY OF CHANGE API’s ToC posits that “IF API continually identifies key opportunities, refines approaches to those opportunities through research and consultation, THEN API can implement policy support interventions that can improve ASEAN priorities, such as expanded implementation of the ASW and DDG, among others”. To realize this theory of change, API visualized the hypothesis based on the logic model presented in Figure 8. Key informants and a review of documents indicate that the API’s ToC is aligned with ASEAN’s policy continuum, which entails ASW and DDG. Figure 8. Graphical Representation of API Project Activities As indicated in the stages of the logic model, API applied the Collaborating, Learning, and Adapting (CLA) approach. Aligned with CLA, API used and incorporated the Plan-Do-Check-Adapt (PDCA) model in developing concrete needs packages. API conducted CLA Pause and Reflection sessions with API Chief of Party, technical program manager, learning officer, and USAID COR at a minimum 52 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION every quarter to reflect on activity implementation and improvement areas. To provide and identify key opportunities by hearing from multiple perspectives, API involved other relevant stakeholders to join pause and reflect activities, including ASEAN stakeholders, other key AMS- or ASEAN-level stakeholders, to review various observations that might provide a fresh perspective. Relevance of API’s ToC to ASEAN and AMS needs Perspectives about API’s TOC were obtained from KIIs from ASEAN Stakeholders, including the ASEAN Secretariat on Trade Facilitation Division, The Project Management Officer of ASEAN Single Window (ASW PMO), and ASEC PCPMD and the API staff. Most of them stated that the TOC is adequate. The TOC supports data collection and analysis for guiding the implementation of ASEAN priorities under DGG and ASW. The method was appropriate for the project, by accommodating the approach to address the needs of AMS. Even though some critical factors such as communication and coordination schemes with ASEAN were not clearly drawn on the logic model, that somehow made ASEAN stakeholders are not clear how they could contribute, support, or intervene in API project activities. Effectiveness of the ToC in guiding project design and implementation In the project design, API has used its ToC as the basis for developing its results framework. API’s results framework is structured to show how the project’s intermediate results feed into the IPS program focus areas, USAID/ASEAN DCF, and the ASEAN Community Vision 2025. Consistent with the ToC, API developed key activities based on the sub-areas targeted for support—ASW and DDG—and used these to formulate key activities that the project will undertake. Subsequently, API identified two objectives that drive its phase 1 priorities: 1. Identify concrete needs and opportunities for supporting the implementation of targeted ASEAN policies at the member-state level. 2. Provide concrete technical assistance to member states to strengthen their implementation of targeted ASEAN policies. Elements of the ToC have further been used to guide the selection of API’s target sub-areas for technical support—ASW and DDG—based on which API’s work plan activities were designed. Under ASW, API’s activities include: • Mapping the status of implementation of AMS for the ASW • Mapping for prioritizing and selecting needs and opportunities for AMS for operationalization of the ASW • Developing an indicative action map for AMS to enhance the operation of NSWs to support the ASW Live Operation’s implementation • Increasing the abilities of selected AMS (Cambodia, Lao People’s Democratic Republic (PDR), Myanmar, and the Philippines) to conduct live operation of exchange of the intra￾ASEAN certificate of origin (ATIGA Form D), Common Message Exchange Header, and the ACDD. This includes increased use of the ACDD as advance information to support the Risk Management System by these selected AMS • Increasing public and business stakeholders’ awareness on the benefits of NSWs establishment and the ASW live operation in Burma, Cambodia, and Laos Under DDG, API’s planned activities include: • Mapping the current status of AMS to the ASEAN Framework of DDG FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 53 • Mapping for prioritizing and selecting needs and opportunities for AMS with the ASEAN Framework on DDG • Conducting a study on current implementation status and needs of AMS on digital data governance, especially about the ASW policy at the national level; mainly on cross-border data flow mechanism, data protection, and digital signature This evidence in applying the ToC to project design and implementation planning demonstrates the effectiveness of API’s ToC. It has effectively guided the development of the project results framework, linking those results to the high-level results and priorities of USAID and ASEAN, influencing the development of project objectives, and guiding the selection of implementation activities. Challenges in applying the Theory of Change The ToC has guided the principles of the API method to set forth project design and implementation. As a fundamental design, the result framework also helps API to stay on track in adjusting the goals and pace of targeted ASEAN policies incorporated with USAID missions. The result framework also has led to some effectiveness of project implementation which was proven by several noticeable progress in some priority areas. Also, the interviews with KIs indicates that no challenges were reported with applying the ToC on guiding project design and implementation. However, in the reality of project implementation, API IP teams have to face challenges beyond internal factors. The list below explains some challenges that hindered project implementation: • Budget constraints regarding the delay of funding disbursement. • COVID-19 pandemic delay of planned activities, such as field research and site visits to undertake mapping of ASW and DDG. • The delayed endorsement that slowed down the planned activities, specifically for technical assistance initiatives for targeted AMS. • Unclear stakeholder involvement and contribution on the regional level, in which ASEAN stakeholders are unsure of their roles since API addresses issues at the national level. • Internal circumstances—ASEAN member state issues, such as 1) Cambodia, which has inferior existing legal frameworks due to low capacity of Cambodian lawmakers, 2) Lao PDR due to limited capacity of the ICT infrastructure, 3) the Philippines, which faced difficulty in the procurement process, and 4) Myanmar, due to political instability and zone restriction because of the military coup and presidential election. Even these challenges were not directly implicated in the API Theory of Change, but those challenges have slackened implementation activities, and this deceleration is out of API’s control. However, since those challenges have been identified or recognized, the ToC could be further developed to adjust recent limitations in mitigating the same challenges in the future. 54 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION Figure 9. API Result Framework Additional indicators for measuring API results API’s MEL plan has 13 indicators that were derived from overarching goals/objectives of the USAID/ ASEAN Mission, ASEAN policies and priorities, and the API results framework. Key informants pointed out some gaps in the indicators. They particularly recommended that API include critical factors to measure the implementation effectiveness of API in three areas: • Approvals of API’s technical assistance documents and deliverables • Extent and effectiveness of API’s coordination strategies • Communication timeliness and effectiveness For determining additional indicators, other areas that the key informants pointed out include gender-related results of API’s technical support. The Philippines API research consultants stated that the initial project scope included a gender component within the ASW and DDG support areas. API should therefore consider developing appropriate indicators for assessing gender-related outcomes of their technical support. Some areas for suggestion mentioned by KII participants for developing indicators include results project interventions for women-owned businesses or other marginalized groups or the extent of participation of these groups in API initiatives to incorporate their viewpoints or develop materials to encourage their participation. However, this gender-related indicator could not be generalized in several activities such as regulatory reform related to the ASW; therefore, gender and social inclusion considerations may not be applicable in some cases. Even though gender considerations may not be applicable for some activities such as regulatory reform related to the ASW, in some cases where gender and social inclusion considerations are not applicable, API will submit a memo to its COR and request approval for a waiver to not include FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 55 gender consideration in the activities as described. API has the flexibility in adjusting the situation related to gender and social inclusion consideration by understanding better how to affect the policies meanwhile notice the feedback loops. Reviewing the development and implementation of the targeted area related to technical design and technical support might help in achieving the expected outcome AEQ4: WHAT LESSONS HAVE BEEN LEARNED ABOUT THE EFFECTIVENESS OF API PROGRAM ELEMENTS (INTERVENTIONS OR SET OF INTERVENTIONS) THAT ARE MOST EFFECTIVE IN THE API DESIGN? AND WHICH PROGRAM ELEMENTS ARE LEAST IMPACTFUL? Despite the short timeframe within which API has been implemented and the challenges experienced, API has learned some important lessons at the midterm of project implementation. MOST EFFECTIVE ASPECTS OF API DESIGN Aligning API strategies and interventions to ASEAN needs increases implementation effectiveness. Overall, API strategies and interventions have been in line with ASEAN needs. The project priorities and activities are designed to respond to USAID, ASEAN, and AMS priorities and are informed by research findings. API conducts research to define the type of technical assistance to offer AMS, such as mapping the status of implementation of the ASW, the ASEAN Framework on DDG implementation, and the AEC Blueprint and its Consolidated Strategic Action Plans (CSAP) 2025. This approach ensures that the most needed support is provided to the AMS. In the Philippines, API’s support for the BOC needs to implement ACDD and NSW approach were based on mutually identified gaps, which resulted in the successful delivery of technical assistance. Implementation flexibility is essential for API to respond to AMS needs and build relationships with AMS. API applies a flexible approach to offer technical support to AMS for ASW. For example, API provided support at the national level for document exchange to four AMS. Several AMS indicated the need for support for the whole ASW operation. Based on current API success in addressing the needs and enhancing AMS capacity under ASW priority areas, demand for API’s support for ASEAN policy implementation, especially under the ASEAN Economic Community Pillar, is emerging. Enhancing policy development capacity and results is an important area of API technical support. API supported AMS (Cambodia, Laos, Myanmar, and the Philippines) to upgrade the ASW and to allow the exchange of more electronic documents. Furthermore, API provided some support for translation of ASEAN’s certification handbook into AMS local languages, working with a strong project management team and national consultants to achieve exceptional results in the Philippines. Despite the achievement of API in supporting AMS with ASW technical support, API has not been successful with national-level policy implementation support. API wanted to strengthen the national capacity to implement regional policies, but project performance in this area was weak because the API team failed to meet ASEC PCPMD’s desk officer and attend the meeting of the sectoral body. This is due to API being seen as a new project and is at a country level and therefore not seen as an ASEAN project. Thus, API lost the opportunity to explain its role to ASEC. Furthermore, API has not conducted national-level stakeholder consultation meetings to share its research findings and engage the private and public sectors. Cambodia’s national consultant had submitted the policy master plan on DDG a month after the report was shared with USAID, but feedback or follow-up on that report has happened. There is also a need to follow up on the proposal for capacity building, as Cambodia needed to understand the DGG policy, digital economy, and digital society, but no 56 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION progress has been seen in these areas. Also, some gaps are found between the private and public sector’s knowledge on digital transformation; the capacity of government officials is limited, but the private sector is more knowledgeable in the digital economy and economy platforms. Technical support to ASEAN and AMS is effective but has not reached scale. The BOC has successfully prepared terms of reference (ToRs) for procurement of ASEAN ACDD systems of the Philippines with API’s technical support. Aside from the ToR for the ACDD system, API supported the Philippines’ BOC to create another ToR for public awareness about the system. API also helped the BOC in several aspects of document exchange in ASEAN. The BOC claimed that API has impressive project management skills. API was very organized and systematic; all meetings were recorded, and the meeting minutes were structured. Since online meetings were a success, the BOC could continue doing online meetings regularly even after the COVID-19 pandemic ended, as it was beneficial for both parties to have constant communication. As API support in the Philippines has proven its effectiveness, other AMS want to receive the same technical assistance as the Philippines. API supported Myanmar’s NSW efforts to trade the certificates they wanted. Unfortunately, the COVID-19 pandemic resulted in less communication due to travel restrictions. It also made it much harder for USAID to work with Myanmar. However, Myanmar has great potential in strengthening the trade and investment climate through the NSW. LEAST IMPACTFUL AND TO BE IMPROVED The BOC experienced a delay of about a month due to the process of finalizing the ToR, negotiation of the procurement, and the bidders were not responsive. The ToR was refreshed and reformatted in the second round to make it easier for the bidders. API helped to revise the ToR. Finally, BOC was able to select the winning bidder and expedite the implementation of the project. The Philippines was the first (10 years ago) in bringing in the ASW, but it was the last to be able to trade the certificate due to its internal issues. The Philippines’ bureaucratic lethargy hindered the country’s ability to implement its national single window and it did not have anything to do with ASEAN. However, at this point, the country seemed to be the most enthusiastic participant in the National Single Window and ASEAN Single Window. ASEAN Secretariat was not sure of its role in the initiative since its scope was at the national level. It only took note and provided coordination if needed at the regional level. API should disseminate the project and its progress to all ASEAN entities to provide a comprehensive understanding of the project. ASEC could also facilitate API in communicating with AMS, for instance, to bridge the communication between API and Myanmar, as the country was hard to reach out to due to the national conflict. The procurement process is less effective due to limited bidder participation. There are no responsive bidders in the first round. API and BOC refresh and reformat ToR to ease future procurement process. The Philippines proceeded with procurement after ToR was created. On the second round, BOC were able to award winning bidder. LESS EFFECTIVE API SUPPORT IMPLICATIONS AND LESSON LEARNED FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 57 “Due to our nature to support the AMS, ASEAN has limitations to assist us because they think it is just a bilateral arrangement, so this is just between the API and AMS and there's no ASEAN in between. That's my reflection and take away so far with regard to ASEC and us doing API’s activities.” - Program Manager, API IP Some AMS still found difficulty in joining the exchange of documents, which made them late compared with other AMS. In response, API should prioritize countries that could not advance the implementation or were slower compared with other countries. CONCLUSIONS ASEAN has received support from USAID in developing and bolstering ASEAN key policies by addressing issues on the regional and bilateral levels. Implementing ASEAN policy at the national level is considered essential. However, ASEAN has no formal authority to enforce ASEAN policy implementation by member states, which impacts achieving API results. API has: • Assumed roles and opportunities in offering technical assistance to the AMS to fill gaps in implementing ASEAN policies by offering direct support to member states through technical assistance. The midterm performance evaluation of API investigated the project’s strategy and implementation adjustments, how USAID has supported project implementation, the effectiveness of the API theory model, and challenges experienced mid-way through project implementation. • Focused mainly on providing direct technical support to member states. API has implemented several interventions at the mid-point, adjusting its strategy to accommodate the needs of ASEAN member states. The API project design closely responds to the targeted ASEAN policy. API’s performance at the member-state level is still limited to the Philippines because of the endorsement delays and budget constraints. Though the implementation of API is progressing well, the delay in its endorsement, the challenges arising from the project breadth, and the diverse needs that have been identified in several AMS have impacted the implementation pace and achievement of results. At midterm, the project has mainly achieved strong results in the Philippines. The baseline study has comprehensively guided the implementation of API initiatives, including the prioritization of technical assistance for selected countries. Midterm evaluation results show that API has contributed to achieving some key results in advancing ASEAN policy implementation. • Identified concrete needs and opportunities for supporting the implementation of targeted ASEAN policies at the member-state level through research that has been conducted across AMS. • Faced difficulties with its start-up activities due to the slow endorsement process, which impacted the provision of technical assistance to member states. The COVID-19 pandemic also made it difficult to promote and engage with the respective countries to implement the project. API’s performance has also been affected by budgetary constraints due to funding disbursement delays, forcing the project to scale back on the implementation of some activities, focusing more on short-term priorities, and pushing some planned long-term technical support activities to FY 2022. • Designed the ToC and logic model that have effectively guided the project implementation. However, since the project is implemented at the state level, ASEAN involvement in supporting the realization of the priorities of the theoretical model is unclear. Another challenge is to adapt to the external factors beyond the theory model, such as 58 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION Myanmar’s political instability and zone restriction due to the military coup and the presidential election, also the Philippines’ bureaucratic lethargy and delayed procurement process. The targeted AMS including Cambodia and Lao PDR mostly face challenges pertaining to ICT infrastructures limitations to be integrated with ASEAN Single Windows. Also, some of the critical factors such as coordination and approval should be included and portrayed in the methodology design. • Offered capacity building and selected national experts (consultants) to provide good networks and facilitate the process of implementation. Technical assistance has been provided by API so far to address the challenges in project implementation in the form of knowledge sharing sessions, ASEAN certification handbook translation, and capacity building on Digital Data Governance for national consultants or researchers by using Brunei Darussalam, Malaysia, and Singapore cases. • Maximized its networks for communication and coordination. Since the communication has shifted into virtual coordination, connecting with the existing networks in AMS should be well maintained, especially in approaching key ministries to get “sign off.” Communication with ASEAN entities also has been optimized through API IP internal connection, e.g., with the former secretary-general of AEC, API builds relationships with ASEAN through the AEC Department. However, the communication and coordination at the AMS level have been hindered due to delayed endorsement, USAID also contributed by helping to introduce API projects in the USAID mission, even though it was not implemented in all AMS due to pandemic restrictions. RECOMMENDATIONS API aims to strengthen ASEAN centrality as a leading IPS partner by supporting member states to implement ASEAN policies under the ASEAN Economic Community. This midterm evaluation examined the extent to which API achieved expected results, including the identification of factors that influenced or impacted project performance. This midterm evaluation also provides analysis and lessons learned to inform the further development and management of API activities in the next work plan. As API has achieved partial results at midterm of project implementation, many aspects could be further improved. API should: • Continually maximize their connection and coordination with AMS to get formal agreement to support the ASW and DDG needs of member states. Key stakeholder interventions could be realized by increasing public and private awareness of the three project priority areas, including ASW, DDG, and AEC. • Increase knowledge of the project among key stakeholders by increasing the dissemination of project information to key stakeholders. API is a new project targeting its implementation at the member state level. When developing technical support strategies and work plans for targeted countries, API should consider the internal political situation, capacity, and priorities. The involvement of various stakeholders, even from USAID, National Government, and private stakeholders should be maximized is critical to the success of API. Furthermore, increasing public awareness of key stakeholders in the targeted policy areas, particularly DDG, should be continued, not only in the form of technical references guidelines but other forms of technical assistance. • Strengthen the relationship with USAID missions at the AMS level. API should dedicate more effort to engage with key stakeholders in ASEAN and AMS before starting FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 59 the implementation of the project. Furthermore, USAID/ASEAN should increase its involvement in continually bridging and fostering API relationships with USAID missions and the national government at the country level, especially the member states with internal issues that can impact project implementation. • Consider, through its theory model, the critical factors in project implementation that might affect project performance, e.g., approval, communication, and coordination. API should identify a role for ASEAN in supporting its work at the country level. This midterm evaluation found that external factors such as the internal situation in supported countries, endorsement process and challenges, and COVID-19 pandemic have delayed implementation of API activities. The ToC should consider factors beyond the theory model, such as the AMS political situation. API should hold regular pause-and-reflect sessions to help the project learn and adapt its strategies to mitigate implementation challenges in developing and implementing its future action plans. • Consider and prioritize the alignment with AMS’ national policy due to the different internal priorities of supported countries. Understanding AMS national policy is important to obtain “sign-off” for a new project like API. Remembering that API implements regional policy, integrating targeted ASEAN regional policy at the regional and national levels is also essential. The bottom-up approach could be continually applied and elaborated to achieve formal agreement in providing national direct support. 60 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION IGNITE PROJECT FINDINGS IEQ1: IN WHAT WAYS IS IGNITE HELPING TO NARROW THE DEVELOPMENT GAP IN ASEAN'S EG SECTOR? WHICH TYPE OF USAID/ASEAN INTERVENTIONS UNDER TRADE FACILITATION AND DIGITAL ECONOMY GROUP ARE MOST EFFECTIVE IN MEETING ASEAN'S NEEDS (PROVIDE EXAMPLES), AND WHY? INTERVENTIONS FOR NARROWING THE DEVELOPMENT GAP IN THE ECONOMIC GROWTH SECTOR AND ITS EFFECTIVENESS IGNITE supports efforts to narrow the development gap in ASEAN by increasing trade, economic integration, and connectivity. The bulk of IGNITE's effort has supported ASEAN's trade facilitation and expansion of the digital economy. Support for trade facilitation has been a traditional area of USAID assistance for ASEAN to narrow the development gap in the region. Most of IGNITE's assistance under trade facilitation and the digital economy supports the ASEAN Trade Facilitation Framework, which is also in line with the WTO Facilitation Agreement. Those frameworks aim to reduce the cost and time to trade and improve market efficiency and consumer welfare. IGNITE’s efforts in the digital economy have been foundational and have promoted global approaches to digital economy use. IGNITE has contributed to increasing ASEAN’s understanding of the issues it faces in adopting 5G systems, helped design ASEAN’s Digital Integration Framework Action Plan, and developed an index to help the AMS better understand its readiness to embrace a digital economy. These activities have helped shape the debate on digital economy issues that encourage cross-border digital commerce, within the region and between regions and the rest of the world. Trade Facilitation Workstream IGNITE has implemented three activities under the trade facilitation workstream: supporting 1) ASW, 2) the initiative to reduce logistical bottleneck, and 3) standards harmonization. As of this midterm evaluation, IGNITE had implemented all planned activities to support ASW, including back￾end systems design, annual ASW Trader Survey, Certified Exporters (CE) database, and Common Header software in ASW gateways, and ACDD and the electronic Phyto Certificates. Other activities included conducting an Annual ASW Symposium, connecting ASW with single windows of Dialogue Partners, and providing training to improve the skills of ASEC staff and AMS officials in operating the ASW. ASW. IGNITE supported back-end systems design for developing economies in Cambodia, Lao PDR, and Myanmar to connect the ASW (ASW gateway software) and join the live operation. Traders in those countries experienced faster trade through expedited cargo clearances. IGNITE also supported the development of the Certified Exporters (CE) database to update CE’s information and authorized signatories under the AWSC scheme and facilitate intra-ASEAN trade. Qualifying companies can now self-certify the origin status for their goods to access lower tariffs. Also, IGNITE assisted in developing the ACDD and the electronic Phyto Certificates as supporting documents for the ASW. These documents help facilitate trade risk management, minimize fraud, expedite customs clearance, and facilitate safe agricultural trade. Recently, IGNITE continued supporting ASW to connect with single windows of the Dialogue Partners, including the U.S., to increase bilateral trade. During this process, IGNITE also collected FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 61 feedback and identified ways to improve the operation of the ASW. For this purpose, IGNITE conducted an annual ASW Trader Survey and ASW Symposium. Meanwhile, to upgrade and enhance the skills of ASEC staff and AMS officials to operate the ASW, IGNITE conducted training on ASW. Figure 10. IGNITE Assistance Under the Trade Facilitation Workstream Initiative to reduce logistical bottleneck. IGNITE plans to conduct a study on best practices for regional and global low-value shipments to support ASEAN's trade facilitation efforts to reduce logistical bottlenecks. This study is important for promoting the cross-border movement of goods in ASEAN and providing a roadmap for further work on ALVS. ALVS will benefit MSME, which usually trades in low-value goods, and under ALVS, are not required to pay duties if the shipment value does not exceed a specified threshold (maximum import value). From its design, this study will be conducted as a pilot project in developing economies (Indonesia, the Philippines, and Thailand) and aims to develop the threshold for ALVS. However, IGNITE postponed this study due to the COVID-19 pandemic. Standards harmonization. IGNITE carried out several workshops and training to support harmonization standards in ASEAN. IGNITE supported a workshop on Post-Market Surveillance of Medical Devices to simplify trade procedures for these products. It also helps AMS implement ASEAN guidelines on Post-Market Surveillance under the ASEAN Medical Devices Directive. Also, IGNITE supported an initial workshop for harmonizing Electric Vehicle (EV) standards in ASEAN. In this workshop, AMS discussed the challenges and strategies for developing EVs in ASEAN. This workshop was important because several AMS were involved and are invested in the EV component industry. ASEAN may potentially be a market and a production base for these products. ASEAN is a large market of traditional medicines and health supplements (TMHS); therefore, IGNITE conducted training to support harmonization standards on this product. IGNITE conducted training on claims and claims substantiation to support AMS in implementing ASEAN’s guidelines on claims policy on this product. IGNITE also supported training on ASEAN Guidelines on Stability and Shelf life for TMHS to facilitate a common understanding of these guidelines among regulators and industry in the AMS. The training helped AMS harmonize product quality assurance for setting expiry dates and handling and storage protocols. 62 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION Digital Economy Workstream Under the digital economy workstream, IGNITE implemented three main activities to help narrow the development gap in ASEAN, including support for 1) adopting regulatory reform on the digital economy, 2) developing regulations for cybersecurity and data privacy, and 3) expanding broadband access. IGNITE has facilitated meetings and workshops to support regulatory reform for the digital economy. IGNITE facilitated meetings and workshops on the ASEAN DIFAP, ASEAN Digital Integration Index, and US-ASEAN dialogue on digital trade and the digital technology energy sector. IGNITE worked with the ASEAN Coordinating Committee on Electronic Commerce (ACCEC) to support activities for DIFAP. IGNITE's consultant assessed the state of digital ecosystem integration and recommended adopting international standards to promote interoperability and identify priority areas for achieving digital integration. The DIFAP was endorsed by the ASEAN Economic Community (AEC) Council at the 18th AEC Council Meeting in October 2019. Figure 11. IGNITE Assistance under Digital Economy Workstream Adopting regulatory reform on the digital economy. In the digital economy, IGNITE's important work is to support the development of ADII, a statistical tool that quantifies key aspects of the region's digital economy. The ADII was developed in alignment with the Digital Integration Framework (DIF) and DIFAP. IGNITE support included developing the technical brief and report for ADII. ADII for each AMS was constructed from six key pillars: 1) digitally-enabled trade and logistics 2) data protection and cybersecurity, 3) digital payment and identities, 4) digital skills and talent, 5) innovation and entrepreneurship, and 6) institutional and infrastructure readiness. Using this ADII index, AMS can measure the progress of their digital economies and the region's digital integration and identify gaps where ASEAN can implement initiatives and policies to further advance the digital economy. The digital economy will enhance regional integration, connectivity, and competitiveness. The ADII report, which was produced with IGNITE’s support, provided data and analysis of priority areas of ADII in each AMS and the region to equip relevant policymakers to implement reforms. ASEAN can propel its regional digital economy forward by identifying and accelerating improvements to ASEAN's most crucial digital integration priority areas. Furthermore, the report will help provide ideas for synchronizing and coordinating AMS efforts to narrow the development gap. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 63 IGNITE completed most of its planned activities to support regulatory reform for the digital economy except one activity that was postponed—assistance for the e-payment readiness index— because ASEAN did not approve the concept note. From ASEAN’s point of view, the scope of the intervention on SME e-payment was too narrow. ASEAN wants support for a larger scope e￾payment index. IGNITE works together with the Dialogue Partner (USG), to support regulatory reform for the digital economy. IGNITE supported the US-ASEAN dialogue on digital trade workshop, which helped resolve the gridlock between ASEAN and the United State Trade Representative (USTR) after years of negotiations on ICT principles to support digital trade. In partnership with the ASEAN Centre for Energy (ACE) and the US-ASEAN Business Council, IGNITE also provided support to complete a report on the application of digital technology in ASEAN’s energy sector. This report will improve the perspectives of AMS on energy efficiency and energy security in the region. Developing regulations for cybersecurity and data privacy. To support the development of cybersecurity and data privacy regulations, IGNITE drafted the concept of Digital Data Governance Strategic Priority 4 (policies and regulations). However, during the drafting process, it found that most policies and regulations on digital data governance were related to national regulations. To avoid overlapping assistance, IGNITE transitioned this task to API. At the time of this midterm evaluation, no intervention had been undertaken by IGNITE in this priority area. Cyber security work is mainly carried out in a different ASEAN Pillar, ASEAN Political Security Community (APSC), which is outside the ASEAN Economic Community (AEC) area where IGNITE works. Also, no funding has been available for work focused on data privacy which required shifting this activity to API. Expanding broadband access. For the work on expanding broadband access, IGNITE has supported meetings and training. IGNITE supported materials and logistics for workshops and training for developing Next-Generation USO 2.0, ASEAN smart cities network, and 5G. AMS want USO 2.0 framework developed because, at the country level, governments often burden telecommunications providers to provide support (funding) for universal service plans. IGNITE supported a workshop on Approaches to ASEAN Next-Generation USO 2.0 to define how USO should be structured and the kind of services considered important in expanding broadband internet access. Following the workshop, IGNITE prepared a report and an implementation plan, titled Approach to Next-Generation USO 2.0 Programs. This report proposed a new strategy for policymakers in ASEAN to develop and roll out USO programs that recognize the significance of marrying supply￾side programs with demand-side initiatives. It focused on ecosystem and community development to promote inclusion and accessibility and facilitate connectivity. The Shared Priorities (STI, gender, and MSME) Overall, in the STI workstream and cross-cutting objectives (gender mainstreaming and MSME development), IGNITE has implemented relevant initiatives that respond to ASEAN needs. 64 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION Figure 12. The Needs of ASEAN in Shared Priorities and IGNITE's Interventions NEEDS INTERVENTIONS Science, Technology, and Innovation Workstream ASEAN’s needs in STI are laid out in the ASEAN Plan of Action on Science, Technology, and Innovation 2016-2025. IGNITE works with the Committee on Science, Technology, and Innovation (COSTI) to determine which needs are of mutual interest and importance but are not receiving support from other donors. The demands in this area are to enhance STI in the ASEAN region and increase women’s participation and contribution in Science, Technology, Engineering, and Mathematics (STEM). STI. In STI, Science Prize for Women and Webinar series on Women in STEM were held to increase women’s participation in STEM. In close coordination with ASEC, IGNITE implemented the annual Underwriters Laboratories-ASEAN-U.S. Science Prize for Women, which aimed to enhance science, technology, and innovation (STI) in ASEAN and promote a more significant role for women in supporting economic development through research activities under STI. This activity also supports a private sector engagement partnership with US-based company UL since 2014. The webinar series in Women in STEM shares learnings and further inspires women’s participation in the STEM fields, which is necessary to create a more diverse workforce and for the region to fully reach social, scientific, and economic potential. Cross-cutting Objectives In gender mainstreaming, IGNITE identified the needs based on the gender analysis conducted earlier by the project and through consultations with PEGD and AEC. The main need for gender assistance in AEC is to understand the concept of gender and how to integrate it into their work. AEC will only commit once the policy and mandate are in place. Lastly, MSME programming priorities are laid out in the ASEAN Strategic Action Plan for SME Development 2016–2025. More recently, support has been needed for ASEAN MSME to economically recover from the COVID-19 pandemic and increase engagement in e-commerce. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 65 Support for MSME. In the MSME area, IGNITE works with the U.S.-ASEAN Business Council and U.S. companies to revamp and transform the existing SME Academy online platform to equip entrepreneurs with the skills needed to finance and grow their businesses during the economic recovery from the pandemic. While not working directly with ASEAN’s Secretariat, Women Entrepreneurs Amplifying Ventures and Economies (WEAVE) also supports women entrepreneurs in growing and expanding their business and thereby contributing to their economies (in Indonesia and Vietnam). It also increased engagement in e-commerce and supported recovery from the COVID-19 recession. Support for gender. In the gender area, IGNITE supported the AWEN (ASEAN Women Entrepreneurs Network), Women CEOs (Chief Executive Executive) Summit to help facilitate economic and financial opportunities for women business owners in the region. IGNITE also supported gender mainstreaming training in AEC to ensure that ASEC staff effectively incorporate a gender perspective in their economic initiatives. It ensures that all ASEAN economic growth programming is inclusive and benefits all genders. IEQ 2: WHICH TYPE OF IGNITE INTERVENTIONS ARE MOST EFFECTIVE IN MEETING ASEAN'S NEEDS AND ACHIEVING DELIVERABLES IN AREAS OF SHARED PRIORITY, AND WHY? (PROVIDE EXAMPLES.) MOST EFFECTIVE INITIATIVES UNDER TRADE AND FACILITATION WORKSTREAM Based on the evidence gathered during this evaluation, three IGNITE initiatives under the trade facilitation workstream have demonstrated a high level of effectiveness in achieving expected results. These include support activities for the ASW, low-value shipments study, and traditional medicines and health supplements. Figure 13. The Most Effective Interventions Under Trade Facilitation Workstream Support for ASW. IGNITE’s support for the ASW involved support for back-end systems design, annual ASW Trader Survey, and Common Header software in ASW gateways, and the ACDD and the electronic Phyto Certificates. Other activities included conducting an Annual ASW Symposium, connecting ASW with single windows of Dialogue Partners, and providing training to improve the skills of ASEC staff and AMS officials to operate the ASW. Evaluation findings show that this support was effective in achieving expected results, including: • Contributed to reducing costs for private firms by an estimated USD1.9 billion 66 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION • Provided assistance to ASW that contributed to an estimated trade cost savings of USD1,307,745,000 to private firms in the second year of IGNITE implementation, which was a significant increase in savings from USD5,756,059 in the first year of project implementation • Increased the number of business messages exchanged through the ASW from 494,932 messages in the first year to 771,758 in the second year of implementation • Increased savings per transaction from USD20 in the first year to USD1,514 per transaction in the second year, after implementing e-certificates of origin. • Reduced the time to trade, saving 20 hours in processing time and 2.7 days in clearance time. Low-value shipment study. Though this study is yet to be conducted, its design was completed. Findings will identify and support efforts to address logistical bottlenecks for low-value shipments through streamlined procedures, which will benefit SMEs. The study aims to establish a roadmap for further work on ALVS, including pilots in developing economies (Indonesia, the Philippines, and Thailand). Due to funding constraints and pandemic conditions, the implementation of this intervention has been postponed. Support for standard harmonization on traditional medicines and health supplements. The support for workshops and training on traditional medicines and health supplements was effective in helping with the harmonization of standards on traditional medicine and health supplements products. The market for and business from traditional medicines and health supplements is becoming robust in ASEAN. However, AMS need guidelines to harmonize 19 critical issues, such as claims and good manufacturing practices. Support for this activity will help ASEAN harmonize standards to reduce trade costs and increase consumer welfare. Figure 14. The Most Effective Interventions Under Digital Economy Workstream MOST EFFECTIVE INITIATIVES UNDER THE DIGITAL ECONOMY WORKSTREAM Furthermore, evaluation findings show that three initiatives under the digital economy workstream were the most effective, including IGNITE’s activities to support: 1. DIFAP and ADII. At midterm, the first most successful initiative of IGNITE under digital economy support to ASEAN is its support for DIFAP and ADII. These interventions effectively support digital integration in ASEAN. DIFAP helped AMS refine and hone its efforts to support SMEs in conducting e-commerce by coordinating their efforts. The digital, FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 67 finance, health care, and workforces ministers contributed to developing a framework to ensure that their efforts are not duplicative. The DIFAP has been effective in supporting SMEs to engage in e-commerce and align on other digital economy issues. As evidence, DIFAP helped AMS to refine and hone efforts to support SMEs by coordinating their efforts. Digital ministers, finance ministers, health care ministers, and workforce ministers came together to contribute to a framework that would ensure that efforts are not duplicative and can effectively support SMEs. Meanwhile, ADII is effective in helping AMS achieve support for e-commerce by SMEs by equipping ASEAN leaders and policymakers with the data needed to drive evidence-informed policy in digital payments, data protection and cybersecurity, and digital skills. 2. ASEAN 5G guideline development. IGNITE provided support to ASEAN to prepare ASEAN 5G Technical Briefs, which aims to support the implementation of 5G, the next generation of mobile Internet connectivity in ASEAN. This support has been effective in enhancing ASEAN connectivity and supporting digital integration. Despite the progress made, this initiative was postponed by USAID/IGNITE due to budget constraints and the different perspectives of the lead country. 3. Next-Generation USO 2.0. Key informants described IGNITE’s support of the ASEAN Next-Generation USO 2.0 Workshop as highly effective. The USO 2.0 aims to make Internet services more affordable and accessible for ASEAN member states. The media has extensively exposed this framework’s potential value and impact on the private sector, business climate, and trade. After the workshop, IGNITE prepared a report and an implementation plan titled, Approach to Next-Generation USO 2.0 Programs in ASEAN. This plan proposes a new strategy for policymakers in developing and rolling out USO programs that recognize the significance of marrying supply-side programs with demand-side initiatives, including the largest demand-side issue of affordability. It focuses on ecosystem and community development to promote inclusion and accessibility and facilitate connectivity. The relevant ASEAN ministers adopted this report in October 2019. Figure 15. The Most Effective Initiatives Under STI MOST EFFECTIVE INITIATIVES UNDER STI Under STI, IGNITE supported three activities that evaluation participants considered most effective: 1. Science Prize for Women. The support IGNITE provided for the Science Prize for Women event helped to enhance human capital in STEM. This event highlighted the work of female scientists in improving society and serving as role models for other women in STEM. 68 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION Winners used their cash awards ($10,000–$15,000) to improve the research quality and skills of female scientists. Applicants for this award increased from 62 female researchers from seven countries in 2019 to 66 applicants from 10 AMS in 2020 despite the pandemic lockdowns in research facilities across the region. 2. Women in STEM Webinar. The webinar promotes women scientists’ profiles and showcases their work to encourage more women and female students to study and embark on careers in STEM. Key informants noted that this activity is effective in increasing women’s contribution and participation in STEM, particularly in pandemic conditions. From the survey conducted on women after the first webinar, 78 percent of respondents in a follow-up survey stated that the webinar had brought about a change in their attitude about the importance of appreciating women scientists. The second survey showed that 86 percent of respondents found the webinar topics relevant to their career/occupation. Under cross-cutting objectives, IGNITE supported three activities that evaluation participants considered most effective: 1. WEAVE. IGNITE supported this initiative in Indonesia and Vietnam in cooperation with The Cherie Blair Foundation. The initiative increased access to learning apps for business development among over 12,000 women-owned MSMEs. This support is considered effective in increasing SME access to finance, technology, and e-commerce. It also helped women strengthen their business skills, knowledge, confidence, and networking. A total of 264 women have completed Phase 2 of WEAVE, the Road to Growth program, an online learning course to develop business management and investment readiness skills for women entrepreneurs. A total of 87 percent of women participants have implemented improved business practices from this online learning, and 53 percent demonstrated improved financial competency. Challenging but potentially effective initiatives This midterm evaluation also found that some activities of IGNITE, though challenging, were potentially effective, including support for: 1. A Mutual Recognition Arrangement (MRA) for STI professionals. It is considered effective in increasing researchers' mobility in ASEAN. It will facilitate talks and negotiations for the cross-border movement of skilled professionals by the government and professional associations in each AMS. IGNITE has conducted an internal desk study for an MRA, which focuses on identifying labor regulations and the skilled labor definition among AMS. It will be beneficial as a basic study for an MRA. However, COSTI has postponed this activity to 2025. 2. The AGMS. This framework ensures that all ASEAN economic growth programming is inclusive and benefits all genders. ASEAN decided to take a more comprehensive approach to developing a framework to advance gender integration on three pillars. The unexpected endorsement delay of the AGMSF stalled further work on gender mainstreaming in the AEC during 2021. However, initial training in 2018-2019 still impacted beneficiaries applying a gender lens in a business sentiment study. 3. The development of electric vehicles (EVs) standards. A workshop was conducted in collaboration with the U.S. Department of Commerce (DoC) that discussed global standards for charging stations and types of batteries of electric vehicles. The proposed next support was to organize a visit to EV development facilities in the U.S. However, since USAID and DoC lacked the resources to support the visit, no follow-up activities have been conducted, and no progress realized. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 69 Figure 16. Challenging and Potentially Effective Interventions MAIN RESULTS ACHIEVED AT MIDTERM The main achievement of IGNITE is its support to help ASEAN to achieve accelerated and inclusive growth by integrating trade through the online platform ASW. In year 2, the actual trade cost savings to private firms from ASW and other systems is higher than the plan. After implementing self-certification, the average cost savings per transaction (US$) increased significantly. Before implementing the e-certificate of origin, cost savings were $20, but after the execution, the cost￾saving per transaction was $1,514. Another intervention from the self-certification initiative could provide more cost-saving, projected to save $100 million or more per year Overall, self-certification is projected to save $100 million or more per year. As stated in the ToC, reducing the time and cost of trade will benefit AMS and U.S. exporters. Cost-saving will likely lead to cheaper ASEAN-origin products and inputs available to ASEAN-US consumers, businesses, and producers. Consumers will pay the lower price. The productivity of ASEAN firms will increase. Business opportunities and employment opportunities will expand in ASEAN, benefiting ASEAN workers. Also, U.S. exporters will also gain from lower export costs and an expanded market. Figure 17. Impacts of ASW to Consumers, Firms, Exporters, and Workers 70 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION Table 14. Impacts of ASW on Trade Costs PERFORMANCE INDICATOR 1ST YEAR 2ND YEAR ACTUAL VS PLAN Estimated trade cost savings to private firms from ASW and other systems (US$) (5,756,059 VS 7,020,000) Lower (1,307,745,000 vs 1,135,650,000) Higher Number of business messages exchanged through the ASW (494,932 vs 540000) Lower (771,758 vs 915,000) Lower Average reported savings per transaction (US$) 20* 1,514 Source: IGNITE AMEL document IEQ3. HOW EFFECTIVE IS IGNITE’S THEORY MODEL IN GUIDING THE IMPLEMENTATION OF PROJECT ACTIVITIES TOWARD ACHIEVING EXPECTED RESULTS (DEVELOPMENT HYPOTHESIS, THEORY OF CHANGE, RESULTS FRAMEWORK, OR LOGIC MODEL)? WHY OR WHY NOT? PERSPECTIVES ABOUT IGNITE THEORY OF CHANGE IGNITE’s ToC holds that “IF ASEAN institutions are able to mobilize sufficient political will and capacity to reduce the time and cost of trade, particularly in key growth sectors (e.g., digital services and science, technology, and innovation), THEN ASEAN member states (AMS) will benefit from accelerated and inclusive growth benefitting AMS consumers and firms as well as U.S. exporters and workers.” Relevance to ASEAN needs Evaluation findings show that IGNITE implementation and design are guided by their ToC and are relevant to ASEAN needs. Since project inception, IGNITE has assisted ASEAN in implementing the AEC Blueprint 2025, which lays out programs for inclusive growth for the region. The three workstreams of IGNITE were designed to provide technical assistance and policy advice that promotes a well-integrated and inclusive AEC by strengthening the enabling environment for cross￾border trade. The workstreams aimed to reduce trade costs, spur e-commerce, and increase productivity to deepen ASEAN economic integration and foster more inclusive economic growth. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 71 Figure 18. IGNITE Results Framework Source: AMEL IGNITE document Effectiveness in guiding project implementation IGNITE’s ToC effectively guides project implementation and directs the progress and performance of each workstream. IGNITE has a Statement of Objectives that makes it easier to assess progress and performance. IGNITE has pursued three strategies to mobilize sufficient political will for enhancing ASEAN institutional capacity: 1. Strong engagement strategy. Identifying champions to work with and guide ASEAN to support ASEAN’s working committee. IGNITE worked with individual ASEAN committees and their chairs to help mobilize support for specific policies or initiatives. IGNITE would sometimes identify champions to work with (usually country proponents) and guide them to help build support among the working committee representatives. For example, in the case of ADII, IGNITE worked closely with Ms. Hang of Vietnam (the country proponent) to support her to drum up support to advance the ADII and eventually have it endorsed by ASEAN Economic Ministers. 2. Consulting and coordinating with relevant ASEC division for planning, executing, and monitoring technical assistance or deliverables based on IGNITE Annual Workplan. IGNITE often conducts studies, develops frameworks, or creates tools on various policy topics that can help ASEAN advance policy in certain areas. One example is the ASEAN Digital Integration Index Report. It uses data and analysis to identify and demonstrate gaps in certain digital economy areas. The fact that the Index will be reproduced several times with 72 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION updated data provides an incentive for individual AMS to implement digital economy priorities of ASEAN to increase their scores in the Index. IGNITE also has conducted capacity-building activities to reduce time and cost to trade by facilitating training and workshops. IGNITE developed training and guidelines for ASW for ASEC staff. Following the training, participants provided examples of ASW daily reports. IGNITE also conducted a capacity development program for ASEC to automate self-certification to support ASW. Another capacity-building strategy by IGNITE is to facilitate or support efforts led by AMS and committee chairs rather than offering direct support. Limitations of IGNITE’s ToC Despite the relevance and effectiveness of IGNITE’s ToC, it also has two key limitations: 1. The ToC does not identify anticipated conditions or assumptions that can potentially impact implementation success. Aspects such as the potential limitations arising from ASEAN policies, laws, regulations, standards, and agreements that need wide consultations and endorsement by ASEAN, AMS, and their stakeholders have had a demonstrated impact on implementation progress. For example, although the consultancy support on 5G provided by IGNITE's network is considered important by ASEC, it cannot be implemented because of complicated internal processes. Another example is in the case of support for the AGMSF. Until now, IGNITE cannot implement any relevant support or initiative because AEC and sectoral bodies need to wait for the approval of the ASEAN Commission on the ACWC for the AGMSF. 2. AMS have a gap in capacities and capabilities. Realizing the expected results for meeting ToC expectations also depends on capacities within supported AMS. For example, AMS have different capacities to implement the e-phytosanitary certificate. Less-developed AMS (Cambodia, Laos. and Myanmar) faced difficulties in following several technical procedures in implementing this e-certificate because of a lack of resources or skills in those countries. Lessons learned/suggestions for improving IGNITE ToC From design to implementation, three key lessons were learned to improve the effectiveness of IGNITE initiatives: 1. The project needs to conduct a more comprehensive analysis of the appropriate activities under each workstream. For example, IGNITE should integrate financial activities into the digital economy workstream. 2. Budget certainty and sufficiency are important assumptions in supporting the realization of IGNITE’s ToC, but this assumption is currently not included in IGNITE’s ToC description. The fact that the STI workstream is considered activities and results are considered lower level does not mean that the demand and impact of this workstream are low. However, STI receives the lowest budget allocation because of budget limitations and no budget to recruit a manager for the STI workstream. 3. To avoid duplicative technical support between IGNITE and API, IGNITE needs to focus on ASEAN (regional) needs, not national-level needs. For example, after IGNITE submitted the draft concept note to the TELSOM Working Group on Digital Data Governance, the feedback received was to support Personal Data Protection under the ASEAN Framework on Digital Data Governance under Strategic Priority 4 (policies and regulations). Framework on personal data protection with national policies relevant to API. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 73 ADDITIONAL INDICATORS FOR MEASURING IGNITE RESULTS IGNITE should add two new indicators to measure its performance adequately, IGNITE should add two new indicators to measure the status of the: 1. ASEAN e-Payment Readiness Index on the policy continuum for the digital economy workstream. This indicator will assess the readiness of AMS to integrate e-payments (indicator for digital economy, workstream), which is essential for tracking the effectiveness of the system over time. 2. ASEAN Trade in Goods Agreement (ATIGA) e-SPS (Sanitary and Phytosanitary) Certification on the Policy Continuum for the trade facilitation workstream. This indicator will assess the readiness of AMS to implement e-SPS measures for further support of ASW (indicator for trade facilitation workstream) IEQ 4. HOW HAS IGNITE CONTRIBUTED TO ACHIEVING INCLUSIVE GROWTH PRIORITIES (PROVIDE EXAMPLES)? WHAT ARE THE CHALLENGES AND POTENTIAL SUPPORT PROVIDED BY USAID TO IMPROVE THE RESULTS? IGNITE SUPPORT FOR INCLUSIVE GROWTH USAID/ASEAN has several mechanisms that guide and enable IGNITE to support the inclusive growth priorities of ASEAN, including 1) USAID-ASEAN DCF, 2) Support for a Prosperous Association of Southeast Asian Nations, 3) Regional Development Cooperation Agreement and Plan of Action to Implement the ASEAN-U.S. Strategic Partnership (2021–2025), and 4) Trade and Investment Framework Agreement between the U.S. and ASEAN. As stated in ASEAN Economic Community (AEC) Blueprint 2025, IGNITE supports ASEAN's priorities in innovation, trade, and e-commerce. Interventions on ASW, digital economy, prizes for women, and WEAVE have supported inclusive growth in ASEAN. The launch of the ASW in all 10 AMS, a major accomplishment recognized by ASEAN, reduced cost and time to trade and increased consumer welfare in ASEAN. IGNITE programs are regularly cited in various ministerial and leaders’ documents as contributing to ASEAN's progress. Support for the ASEAN Framework for USO 2.0 guides policymakers in aligning USO policies to focus on supply-side connectivity and demand-side solutions to increase inclusive growth. The ASEAN Framework for the Next-Generation USO 2.0, developed by IGNITE, was highlighted in the AEC Blueprint 2025 Midterm. Support for the ASEAN Digital Integration Index helps AMS measure its readiness to integrate its digital economy. Digital integration will provide business opportunities for less-developed AMS. Support for prizes for women increases women's participation in STEM. Support for WEAVE increases the participation and capacity of women to engage in business. USAID/ASEAN SUPPORT FOR IGNITE IMPLEMENTATION USAID supports IGNITE in promoting inclusive growth in multiple ways, including promoting or amplifying project efforts on social media platforms. Support includes: 1. Providing reviews and feedback on all promotional and outreach materials, social media-related content, and other special requests. 74 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION 2. Getting USAID clearance and approvals on all promotional/outreach-related materials and promoting social media content on some USG social media platforms. 3. Coordinating with other USG entities, including the U.S. Mission to ASEAN. This support is effective in showing the credibility of project events and activities. For instance, USAID/ASEAN's communications work increased the number of applications to the annual Science Prize from women scientists. While IGNITE support for ICT issues was challenging because the U.S. did not regularly meet the ASEAN TELSOM group, USAID and the ASEAN project at that time worked closely with the US-ASEAN Business Council and the USG to generate an official engagement that has facilitated IGNITE's work program 4. Funding IGNITE’s activities. USAID helps IGNITE to secure additional funding by submitting numerous funding proposals on a wide range of topics each year to enable the project to implement more activities. USAID/ ASEAN's efforts to secure funding have been successful at times, such as securing funding for the WEAVE grant. USAID/ASEAN also provided support in diplomatic and administrative engagement for IGNITE. USAID/ASEAN provides a range of support, from delivering opening or closing remarks at an IGNITE-led event to the substantive review of draft deliverables to amplifying project efforts through USAID's and the U.S. Mission to ASEAN's social media platforms. Despite this support, IGNITE has two additional needs for support from the U.S. Government : 1. Increased and consistent budget. Considering the breadth of IGNITE’s activities and the needs of ASEAN, key informants suggested that USAID consider increasing the level of funding for project activities. Funding amount and consistency is the biggest challenge experienced by IGNITE compared to other Dialogue Partners. ASEAN Committee of the Whole recommended AIMD to Consolidated Strategy on 4IR (Fourth Industrial Revolution), and AIMD approached IGNITE for support. However, IGNITE could not provide technical support due to funding limitations, and another dialogue partner stepped up to offer the support. With more consistent funding, IGNITE could be more responsive to requests from ASEAN. Funding certainty affects the result of assistance and is important for multi-year activities. Due to the limited budget, the STI manager had to be dismissed, and only a few activities are conducted. The effects of multi-year activities are higher than short-term activities. e.g., ASW (more than a decade but reduced cost significantly). 2. Prioritizing support for high-impact projects. Participants in this evaluation also suggested that USAID should place more emphasis on supporting high-impact projects by financing long-term projects. For example, assistance for ASW needs more than a decade to get significant results. USAID needs to have commitment and support for this kind of project. IMPLEMENTATION CHALLENGES IGNITE faces several challenges in its implementation. The COVID-19 pandemic has impacted IGNITE’s performance positively and negatively. The positive impact of the pandemic is that IGNITE can implement more events with less funding. It has saved travel costs, such as accommodation, flights costs, event venue hire, etc. The negative effect is that virtual meetings are not as effective in resolving issues as in-person meetings. IGNITE has applied three strategies to cope with the COVID-19 situation: FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 75 1. Switching activities to the virtual environment. IGNITE has subscribed to online platforms such as Zoom and made an effort to learn to use new interactive tools such as Mentimeter to ensure the robust engagement of participants. ‘ 2. Successfully conducting online coordination meetings with partners and stakeholders (emails, calls, virtual meetings). 3. Shifting the WEAVE project to be an online course. The Road 2 Growth training (Phase 2) part of WEAVE was initially envisioned to be delivered in person but was delivered online. Given the online format, reaching high-participant retention rates was not easy, but the trainers adapted by regularly checking in with participants to motivate them and encourage them to complete the course. Besides COVID-19, IGNITE also faced other challenges in its implementation and applied several coping strategies to resolve and deal with this condition: 1. Unclear mandate within ASEAN. IGNITE lacks a clear mandate within ASEAN for the initiatives that the project seeks to support. For example, with the gender mainstreaming activities, approaching the AEC with proposed interventions without any clear mandate on this was challenging. IGNITE’s approach to resolving this challenge focused on supporting PROSPECT in developing the ASEAN Gender Mainstreaming Strategic Framework draft as a basis for ASEAN to come up with advanced gender initiatives. IGNITE will then use the framework to implement gender mainstreaming activities in the AEC in subsequent years. 2. Inconsistent and low funding level. IGNITE’s activities are robust, but they are affected by USAID's inconsistent and low funding disbursement. The project has coped with this limitation by maintaining its support for ASEC’s priority activities through high-quality technical assistance from its staff. IGNITE has also leveraged the COVID-19 conditions to hold more virtual events that are little to no cost to continue the implementation with limited funds. IGNITE also harnessed technical experts across workstreams to help provide ad-hoc support to deliverables with no budget but to ensure high quality. 3. Annual changes in the chairmanship of ASEAN. The chairmanship of ASEAN changes every year. These frequent leadership changes are a challenge because IGNITE needs more effort to communicate and maintain the commitment to support the IGNITE annual plan activity. IGNITE has coped with this challenge by maintaining close coordination and cooperation with ASEC to ensure some significant deliverables of the Chairman are supported by the USG through IGNITE. 4. Process-level challenges. The challenge related to strict and lengthy processes for vetting the participation of participants from Myanmar after the coup has limited the performance of IGNITE in Myanmar. The IGNITE strategy for this challenge is to set earlier participant registration deadlines to ensure adequate time for the vetting process. IMPLEMENTATION LESSONS Several lessons have been learned in designing and implementing IGNITE at the midterm evaluation: 1. Regional progress on economic issues is critical. Sometimes individual ASEAN member states require assistance to catch up to other countries in the region and move forward in an integrated and coordinated manner. IGNITE has created opportunities to solve development issues at the bilateral level, such as increasing Internet access through the USO 2.0 work and facilitating trade through individual countries’ ASW Gateways. These opportunities have helped target country-level technical assistance, which the API project can implement. 76 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION 2. U.S. Government should continuously support ASEAN on a particular issue or initiative and ensure that the funding is available. The fragmented support because of the funding constraints can inhibit momentum and progress on ASEAN initiatives. 3. Gender mainstreaming efforts that cover all of ASEAN’s economic community are effective to a certain extent. ASEC needs to understand the impact of gender mainstreaming to incentivize them to incorporate gender in their programming. IGNITE plans to implement a tailored and targeted gender pilot project on one particular sector in year 4. This project can then be used as a model for other sectors. 4. Strategic communications efforts, including messaging, are essential. Messaging sets the tone for the project’s future success. Communicating such technical and high-level, and/or incremental, consensus-driven policy changes across ten member countries is difficult, but opportunities arise in lieu of done correctly by intense and informative communication about the progress and benefit of each assistance. success. It can be impactful in helping stakeholders understand the benefits of IGNITE’s work and stimulate interest in continuing the work. 5. The design of IGNITE was well thought out and proved to be highly strategic. This was particularly true with the three workstreams of 1) trade facilitation, 2) digital economy, and 3) STI. However, IGNITE also needs to allow the U.S. to weigh in on substantive policy issues in ASEAN, such as 5G. To support AMS in their transition to 5G, ASEAN agreed to develop best practice guidelines for 5G in the region. 5G Briefs helped to equip AMS ahead of the development of the regional guidelines and facilitate the startup of Pro ICT. CONCLUSIONS From the findings of the midterm evaluation of IGNITE’s performance, the evaluation team arrived at the following conclusions: 1. IGNITE assistance showed good progress and contribution in narrowing the development gap in ASEAN. The assistance addressed the needs of ASEAN effectively in trade facilitation, digital economy, STI, and cross-cutting objectives (gender mainstreaming, and MSME development). For the trade facilitation and digital economy workstreams, most of the planned activities have been implemented, with the exception of support for ALVS, the e-payment index. Only a few activities under shared priority have been implemented, due to funding limitations. However, ASEAN's demand for support in this area is high. Therefore, considerations for increasing funding for shared priority needs are crucial. 2. Interventions under trade facilitation showed substantial progress and significantly reduced costs and time to trade. The most effective interventions under the trade facilitation workstream are the support for ASW and support for workshops and training on traditional medicines and health supplements. ASEAN’s low-value shipment study is expected to be an effective support to reduce costs for SMEs. Meanwhile, the most effective interventions under the digital economy workstream are the DIFAP, ADII, ASEAN 5G Development guideline, and support for the ASEAN Next-Generation USO 2.0 Workshop. 3. Interventions that were most effective in the STI workstream, include support for 1) the Science Prize for Women, and 2) Women in STEM Webinar and WEAVE. 4. This evaluation determined that three activities are challenging but potentially effective. They include support for 1) an MRA for STI professionals, 2) a Gender Mainstreaming Strategic Framework, and 3) Standards development on EVs. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 77 5. IGNITE’s ToC is relevant and effective in guiding the project design implementation. However, the ToC should include theoretical assumptions that have or could impact achieving results. Examples include the policy conditions of the AMS, ASEAN’s inability to enforce policy implementation in member countries, and the dialogue that is needed to influence the implementation of ASEAN policies, laws, regulations, standards, and agreements. 6. Two potential gaps have been found in this evaluation. They include the gap in the results framework regarding activities on financial integration and the gap between the volume of planned activities and funding level. The project and USAID need to review the project funding level versus implementation needs and priorities. 7. IGNITE has done a tremendous job in improving ASEAN institutional capacity by facilitating training workshops. Specifically, IGNITE has excelled in training and workshops related to trade facilitation and the digital economy involving AMS and ASEC. 8. The online platform ASW has accelerated progress in integrating trade. This is IGNITE’s main accomplishment in helping ASEAN to achieve accelerated and inclusive growth. 9. The current indicators of IGNITE do not adequately measure IGNITE’s performance. Specific gaps exist in the measurement of project performance in the trade facilitation and digital economy workstreams. Two new indicators need to be included to measure performance results in these areas. 10. IGNITE interventions on ASW, digital economy, prizes for women scientists, and WEAVE have contributed to inclusive growth and narrowing of the development gap. 11. The COVID-19 pandemic has had positive and negative impacts on IGNITE results. By conducting virtual events, IGNITE can implement more events with less funding. However, virtual meetings are not as effective in resolving issues as in-person meetings. 12. U.S. Government could support IGNITE by providing adequate funding and supporting high-impact projects. RECOMMENDATIONS USAID/IGNITE should: 1. Continue to support the ASW workshop and training to harmonize standards, DIFAP, and ADII. This intervention showed significant results under trade facilitation and digital economy workstream. Support for ASEAN low-value shipment, USO, 5G, and ASCN should also continue. Additional support needed for ASCN is support for baseline study or survey of existing infrastructure, regulations, human resources in ASCN. From those studies, AMS could provide better guidance to build their capacities and be valuable for future ASCN evaluations. Activities in developing cybersecurity and data privacy need more comprehensive analysis. 2. Conduct analysis to determine high-impact shared-priority activities, including STIs, gender, and MSME, and provide more funding for them. Continue to support the Science Prize for Women, Women in STEM Webinar this intervention showed significant results. 3. Continue to support ASW, ALVS, workshop or training to harmonize standards on traditional medicines and health supplements, DIFAP and ADII, ASEAN 5G Development guideline, and the ASEAN Next-Generation USO 2.0 Workshop. This intervention showed significant results under the trade facilitation and digital economy 78 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION workstreams. To update and improve the framework, IGNITE needs to update recent and significant digital economics advancements, which are not included in the framework. 4. Continue to support the Science Prize for Women, Women in STEM Webinar, in the STI workstream. Implementation of these activities has demonstrated effectiveness in increasing human capital and women’s participation in STEM. USAID should provide more funding and work with IGNITE in conducting analyses to determine the high-impact activities under this area. 5. Adjust the results framework regarding the activities to support alternative finance. This should be included in the digital economy workstream. The U.S. Government could support IGNITE by providing adequate funding and prioritizing high￾impact interventions. For ASEAN, multi-year assistance provides significant results. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 79 OVERALL CONCLUSIONS CONCLUSIONS GEQ1: HOW EFFECTIVE ARE THE USAID/ASEAN ACTIVITIES IN ACHIEVING THEIR EXPECTED RESULTS? USG has committed substantial resources to support ASEAN over the years. USAID’s financial commitment to the three projects (PROSPECT, API, and IGNITE) evaluated in this joint midterm evaluation is approximately $31 million. The three projects support ASEAN’s multiple priorities by: 1. Increasing capacity to resolve regional and global challenges, and uphold human dignity and the RoL 2. Strengthening the enabling environment for cross-border trade 3. Accelerating the implementation of key ASEAN initiatives that support USG’s and ASEAN’s goals The midterm evaluation finds that these projects have collectively made steady progress in implementing initiatives and achieving expected results. PROSPECT has undertaken multiple initiatives to increase ASEAN’s capacity to address non-traditional security threats in the areas of trafficking in persons, countering violent extremism, wildlife trafficking, and disaster management (natural disaster and health). Based on its planned activities, the project has achieved partial progress in implementing activities for promoting responsive and transparent government. These activities include strengthening ASEAN’s capacity to ensure good governance and the RoL and enhancing civil society organizations’ networks. PROSPECT has also achieved partial progress at the midterm in implementing activities for promoting human rights and expanding opportunities for women and other vulnerable groups. The planned activities include human rights, gender mainstreaming, WPS, and youth. PROSPECT needs to reduce logistical bottlenecks and make adaptations to maximize implementation and results in the areas of partial progress at midterm. Adaptations could include re￾prioritizing and scaling down activities to focus on ASEAN’s most-needed support, which includes TIP, CVE, and Gender Mainstreaming. IGNITE has achieved substantial progress in implementing activities under the trade facilitation workstream. Specific areas of significant progress include work under the ASW, harmonization of standards, regulatory reform for the digital economy, and expansion of broadband access. However, IGNITE has not made little progress at midterm on developing regulations for cybersecurity and data privacy. Based on the activities that IGNITE has undertaken, the most effective contributions come from its support to ASW and harmonizing standards for traditional medicine and health supplements. Other areas where IGNITE has thrived at midterm include its support for DIFAP and ADII, ASEAN Next￾Generation USO framework, Science Prize for Women, Women in STEM Webinar, and Women Entrepreneurs Amplifying Ventures and Economies. Overall, IGNITE initiatives have contributed to significantly reducing trade costs and time to trade at midterm. Despite IGNITE’s significant progress and achievements in the midterm, performance on activities for supporting an MRA for STI professionals, Gender Mainstreaming Strategic Framework, and 5G development guidelines need to be enhanced. 80 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION API performance at midterm is generally mixed, mainly because of the impact of the delayed endorsement of the project, which was exacerbated by the COVID-19 pandemic. The main achievement at midterm was completing desk research, mapping, prioritization, and selection of needs and opportunities for AMS. Also, API has provided technical assistance to the AMS in knowledge sharing, the development of ASEAN’s certification handbook, and DDG capacity building. Despite these achievements, the project has achieved partial progress in identifying ASEAN’s critical needs. Out of the 10 countries that API targeted to assess, seven countries had been completed at midterm. Assessments in the other three countries were still in draft at the time of this evaluation. API had planned to offer technical support for ASW in four countries: 1) Cambodia, 2) Laos, 3) Myanmar, and 4) the Philippines. However, notable engagement and results have been realized only in the Philippines for policy work and ICT systems development. API technical support DDG was still in the initial stages at the time of this evaluation. Furthermore, API has not yet undertaken technical assistance for DDG in the planned seven countries: 1) Cambodia, 2) Indonesia, 3) Lao, 4) Myanmar, 5) the Philippines, 6) Thailand, and 7) Vietnam. GEQ2: WHAT ARE THE MAIN CHALLENGES THAT AFFECT PROJECT IMPLEMENTATION, AND HOW CAN THOSE CHALLENGES BE MITIGATED? Common challenges affecting all projects include budgetary limitations, negative impacts of COVID￾19, the extensive priorities and needs of ASEAN and AMS, the slow approval process for deliverables, and ASEAN’s lack of mandate to enforce implementation of regional policies at the AMS level. The challenges that have been more pronounced for API compared with the other two projects include internal problems within AMS, e.g., political instability and poor local capacity, the lengthy and slow endorsement of the project, and delayed funding disbursement. PROSPECT has experienced challenges in implementing its broad scope within its available resources. The projects have adapted their implementation strategies to mitigate the limitations experienced. Adaptations include reducing and reprioritizing the scope to fit activities to the budget budgetary constraints. Transitioning to the use of online applications for activities such as workshops and training has helped the projects to maintain implementation progress in the face of COVID-19. The use of online modalities has also helped mitigate project budget constraints. API has responded to the challenges from internal problems of the AMS by strengthening its networking at the AMS level, hiring local nationals within each supported country, and seeking the support of USAID/ASEAN and USAID missions within the AMS. GEQ3: WHAT WAS LEARNED (FROM INFORMATION OR DATA) THROUGH PROJECT IMPLEMENTATION THAT CAN BE USED TO REFINE OR CHANGE OUR ACTIVITIES’ THEORY MODELS (DEVELOPMENT HYPOTHESES, THEORY OF CHANGES, RESULTS FRAMEWORKS, OR LOGIC MODELS)? IN WHAT WAYS SHOULD USG ASSISTANCE TO ASEAN BE MORE TARGETED OR FOCUSED? The main lessons learned from implementing USAID/ASEAN projects include the following: • Aligning the ToC with ASEAN policy framework, needs, and priorities increases the acceptability of USAID/ASEAN initiatives and implementation success • Critical assumptions must consider the potential implications of internal issues and natural and health disasters on project success FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 81 • Timely investments in learning products (studies, assessments, research, dissemination) and promoting utilization are critical for assessing the effectiveness of the ToC in guiding implementation, examining progress toward expected results, and making timely adjustments to implementation strategies GEQ4: WHAT LIMITATIONS TO USAID ASSISTANCE HAVE BEEN OBSERVED, AND HOW SHOULD USAID/ASEAN AND/OR USG PLAYERS ADDRESS THEM? Findings from this midterm evaluation established two main limitations to USAID’s assistance to ASEAN. These include: 1. Insufficient budget relative to ASEAN’s needs and implementation challenges. As described in multiple sections of this report, ASEAN has many priorities and needs, but its resources and technical capacity are still weak. The insufficient budget is further impacted by delayed disbursement of funds from USAID, while the weak technical capacity is impacting progress in policy development and adoption, and implementation within the AMS. To achieve greater results, USAID needs to increase budget allocation to projects, improve the timeliness of budget disbursement and work with the implementing partners and ASEAN to refine priority ASEAN needs that USAID should support. USAID should also consider using cocreation approaches to develop follow-on initiatives 2. Inadequate introductory support from USAID. Key informants noted that the support received from USAID/ASEAN and USAID missions in the AMS is inadequate to help implementing partners build strong relations with the assistance beneficiaries. Working successfully with ASEAN and the AMS requires significant investment in networking and relationship building, particularly during the foundational phase at the project start-up. The deeper involvement of USAID/ASEAN and USAID missions and embassies in the AMS to introduce IPs and the project scopes is extremely important to the successful launch of the projects and implementation of activities. Key informants also recommended more involvement of USAID in the implementation of activities, e.g., joining some of the implementation activities. 82 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION ANNEXES All evaluation reports must include the following as annexes: 1) a timeline showing dates of data collection, baseline, and subsequent data collection, if applicable, 2) the Evaluation Statement of Work, 3) a detailed description of the evaluation design and methods, 4) copies of the actual data collection and analysis tools, 5) a list of information sources (properly identified and listed), 6) disclosure of any conflicts of interest by evaluation team members, 7) any Statement of Differences, if applicable, submitted per Automated Directives System 201.3.6.9 regarding significant unresolved differences of opinion by USAID, other funders, implementing partners, and evaluation team members, 8) summary information about evaluation team members, including qualifications, experience, and role on the team. Additional annexes may be included as needed. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 83 ANNEX I: TIMELINE DELIVERABLE DUE DATE Draft evaluation design and work plan, including detailed research methodology, data collection instruments, sampling plan, and implementation plan and schedule. Oct. 18, 2021 Final evaluation design and work plan Nov. 19, 2021 Complete data collection Nov. 30, 2021 Data analysis Nov. 31 – Dec. 15, 2021 Presentation(s) to USAID of key findings and any preliminary conclusions and recommendations. Dec. 17, 2021 Draft evaluation report Jan. 10, 2022 Final evaluation report Feb. 14, 2022 84 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION ANNEX II: EVALUATION STATEMENT OF WORK INTRODUCTION The Association of Southeast Asian Nations (ASEAN) is Southeast Asia’s primary multilateral institution for regional cooperation. Founded on the vision of a rules-based order, ASEAN provides a platform for economic cooperation, the peaceful arbitration of disputes, and the promotion of the rule of law to ensure a free and open Southeast Asia. The USAID/ASEAN Office, based in Jakarta, builds the capacity of ASEAN and its member states (AMS) to contribute to sustainable and inclusive growth and encourage rules-based systems to support a prosperous Southeast Asia through the three activities which are: (i) ASEAN-USAID Partnership for Regional Optimization within the Political-Security and Socio-Cultural Communities (PROSPECT); (ii) ASEAN-USAID Inclusive Growth in ASEAN Through Innovation, Trade, and E￾Commerce (IGNITE), and ASEAN Policy Implementation (API). USAID/ASEAN seeks to undertake an independent midterm evaluation to collect valuable insight on project design, management, and implementation of these three activities in order to adaptively manage its existing portfolio and to inform the design of its future programming. DESCRIPTION OF ACTIVITIES TO BE EVALUATED The PROSPECT is a $11,329,555 activity that started in August 2018 and is scheduled to in August 2023. The purpose of this activity is to provide support for ASEAN’s objective of building a more inclusive, rules-based, and prosperous Southeast Asia by engaging the services of the Contractor to advance the US Government to increase ASEAN’s capacity to resolve regional and global challenges and advance an effective, rules-based architecture for cooperation among governments in the region to uphold human dignity and the rule of law. The activity provides technical assistance, training, workshops, and other resources to strengthen the capacity of targeted ASEAN institutions, including the ASEAN Secretariat (ASEC) as well as other ASEAN institutions and sectoral bodies, to enable these bodies, and therefore ASEAN as a whole, develop effective multilateral policy standards and agreements, and strengthen its ability to implement these policies. PROSPECT focuses on three primary components: 1) building ASEAN and AMS capacity to resolve regional and global challenges, 2) supporting ASEAN and AMS to advance effective regional integration to enhance the rule of law and uphold human rights, and 3) human rights and opportunities for marginalized and/or vulnerable groups. The implementing partner (IP) is DAI. The IGNITE is a $10,496,880 activity that started in October 2018 and is scheduled to end in September 2023. IGNITE was designed to promote a well-integrated and inclusive ASEAN Economic Community AEC by strengthening the enabling environment for cross-border trade to benefit consumers and workers in ASEAN and the United States. The activity’s three focus areas of Trade Facilitation; the Digital Economy; and Science, Technology, and Innovation aim to reduce trade costs, spur ecommerce, and increase productivity to deepen ASEAN economic integration and foster more inclusive economic growth. The IP is Nathan Associates. The API is a $9,000,000 that started in November 2019 and is scheduled to end in November 2023. The activity aims to strengthen ASEAN centrality as a leading Indo-Pacific Strategy (IPS) partner FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 85 through supporting Member State implementation of ASEAN policies under the ASEAN Economic Community. The activity supports the national implementation of host country commitments to ASEAN policies related to addressing transnational challenges such as human and wildlife trafficking and increasing regional capacity to respond to disasters. This activity specifically targets priority policies that contribute to IPS objectives. The IP is International Business Initiative (IBI). USAID/ASEAN DEVELOPMENT COOPERATION FRAMEWORK AND DEVELOPMENT OBJECTIVES The USAID/ASEAN DCF guides its ASEAN-specific programming, supports the US – ASEAN Strategic Partnership and the ASEAN Integrated Country Strategy 2018–21 to achieve a shared vision of ASEAN as an increasingly effective body for addressing regional and global issues. The DCF outlines the context, challenges, and opportunities faced by ASEAN and AMS and describes USAID’s development contributions to ASEAN-driven initiatives in areas of common interest. USAID’s technical assistance to ASEAN and its cooperation with the USAID bilateral missions focuses on maintaining the centrality of ASEAN by building its institutional capacity and closing the development gap between member countries. Specifically, the DCF goal is for ASEAN and its member states to contribute to sustainable and inclusive growth and encourage rules-based systems to support a peaceful, secure, and prosperous Southeast Asia. To achieve this goal, USAID focuses on the following on three Development Objectives (DOs): ● DO1: Regional and Global Challenges Faced by ASEAN and Its Member States Effectively Addressed. ● DO2: ASEAN Capacity to Further Regional Economic Integration and Sustainable Economic Growth Increased. ● DO3: Rules-Based Architecture to Uphold Human Dignity and the Rule of Law Advanced by ASEAN and its member states. 86 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION RESULTS FRAMEWORK INFORMATION SOURCES USAID recommends the following materials for the desk review of this midterm evaluation: • PROSPECT’s monitoring, evaluation, and learning (MEL) plan • IGNITE MEL plan • API MEL plan • PROSPECT Quarterly and Annual Progress Reports • IGNITE Quarterly and Annual Progress Reports • API Quarterly and Annual Progress Reports • Core list and/or copies of the awards, performance reports, relevant assessment reports, and other key US-ASEAN, Government of Indonesia (GOI), and IP documentation. EVALUATION PURPOSE, AUDIENCE, AND INTENDED USE The purpose of this evaluation is to investigate and determine the effectiveness of three USAID/ ASEAN activities: PROSPECT, IGNITE, and API. Specifically, the objectives of this evaluation are to: 1. Review performance of the ASEAN activities with particular focus on project design management, and implementation, 2. Identify success and opportunities, and 3. Identify key lessons learned that can be used to improve program management and to inform the design of future programming. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 87 With the exclusion of any procurement sensitive sections, USAID intends to disseminate the report widely among a variety of stakeholders such as AMS, other donors, and IPs. This evaluation shall be a performance evaluation as defined in the USAID Evaluation Policy (Annex A). All evaluation materials shall be posted to the USAID Development Exchange Clearinghouse. EVALUATION QUESTIONS In collaboration with USAID/ASEAN, prior to finalizing the evaluation design, evaluators shall review and finalize the following questions. GENERAL EVALUATION QUESTIONS As previously indicated, USAID/ASEAN seeks to undertake a joint midterm evaluation to collect valuable insight on project design, management, and implementation in order to adaptively manage its existing portfolio and inform the design of its future programming. The illustrative general evaluation questions, which are applicable to the three activities that will be evaluated, include: 1. Which type of USAID/ASEAN assistance (interventions or set of interventions) was most successful or effective? And why? a. What are the key successes, lessons learned, and opportunities gleaned from the activities’ implementation that can be used or replicated in the future? How have these interventions or set of interventions remained sustainable and planned for funding uncertainty? 2. What were the main challenges that affected project implementation (provide examples)? And how can USAID/ASEAN help address similar challenges in the future? 3. In what ways can USAID/ASEAN improve its effectiveness in supporting project implementation (provide examples)? Which actions or strategies can USAID/ASEAN adopt to improve support to its contractors and coordination among USG players (such as bilateral Missions and State Department)? 4. What was learned (from information or data) through project implementation that can be used to refine or change our activities’ theory models (development hypotheses, theory of changes, results frameworks, or logic models)? In what ways should USG assistance to ASEAN be more targeted or focused? a. To advance policy implementation and achieve tangible results, which type of assistance, under USAID’s manageable control, should be prioritized and why? How should USAID/ASEAN define success for its policy interventions? And what are some ways in which USAID/ASEAN can better measure and communicate these impacts? 5. What limitations to USAID assistance have been observed and how should USAID/ASEAN and/or USG players address them? SPECIFIC EVALUATION QUESTIONS FOR EACH ACTIVITY TO BE EVALUATED The following sections provide the illustrative questions applicable to each of the three activities that will be evaluated. 88 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION PROSPECT 1. How has PROSPECT adjusted its strategy and approach in supporting ASEAN with the current set of ASEAN partners? What are the specific instances in which these adjustments have been made and why? a. Which of the PROSPECT interventions or set of interventions were most successful (provide examples)? Why? b. How does ASEAN assess PROSPECT’s strategy to support its priorities? 2. To what extent has USAID provided the right kind of administrative, diplomatic and engagement to support project implementation and ASEAN in developing institutional capacity and human capital to better advance shared priorities? 3. Is PROSPECT’s theory model (development hypothesis, theory of change, results framework, or logic model) still valid? Why or why not? 4. What are the challenges faced with implementing an assistance program that spans such a broad spectrum of issues? What are the advantages to such an approach? Which type of assistance should be prioritized and what criteria should we apply to making this decision? API 1. What challenges has or is API faced in achieving the project’s intended objectives or vision? In which aspects of the program implementation has API been the most and least successful? What are the reasons for this? 2. What were the greatest challenges faced by the project during project start up, which coincided with the start of the COVID-19 Pandemic? a. How did the pandemic impact API's ability to reach full productivity? b. What kind of support provided by USAID/ASEAN were most effective? What, if any, additional support could USAID/ASEAN have provided to ensure a smooth start-up? c. What are the key lessons learnt? How should USAID/ASEAN modify activities’ interventions and deliverables to ensure maximum program impact? 3. In what ways did the ASEAN endorsement process impact the pace of project implementation? What interventions could or could not be undertaken while awaiting the endorsement process? 4. In what ways should USAID/ASEAN adjust API’s theory model (development hypothesis, theory of change, results framework, or logic model), why? 5. What has been learned about which program elements (interventions or set of interventions) are most effective in the API design? And which program elements are least impactful? IGNITE 1. In what ways is USAID/ASEAN assistance helping narrow the development gap in ASEAN's EG sector? a. Particularly under the digital group and Trade Facilitation Group, which type of USAID/ASEAN assistance (interventions or set of interventions) was most effective in meeting ASEAN's needs (provide examples)? And why? FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 89 2. Which type of USAID/ASEAN assistance (interventions or set of interventions) are most effective in meeting ASEAN's needs and achieving deliverables in areas of shared priority (provide examples)? And why? 3. In what ways has USAID/ASEAN contributed to inclusive growth (provide examples)? a. What specific actions should USAID/ASEAN take to improve and sustain its support to project implementation? GENDER CONSIDERATIONS In accordance with USAID’s Automated Directive System 201 point 7, the research design for this evaluation shall consider gender-specific aspects pertinent to USAID/ASEAN assistance. As such the evaluation team shall explore gender aspects in line with the evaluation questions and available data sources. DATA COLLECTION AND ANALYSIS METHODS The evaluation team shall consider a mixed-method evaluation approach to the extent possible. The methodology shall combine a review of quantitative data and application of qualitative evaluation techniques to obtain information, opinions, and data from relevant ASEAN entities, counterparts, contractors, partners, beneficiaries, and other donors as appropriate. The approach shall be participatory and involve the use of appropriate data collection tools. By using a mixed approach, the evaluation team shall gain insight on the outcomes of the USAID/ASEAN activities (mostly from quantitative data collected by the different activities) and the processes (mostly qualitative information provided by the IP staff and key informants) that lead to those outcomes. The evaluation team shall draft an evaluation methodology or design for USAID approval. The detailed methodology of this evaluation shall be described by the evaluation team in the Work Plan. It shall include an evaluation matrix that explicitly links the evaluation questions to particular data collection approaches and data sources. During the initial team planning meeting, the evaluation shall select specific methodologies for the various activities. In choosing possible data collection methods, the evaluation design must consider the implications of an operating environment that has been altered by the COVID-19 pandemic. Depending on the prevailing environment at the time of the evaluation, the evaluation team must propose an evaluation design that considers remote monitoring or data collection methods as appropriate. This may include methods such as the use of cellphones or SMS to conduct interviews, interactive voice response, voice calls, or maximizing national data collection activities that are still active etc. The following essential elements shall be included in the methodology as well as the additional innovative methods proposed by the evaluation team: Document Review: USAID/ASEAN will provide the evaluation team with a core list and/or copies of the awards, performance reports, relevant assessment reports, and other key USAID/ASEAN, ASEAN Secretariat, and IP documentation before the evaluation begins. The evaluation team leader shall be responsible for expanding this background documentation as appropriate, review, prioritize, and distribute it to the other team members for their review. All team members shall review relevant documentation before their initial team meetings. 90 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION Key informant interviews: The evaluation team shall conduct interviews to obtain feedback from a number of stakeholders sufficient to achieve saturation of perspectives from the following groups (the list is not exhaustive): Relevant ASEAN representatives, including but not limited to, ASEC, Sectoral Bodies, the Project Management Office of the ASEAN Single Window, Senior Officials Meeting for ASEAN Ministers on Agriculture and Forestry, ASEAN Sanitary and Phyto-sanitary Contact Points, ASEAN Senior Officials on Rural Development and Poverty Eradication, ASEAN Coordinating Committee on MSME, ASEAN Intergovernmental Commission on Human Rights, Senior Official Meeting on Transnational Crime, ASEAN Commission on the Promotion and Protection of the Rights of Women and Children, Senior Official Meeting on Youth, ASEAN Humanitarian Assistance Center, ASEAN Committee on Disaster Management, ASEAN Inter-Parliamentary Assembly, Council of ASEAN Chief Justices, ASEAN Foundation. Private Sector: representatives of business associations and business networks, US-ASEAN Business Council, and individual businesses. Relevant IP staff from their home and field offices. Relevant USG officials from USAID/ASEAN, bilateral Missions and State Department. Other stakeholders such as dialogue partners. Focus Group Discussions: To the extent possible given travel restrictions and social distancing measures, the team shall conduct group discussions (through group video conversations) with a range of representatives across the USAID/ASEAN activities’ beneficiaries and stakeholders to obtain feedback on the perceived effects of the activities. Discussion groups shall include balanced numbers of men and women; in addition, as appropriate to local circumstances sex- or age￾segregated discussion groups shall be used to promote free discussion by women, men, and youth. If focus group discussions are not feasible, the team shall define alternative methods of gaining beneficiary perspectives through an online survey, SMS survey, or other means. Site Visits: Where site visits are not possible, the evaluation team can solicit time-stamped photos and videos of activity locations as appropriate while paying attention to matters of confidentiality. Staff from MEL-P and the USAID/ASEAN team shall assist in organizing logistics for all site visits or remote-based data collection approaches for the Evaluation Team. Data Analysis: Team members shall analyze USAID/ASEAN team documents and information gained from key informant interviews and other data collection methods in order to inform their findings and recommendations. The team shall keep a record of meetings that take place and record the summaries of each meeting. Some quantitative analyses may be featured, for example, in the review of performance monitoring data or in the analysis of the activity’s efficiency. The evaluation team shall analyze the information collected to establish credible answers to the questions and provide major trends and issues. USAID requires that evaluations explore issues of gender; thus, the evaluation shall examine gender issues within the context of the evaluation of these activities. Limitations: Since key informant interviews will be a major source for validation of information available from the activities, chances of bias are likely. The evaluation shall carefully decide on the methodology and select interviewees in a way that the possibility of bias is avoided or reduced to a minimum. The evaluation team shall also take systematic actions to counter any biases in interpretations of collected data by the team. FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 91 The methodology narrative of the evaluation report shall discuss the merits and limitations of the final evaluation methodology, in particular, issues related to remote data collection methods as appropriate. The evaluation team shall design appropriate tools for collecting data from various units of analysis. The tools shall be shared with USAID during the evaluation and as part of the evaluation report. DELIVERABLES The evaluation team shall be responsible for the following deliverables. Specific due dates shall be proposed in the evaluation design, following the team planning meeting. DELIVERABLES ESTIMATED DUE DATES Draft Evaluation Design and Work Plan, including detailed research methodology, data collection instruments, sampling plan, and implementation plan and schedule. October 18, 2021 Final Evaluation Design and Work Plan October 24, 2021 Presentation(s) to USAID of key findings and any preliminary conclusions and recommendations. December 6 to 10, 2021 Draft Evaluation Report December 27, 2021 Final Evaluation Report January 21, 2022 REPORTING AND DISSEMINATION The format of the evaluation report shall follow USAID guidelines set forth in the USAID Evaluation Report Template (http://usaidlearninglab.org/library/evaluation-report-template) and the How-To Note on Preparing Evaluation Reports (http://usaidlearninglab.org/library/how-note-preparing￾evaluation-reports). Evaluation team members shall be provided with the USAID’s mandatory statement of the evaluation standards they are expected to meet (see Annex A). TEAM COMPOSITION The evaluation team shall be composed of the following six illustrative experts: • Team Leader, an international evaluation expert with prior successful experience in leading evaluation or research teams. • Research Assistant, an Indonesian national with M&E knowledge and experience, and in research, analysis, or reporting on one or more of the following issues: rule of law, human rights, trade, or wildlife trafficking. • Policy expert with knowledge and experience in researching, assessing, or evaluating regional integration of rule of law and human rights programs, preferably in Southeast Asia. Preference shall be given to qualified and experienced nationals from the AMS (i.e., Indonesia, Malaysia, Philippines, Singapore, or Thailand). • Trade and Investment expert with knowledge and experience in researching, assessing, or evaluating trade facilitation, economic growth, or related fields, preferably in Southeast Asia. Preference shall be given to qualified and experienced nationals from the AMS. • Anti-Trafficking expert with knowledge and experience in researching, assessing, or evaluating regional and/or transnational human and wildlife trafficking challenges, preferably 92 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION in Southeast Asia. Preference shall be given to qualified and experienced nationals from the AMS. Each team member shall have writing responsibilities for the draft and final evaluation report, under the guidance of the team leader. The evaluation team shall have combination of the following qualifications: • Strong knowledge of ASEAN activities or programs • Expertise in program evaluations, including qualitative & quantitative evaluation practices. • Knowledge of USAID programming practices. • Excellent writing and interpersonal communication skills. USAID PARTICIPATION Regular communication between the evaluation team and the designated USAID/ASEAN Activity Manager will be essential to the successful execution of this midterm evaluation. The evaluation team shall keep USAID apprised of changes and developments that necessitate any significant decision￾making or modification of the approved evaluation design. Possible USAID participation in the data collection phase of the evaluation will be discussed in the team planning meeting, prior to the start of the field work. SCHEDULING AND LOGISTICS MEL-P shall be responsible for handling all logistics. The table below provides the estimated start and end dates for key tasks or deliverables. ESTIMATED EVALUATION TIMELINE TASKS OR DELIVERABLES ESTIMATED START DATE DD/MMM/YY ESTIMATED END DATE DD/MMM/YY SOW and Budget development 31-Mar-21 06-Jul-21 Evaluation team: Development of SOWs, recruitment and contracting 25-Jun-21 03-Sep-21 Desk review and team planning meetings with USAID, IPs and ASEAN 06-Sep-21 23-Sep-21 Draft Evaluation Design & Work Plan, including presentations to USAID 24-Sep-21 13-Oct-21 Final Evaluation Design & Work Plan, including USAID’s approval 14-Oct-21 22-Oct-21 Fieldwork & Data Analysis 01-Nov-21 26-Nov-21 Debriefs on initial findings to USAID, IP, and ASEAN, including preparation 29-Nov-21 10-Dec-21 Report Writing 13-Dec-21 24-Dec-21 Submission of Draft Evaluation Report 27-Dec-21 USAID’s and IP’s review of Draft Evaluation Report 28-Dec-21 31-Dec-21 Drafting and submission of Final Evaluation Report 03-Jan-22 18-Jan-22 FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 93 ANNEX A (From SOW): USAID EVALUATION POLICY APPENDIX I USAID EVALUATION POLICY, APPENDIX 1 CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT • The evaluation report should represent a thoughtful, well-researched and well-organized effort to objectively evaluate what worked in the project, what did not and why. • Evaluation reports shall address all evaluation questions included in the scope of work. • The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by the technical officer. • Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists, and discussion guides will be included in an Annex in the final report. • Evaluation findings will assess outcomes and impact on males and females. • Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). • Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence. • Sources of information need to be properly identified and listed in an annex. • Recommendations need to be supported by a specific set of findings. • Recommendations should be action-oriented, practical, and specific, with defined responsibility for the action. 94 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION ANNEX III: EVALUATION DESIGN MATRIX EVALUATION DESIGN MATRIX GENERAL EVALUATION DESIGN MATRIX EQ# Evaluation question Objectives/ components/ results contributed to by EQ Investigation issues/sub-questions Data collection methods/sources Data analysis approaches GEQ1 How effective are the USAID/ASEAN projects in achieving their expected results? PROSPECT: ALL Components ✔ Progress towards expected results ✔ Implementation gaps or opportunities for achieving greater results ✔ How USAID/ASEAN can support activities to improve implementation effectiveness and results achievement: – Actions or strategies to improve support to its contractors – Coordination among USG players (such as bilateral Missions and State Department)? ✔ KIIs ✔ FGDs ✔ Secondary analysis of monitoring data Thematic analysis of KII and FGD transcripts Descriptive analysis for quantitative data IGNITE: ALL objectives API: ALL components GEQ2 What are the main challenges that affect project implementation, and how can those challenges be mitigated? PROSPECT: ALL Components ✔ Challenges/limitations experienced ✔ How the activities have coped with challenges experienced ✔ What USAID/ ASEAN can do to improve the effectiveness of activities implementation – Support to contractors – Coordination of USG players (such as bilateral Missions and State Department)? ✔ KIIs ✔ FGDs Thematic analysis of KII and FGD transcripts IGNITE: ALL objectives API: ALL components GEQ3 What was learned (from information or data) PROSPECT: ALL Components ✔ KIIs ✔ FGDs Thematic analysis of KII FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 95 through project implementation that can be used to refine or change our activities’ theory models (development hypotheses, theory of changes, results frameworks, or logic models)? In what ways should USG assistance to ASEAN be more targeted or focused? IGNITE: ALL objectives ✔ Relevance, adequacy and viability of the theories of change in relation to implementation strategies and needs ✔ Strategies for ensuring sustainability of USAID/ASEAN supported initiatives ✔ Key lessons learned at the mid-term ✔ What assistance/interventions USAID should prioritize to advance policy implementation and achieve tangible results and why (based on the focus of the current three activities) ✔ How/what USAID/ASEAN should measure to assess and communicate the results/successes of their interventions better – How should USAID/ASEAN define success for its policy interventions? – What are some ways in which USAID/ASEAN can impacts? and FGD transcripts API: ALL components GEQ4 What limitations to USAID assistance have been observed and how should USAID/ASEAN and/or USG players address them? PROSPECT: ALL Components ✔ Limitations to USAID’s current assistance to ASEAN ✔ Limitations to measurement of USAID assistance results/impact ✔ Type of assistance within manageable control that USAID/ASEAN should prioritize ✔ KIIs ✔ FGDs Thematic analysis of KII and FGD transcripts IGNITE: ALL objectives API: ALL components 96 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION PROSPECT EVALUATION DESIGN MATRIX EQ# Evaluation question Objectives/expected results Investigation issues/sub-questions Data collection methods/sources Data analysis approaches PEQ1 How has PROSPECT adjusted its strategy and approach in supporting ASEAN with the current set of ASEAN partners? What are the specific instances in which these adjustments have been made and why? ✔ Help ASEAN to take a united, collaborative and proactive approach to addressing transnational challenges ✔ Enable a more robust response to emerging humanitarian crises, particularly relating to natural disasters ✔ Address governance gaps in implementing and enforcing ASEAN convention/Agreements ✔ Strengthen civil society engagement and influence on critical issues ✔ Expand rights and opportunities for vulnerable and marginalized groups ✔What are the key interventions undertaken by the PROSPECT activity ✔What specific strategies and approaches has PROSPECT applied to implement each intervention? ✔What adjustments has PROSPECT made in its approaches, if any, and why? ✔Which of the PROSPECT interventions or set of interventions have been most successful (provide supporting results)? Why? ✔What are the results of PROSPECT interventions at mid-term ✔ How does ASEAN assess PROSPECT’s strategy to support its priorities? ✔ In what way has the strategy been working out and help ASEAN to accomplish its objectives? What are the challenges? ✔ How should the strategy and approach be improved for more effective and better results? ✔ How has COVID-19 pandemic has affected project implementation and progress towards expected results? ✔ How has PROSPECT mitigated and adjusted its strategy due to COVID-19 ✔ Desk review (Documents, reports, publication of USAID / IPs ASEAN Secretariat/ Sectoral Bodies/AMS KII: USAID, IPs (DAI), Relevant Divisions at ASEAN Secretariat, ASEAN Sectoral Bodies, Entities Associated with ASEAN, ASEAN Member States, CSOs Qualitative data analysis FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 97 pandemic? And what are the key lessons learned? PEQ2 What support has USAID provided to PROSPECT with respect to administrative and diplomatic engagement to support project implementation and ASEAN in developing institutional capacity and human capital to better advance shared priorities? ✔ Help ASEAN to takes a united, collaborative and proactive approach to addressing transnational challenges ✔ Enable a more robust response to emerging humanitarian crises, particularly relating to natural disasters ✔ Address governance gaps in implementing and enforcing ASEAN convention/Agreements ✔ Strengthen civil society engagement and influence on critical issues Expand rights and opportunities for vulnerable and marginalized groups ✔ How has USAID/ASEAN supported PROSPECT to create a positive implementation environment? – Diplomatic engagement support – Administrative support ✔Which support provided by USAID are most relevant to ASEAN needs, and which was not? Why? ✔ How effective is the support from USAID in helping PROSPECT with project implementation? Give specific examples of the achievements that resulted from the diplomatic and administrative engagement support from USAID ✔What are additional actions/support that USAID/ASEAN could provide to strengthen the performance of PROSPECT? ✔What specific improvements do you anticipate will happen in PROSPECT implementation and results if USAID provided that additional support? ✔ Desk review (Documents, reports, publication of USAID / IPs ASEAN Secretariat/ Sectoral Bodies/AMS ✔ KII: USAID, IPs (DAI), Relevant Divisions at ASEAN Secretariat, ASEAN Sectoral Bodies, Entities Associated with ASEAN, ASEAN Member States, CSOs Qualitative data analysis PEQ3 How effective is PROSPECT’s theory model in guiding the implementation of project activities towards achieving expected results (development ✔ Help ASEAN to takes a united, collaborative and proactive approach to addressing transnational challenges ✔ Enable a more robust response to emerging ✔ Perspectives about PROSPECT TOC – Does it relate well to PROSPECT design and implementation needs? How? (Relevance) – How effective is it in guiding the design and implementation of ✔ Desk review (Documents, reports, publication of USAID / IPs ASEAN Secretariat/ Sectoral Bodies/AMS ✔ Qualitative data analysis 98 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION hypothesis, theory of change, results framework, or logic model)? Why or why not? humanitarian crises, particularly relating to natural disasters ✔ Address governance gaps in implementing and enforcing ASEAN convention/Agreements ✔ Strengthen civil society engagement and influence on critical issues ✔ Expand rights and opportunities for vulnerable and marginalized groups PROSPECT? (effectiveness/viability) – What gaps, if any, exist in PROSPECT TOC? How should PROSPECT TOC be improved? (what should be removed, added or modified)? (Adequacy) – What challenges has PROSPECT experienced when applying the TOC to develop and implement interventions? (Viability/effectiveness) – What has PROSPECT done to address those challenges? – What are the key lessons learnt from PROSPECT’s theory of change? ✔ Consistent with the TOC, what has PROSPECT done to contribute to improving ASEAN capacity to: – Address transnational challenges related to human security? Give specific examples – Strengthen effective response to humanitarian crises? Give specific examples ✔What are PROSPECT's main achievements/results on strengthening ASEAN capacity to address transnational human security challenges and humanitarian crises? ✔KII: USAID, IPs (DAI), Relevant Divisions at ASEAN Secretariat, ASEAN Sectoral Bodies, Entities Associated with ASEAN, ASEAN Member States, CSOs FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 99 ✔ Consistent with the TOC, what has PROSPECT done to increase ASEAN capacity to strengthen good governance and the rule of law? ✔ How has PROSPECT support impacted ASEAN’s capacity to: – Engage with judicial, legislative, and executive branches? Give specific examples – Increase civil society participation in ASEAN? Give specific examples – Expand rights and opportunities for women, youth, and other vulnerable and marginalized groups? Give specific examples ✔What are the main achievements/results of PROSPECT support to increase ASEAN capacity to promote good governance and advance the rule of law? Critical Assumption: ✔What critical factors (both from ASEAN and USAID internal and external sides) should (1) USAID, and (2) PROSPECT address in their TOC to strengthen project design, implementation and results? ✔What additional indicators should USAID use to measure the results of PROSPECT besides the existing indicators? 100 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION PEQ4 What challenges has PROSPECT faced with implementing an assistance program that spans such a broad spectrum of issues [SPECIFY THE SPECTRUM OF ISSUES]? ✔ Help ASEAN to takes a united, collaborative and proactive approach to addressing transnational challenges ✔ Enable a more robust response to emerging humanitarian crises, particularly relating to natural disasters ✔ Address governance gaps in implementing and enforcing ASEAN convention/Agreements ✔ Strengthen civil society engagement and influence on critical issues Expand rights and opportunities for vulnerable and marginalized groups ✔What are the benefits of the broad spectrum approach of PROSPECT encompassing political, security, and socio-cultural issues? ✔What are the challenges/limitations experienced by PROSPECT with the implementation of a broad spectrum of issues targeted by the project? ✔ Considering the different interventions of PROSPECT and the needs of ASEAN, which type of assistance should USAID/ASEAN prioritize and why? ✔What criteria should USAID/ASEAN use to decide or prioritize their assistance to ASEAN? Give specific suggestions and reasons for those suggestions? ✔ Desk review (Documents, reports, publication of USAID / IPs ASEAN Secretariat/ Sectoral Bodies/AMS ✔ KII: USAID, IPs (DAI), Relevant Divisions at ASEAN Secretariat, ASEAN Sectoral Bodies, Entities Associated with ASEAN, ASEAN Member States, CSOs ✔ Qualitative data analysis FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 101 API EVALUATION DESIGN MATRIX EQ# Evaluation question Objectives/expected results Investigation issues/sub-questions Data collection methods/sources Data analysis approaches AEQ1 What are the main successes of the API activity? And what challenges is the activity facing in achieving its intended objectives or vision? ✔ Identifying concrete needs and opportunities for supporting the implementation of targeted ASEAN policies at the Member State level. ✔ Providing concrete technical assistance to Member States to strengthen their implementation of targeted ASEAN policies. ✔ USAID contribution to the IPS reinforced. ✔ ASEAN centrality in the U.S. Indo-Pacific vision strengthened. ✔ Implementation of ASEAN policies at the Member State level advanced. ✔ What concrete needs of ASEAN has API identified regarding the development and implementation of ASEAN policies? – What are your opinions about those needs? Probe for appropriateness and importance of those needs to the implementation of ASEAN policies? – What activities have API undertaken to address the identified ASEAN policy implementation needs? Probe for API interventions to: ▪ Support the achievement of the IPS vision of enhancing shared prosperity ▪ Unlocking private enterprise-led economic growth – What are the main gaps in the support provided by API to meet ASEAN needs? ✔ What Technical Assistance has API provided directly to the AMS to realize the API vision? – Give examples of results achieved from that API Technical Assistance to AMS? ✔ What are the main achievements/results of API at mid-term? ✔ Desk review (Documents, reports, publication of USAID / IPs ASEAN Secretariat/ Sectoral Bodies/AMS ✔KII: USAID, IPs (IBI), Relevant Divisions at ASEAN Secretariat, ASEAN Sectoral Bodies, Entities Associated with ASEAN, ASEAN Member States ✔ Qualitative data analysis 102 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION ✔ How has USAID/ASEAN supported API with the implementation of project activities? – Was that support effective? How? (Give specific examples) – What additional actions/support should USAID/ASEAN provide to assist (1) ASEAN, (2) API with the implementation of ASEAN policies – What difference could be anticipated if USAID/ASEAN provided this additional support to API (implementation effectiveness and results)? ✔ How has API supported the implementation of the following four areas of the ASEAN Framework on DDG e.g: – Data life cycle and ecosystem – Cross-border data flows – Digitalization and emerging technologies – Legal, regulatory, and policy provisions ✔ What progress has been realized from API’s support for the implementation of the Digital Data Governance (DDG) framework? ✔ How has COVID-19 pandemic impacted API implementation? ✔ What coping strategies has API applied to mitigate the challenges from COVID-19 pandemic? FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 103 ✔ What other challenges has API experienced with the activity implementation, and how has API coped? (identifying needs, technical assistance to ASEAN and AMS challenges etc) AEQ2 In what ways did the ASEAN endorsement process impact the pace of project implementation? What interventions could or could not be undertaken while awaiting the endorsement process? ✔ Identifying concrete needs and opportunities for supporting the implementation of target ed ASEAN policies at the Member State level. ✔ Providing concrete technical assistance to Member States to strengthen their implementation of targeted ASEAN policies. ✔ USAID contribution to the IPS reinforced. ✔ ASEAN centrality in the U.S. Indo-Pacific vision strengthened. ✔ Implementation of ASEAN policies at the Member State level advanced. ✔ What were the experiences of API regarding the project endorsement process by ASEAN? – What process was followed? – What are your opinions about the endorsement process? – How effective was the endorsement process? Give concrete reasons and examples. ✔ How did the endorsement process impact the implementation of API activities? – Probe for positive and negative impact – What interventions could/could not be undertaken while awaiting the endorsement process? ✔ What support did USAID/ ASEAN provide in support of API endorsement? – Was that support effective in obtaining API endorsement? – If yes/no, how/why? ✔ How has the endorsement of API by ASEAN been beneficial in supporting the implementation of API activities ✔ Desk review (Documents, reports, publication of USAID / IPs ASEAN Secretariat/ Sectoral Bodies/AMS ✔KII: USAID, IPs (IBI), Relevant Divisions at ASEAN Secretariat, ASEAN Sectoral Bodies, Entities Associated with ASEAN, ASEAN Member States ✔ Qualitative data analysis AEQ3 In what ways should USAID/ ASEAN adjust ✔ Identifying concrete needs and opportunities for supporting the implementation of targeted ✔ Can you please share your perspectives about API Theory of Change (TOC)? ✔ Desk review (Documents, reports, publication of USAID / ✔ Qualitative data analysis 104 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION API’s theory model (development hypothesis, theory of change, results framework, or logic model), why? ASEAN policies at the Member State level. ✔ Providing concrete technical assistance to Member States to strengthen their implementation of targeted ASEAN policies. ✔ USAID contribution to the IPS reinforced. ✔ ASEAN centrality in the U.S. Indo-Pacific vision strengthened. ✔ Implementation of ASEAN policies at the Member State level advanced. – Does it relate well to the design and implementation needs of ASEAN and AMS? How? (Relevance) – How effective is it in guiding the design and implementation of API activities? (Effectiveness/viability) – What gaps, if any, exist in API TOC? How should API TOC be improved? (what should be removed, added or modified)? (Adequacy) – What challenges has API experienced with applying the TOC to develop and implement interventions? (Viability/effectiveness) – What has API done to address those challenges? – What are the key lessons learnt from API’s theory of change? ✔ Consistent with the API TOC, what has API done to continually – Identify key opportunities through research and consultation – Refine API approaches to those opportunities based on gathered evidence ✔ What are the main achievements/results of API support to ASEAN regarding the implementation of policies under AEC? ASEAN Single Window (ASW) and Digital Data Governance (DDG), among others? IPs ASEAN Secretariat/ Sectoral Bodies/AMS ✔KII: USAID, IPs (IBI), Relevant Divisions at ASEAN Secretariat, ASEAN Sectoral Bodies, Entities Associated with ASEAN, ASEAN Member States FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 105 ✔ What critical factors should (1) USAID, and (2) API address in their TOC to strengthen project design, implementation and results? – Probe for (something related to external and internal factors) ✔ What additional indicators should USAID use to measure the results of API besides the existing indicators? AEQ4 What lessons have been learned about program elements (interventions or set of interventions) are most effective in the API design? And which program elements are least impactful? ✔ Identifying concrete needs and opportunities for supporting the implementation of targeted ASEAN policies at the Member State level. ✔ Providing concrete technical assistance to Member States to strengthen their implementation of targeted ASEAN policies. ✔ USAID contribution to the IPS reinforced. ✔ ASEAN centrality in the U.S. Indo-Pacific vision strengthened. ✔ Implementation of ASEAN policies at the Member State level advanced. ✔ What are the key lessons learnt from API implementation with regard to: – The extent and value of aligning API strategies and interventions to ASEAN needs – Best practices/strategies for enhancing policy development capacity building and results? – Best practices for providing technical support to ASEAN and AMS – Types and value of USAID/ASEAN support in the implementation of API performance – API TOC and Result Framework ✔ Desk review (Documents, reports, publication of USAID / IPs ASEAN Secretariat/ Sectoral Bodies/AMS ✔KII: USAID, IPs (IBI), Relevant Divisions at ASEAN Secretariat, ASEAN Sectoral Bodies, Entities Associated with ASEAN, ASEAN Member States ✔ Qualitative data analysis 106 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION IGNITE EVALUATION DESIGN MATRIX EQ# Evaluation question Objectives/expected results Investigation issues/sub-questions Data collection methods/sources Data analysis approaches IEQ1 In what ways is USAID/ ASEAN assistance helping narrow the development gap in ASEAN's EG sector? Which type of USAID/ ASEAN assistance (interventions or set of interventions) under trade facilitation and digital economy group was most effective in meeting ASEAN's needs (provide examples)? And why? ✔ To develop and harmonize policy and standards and reduce barriers to increased trade and investment ✔ To increase the availability within and among AMS of relevant, affordable, and secure digital￾enabled services ✔ To engage ASEAN citizens, esp. youth, women, and small businesses, in economic prosperity ✔ What interventions is IGNITE currently undertaking to narrow the development gap in EG? Probe for: • Trade facilitation – ASEAN Single Windows (ASW) – Logistics – Harmonization of standards • Digital Economy – Regulatory reform for the digital economy – Improved regulations for cybersecurity and data privacy – Expanded broadband access ✔ How has the implementation of the ASEAN Trade Facilitation Framework and the World Trade Organization (WTO) Trade Facilitation Agreement affected the time and cost of trade? – Which aspects of trade were impacted by the framework and how? – if it decreased the time, by how much? – If it did not decrease, why? ✔ Which interventions under the trade facilitation are most effective? Probe for: – reduce trade cost for private firms – support financial sector integration Methods: ✔ Desk Review ✔ KII (USAID, IP, ASEC) Qualitative Analysis FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 107 – reduce logistical bottlenecks – harmonizing standards ✔ Which interventions under trade facilitation are not effective? Why? ✔ Which interventions under the digital economy are most effective in helping AMS to achieve: – support e-commerce by SMEs, including digital finance – strengthen cyber security and data privacy – making the internet more affordable and accessible Why? and give specific examples. ✔ Which interventions under the digital economy are not effective? Why? IEQ2 Which type of USAID/ ASEAN IGNITE assistance (interventions or set of interventions) are most effective in meeting ASEAN's needs and achieving deliverables in areas of shared priority (provide examples)? And why? ✔ To engage ASEAN citizens, esp. youth, women, and small businesses, in economic prosperity ✔ To increase the availability of relevant, affordable, and secure digital-enabled services within and among AMS ✔ To increase the use of science, technology, and ✔ What are the main needs of ASEAN to meet expected results in the areas of shared priority? ● Science, Technology, and Innovation ● Gender ● MSMEs ✔ What interventions does IGNITE implement to address those needs? ✔ What has IGNITE done under Science, Technology, and Innovation (STI) to support the realization of cross cutting results? Probe for – Enhancing the human capital in science, technology, engineering, and mathematics (STEM) Methods: ✔ Desk Review ✔ KII (USAID, IP, ASEC) Qualitative Analysis 108 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION innovation in ASEAN entrepreneurship – Increased the mobility of regional research talent – Increased regional collaboration and/or research talent mobility in STEM – Increased women’s contribution and participation in STEM fields increased and recognized – Increased research commercialization – Increased SME access to finance, technology and e-commerce – Increased women’s participation in STEM fields ✔ How effective is it in meeting ASEAN needs? Give specific examples in each intervention ✔ In your opinion, which interventions are not effective? Why? Give specific examples. ✔ Which interventions under gender and SMEs are most effective to increase the prosperity of women and small businesses in ASEAN? How? Which are not and why? IEQ3 How effective is IGNITE’s theory model in guiding the implementation of project activities towards achieving expected results (development hypothesis, theory of change, results framework, or logic model)? Why or why not? ✔ To develop and harmonize policy and standards and reduce barriers to increased trade and investment. ✔ To engage ASEAN citizens, esp. youth, women, and small businesses, in economic prosperity ✔ Perspectives about IGNITE Theory of Change (TOC): – Does it relate well to the design and implementation needs? How? (Relevance) – How effective is it in guiding the design and implementation of IGNITE activities? (Effectiveness/viability) – What gaps, if any, exist in IGNITE TOC? How should IGNITE TOC be improved? Methods: ✔ Desk Review ✔ KII (USAID,IP) Qualitative Analysis FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 109 ✔ To increase the availability of relevant, affordable, and secure digital-enabled services within and among AMS ✔ To increase the use of science, technology, and innovation in ASEAN entrepreneurship (What should be removed, added or modified)? (Adequacy) – What challenges has IGNITE experienced with applying the TOC to develop and implement interventions? (Viability/effectiveness) – What has IGNITE done to address those challenges? – What are the key lessons learnt from IGNITE’s theory of change? ✔Consistent with the TOC, what has IGNITE done to improve ASEAN institutional capacity to: – Mobilize sufficient political will to advance ASEAN priorities – Reduce the time and cost of trade, particularly in key growth sectors (e.g. digital services and science, technology, and innovation) ✔ What are the main achievements/results of IGNITE in helping ASEAN to achieve accelerated and inclusive growth among AMS? – How has IGNITE support contributed to benefitting AMS consumers, firms and US exporters and workers? Give specific examples. (Probe for Outcome) ✔ What critical factors (both from ASEAN and USAID internal and external sides) should (1) USAID, and (2) IGNITE address in their TOC to strengthen project design, implementation and results? 110 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION ✔ What additional indicators should USAID use to measure the results of IGNITE besides the existing indicators? IEQ4 In what ways has USAID/ ASEAN contributed to inclusive growth (provide examples)? ✔ To develop and harmonize policy and standards and reduce barriers to increased trade and investment. ✔ To engage ASEAN citizens, esp. youth, women, and small businesses, in economic prosperity ✔ To increase the availability of relevant, affordable, and secure digital-enabled services within and among AMS ✔ To increase the use of science, technology, and innovation in ASEAN entrepreneurship ✔ How has USAID/ASEAN supported the inclusive growth priorities of ASEAN? ✔ What kind of support provided by USAID/ASEAN is most effective in facilitating inclusive growth in ASEAN in the areas of Innovation, Trade and E￾Commerce? ✔ How has USAID/ASEAN supported IGNITE with the implementation of project activities? Probe for support with diplomatic and administrative engagement – Was that support effective? How? ✔ What additional actions/support could USAID/ASEAN provide to (1) ASEAN, (2) IGNITE in their efforts to achieve inclusive growth in the AMS? – What difference could be anticipated if USAID/ASEAN provided this additional support (Probe for difference in implementation, effectiveness and results)? ✔ How has COVID-19 impacted IGNITE implementation? Methods: ✔ Desk Review ✔ KII (USAID, IP, ASEC) Qualitative Analysis FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 111 – What coping strategies has IGNITE undertaken to mitigate the challenges posed by COVID-19? ✔ What other challenges has IGNITE faced and how has it coped with those challenges? ✔ What lessons have been learnt from the design and implementation of IGNITE? 112 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION ANNEX IV: DATA COLLECTION AND ANALYSIS TOOLS KEY INFORMANT INTERVIEW GUIDE PROSPECT DATA COLLECTION TOOL Interviewee’s Title / Position Organization (ASEC/Sectoral Body/Entity/Others (please specify) Level ☐Regional ☐National Country Gender Support Area Date # Interview Questions Responding Participants’ Institution USAID/ PROSPECT ASEAN Sectoral Bodies/ Entities ASEC CSOs and OTHERS PEQ1 How has PROSPECT adjusted its strategy and approach in supporting ASEAN with the current set of ASEAN partners? What are the specific instances in which these adjustments have been made and why? 1.1 What are your main roles related to USAID’s PROSPECT activity? ✔ ✔ ✔ ✔ 1.2 What are the key interventions undertaken by PROSPECT? ✔ 1.3 Please describe the support that PROSPECT provides to ASEAN/your institution/office in the areas of: a. Non-traditional security b. Responsive and Transparent Governance c. Human Rights and Opportunities for Marginalized and Vulnerable Groups a. Non-traditional security. Probe for PROSPECT support for: ● Cross-sectoral coordination on the implementation of the ACTIP and APA ● Counter violent extremism, including supporting the multi￾sectoral coordination ✔ ✔ ✔ FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 113 ● Address non-traditional security threats such as cybercrime, wildlife trafficking, and other potential areas ● Strengthening ASEAN capacity to respond to emerging humanitarian crises For each area supported, probe: ● What have been the results of that support? ● What improvements, if any, have occurred in the following areas supported by PROSPECT? ⮚ Cross sectoral coordination? ⮚ Countering violent extremism? ⮚ Addressing non-traditional security threats ⮚ Strengthening ASEAN capacity to respond to emerging humanitarian crises b. Responsive and Transparent Governance. Probe for PROSPECT support for: ● Capacity building to strengthen linkages between the ASEAN Secretariat, CACJ, and AIPA to promote rule of law in ASEAN ● Transforming ASEAN into a rules-based organization, including improving implementation of agreed upon ASEAN instruments ● Cultivating and strengthening civil society networks to support existing platforms and new ones for effective engagement with ASEAN on critical issues For each area supported, probe for: ● What have been the results of that support? ● What improvements, if any, have occurred in the following areas supported by PROSPECT? ✔ 114 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION ⮚ ASEAN Capacity to strengthen linkages between the ASEAN Secretariat, CACJ, and AIPA? ⮚ Transformation of ASEAN into a rules-based organization ⮚ Civil society networks c. Human Rights and Opportunities for Marginalized and Vulnerable Groups. Probe for PROSPECT support for: ● Supporting the AICHR and other regional actors to promote, protect, and mainstream human rights issues in ASEAN ● Engaging youth to more frequently contribute to ASEAN processes and strengthen shared ASEAN identity ● Expanding the rights and opportunities of women and other vulnerable and marginalized groups For each area supported, probe for: ● What have been the results of that support? ● What improvements, if any, have occurred in the following areas supported by PROSPECT? ⮚ Promoting, protecting, and mainstreaming human rights issues in ASEAN? ⮚ Engaging youth to contribute to ASEAN processes and strengthen shared ASEAN identity ⮚ Expanding the rights and opportunities of women and other vulnerable and marginalized groups 1.4 What specific strategies and approaches has PROSPECT applied to implement each intervention? ✔ 1.5 During your engagement with PROSPECT, how has PROSPECT ✔ ✔ ✔ FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 115 approached your institution in order to implement each intervention? a. What are the specific strategies have they applied? 1.6 What adjustments has PROSPECT made in its approaches, if any, and why? ✔ Since you started receiving support from PROSPECT, have you witnessed times when they have made adjustments to their approaches? a. What adjustments did they make? ✔ ✔ ✔ 1.7 Which of the PROSPECT interventions or set of interventions have been most successful (provide supporting results)? Why? ✔ ✔ ✔ ✔ 1.8 What are the main results of PROSPECT interventions at mid-term? ✔ ✔ ✔ ✔ 1.9 Do you receive any feedback from ASEAN regarding PROSPECT strategy to support their priorities? If so, what feedback has been provided by ASEAN so far? ✔ 1.10 What are your opinions about PROSPECT’s strategy? a. What are the benefits/advantages of applying the strategy? ✔ ✔ ✔ 1.11 How should the strategy and approach be improved for more effective and better results? ✔ ✔ ✔ ✔ 1.12 How has Covid-19 pandemic has affected project implementation and progress towards expected results? ✔ ✔ ✔ ✔ 1.13 How has PROSPECT mitigated and adjusted its strategy due to COVID-19 pandemic? And what are the key lessons learned? ✔ ✔ ✔ ✔ PEQ2 What support has USAID provided to PROSPECT with respect to administrative, diplomatic and engagement to support project implementation and ASEAN in developing institutional capacity and human capital to better advance shared priorities? 2.1 How has USAID/ASEAN supported PROSPECT to create a positive implementation environment? Probe for: a. Diplomatic engagement support b. Administrative support ✔ 2.2 Which support provided by USAID are most relevant to ASEAN needs, and which was not? Why? ✔ 116 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION 2.3 How effective is the support from USAID in helping PROSPECT with project implementation? Give specific examples of the achievements that resulted from the diplomatic and administrative engagement support from USAID ✔ 2.4 What are additional actions/support could that USAID/ASEAN could provide to strengthen the performance of PROSPECT? a. What specific improvements do you anticipate will happen in PROSPECT implementation and results if USAID provided that additional support? ✔ 2.5 Do you have any direct engagement with the USAID with regard to PROSPECT? a. How has USAID reach out or engage with you in the context of PROSPECT’s support? ✔ ✔ ✔ 2.6 What do you expect from USAID with regard to the coordination of the support that PROSPECT provides? ✔ ✔ ✔ PEQ3 How effective is PROSPECT’s theory model in guiding the implementation of project activities towards achieving expected results (development hypothesis, theory of change, results framework, or logic model)? Why or why not? 3.1 What are your perspectives about PROSPECT TOC? a. Does it relate well to PROSPECT design and implementation needs? How? (Relevance) b. How effective is it in guiding the design and implementation of PROSPECT? (effectiveness/viability) c. What gaps, if any, exist in PROSPECT TOC? How should PROSPECT TOC be improved? (what should be removed, added or modified)? (Adequacy) d. What challenges has PROSPECT experienced when applying the TOC to develop and implement interventions? (Viability/effectiveness) e. What has PROSPECT done to address those challenges? f. What are the key lessons learnt from PROSPECT’s theory of change? ✔ FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 117 3.2 Is the support provided by PROSPECT meeting your (ASEAN) real needs? How? (Relevance) a. Can you give example of the need that is meeting? Are they some other needs that this activity should be meeting? Which one are those? (Relevance) ✔ ✔ ✔ 3.3 Consistent with the TOC, what has PROSPECT done to contribute to improving ASEAN capacity to: a. Address transnational challenges related to human security? Give specific examples b. Strengthen effective response to humanitarian crises? Give specific examples ✔ 3.4 How has PROSPECT support impacted ASEAN’s capacity to improve ASEAN capacity to: a. Address transnational challenges related to human security? Give specific examples b. Strengthen effective response to humanitarian crises? Give specific examples ✔ ✔ 3.5 What are PROSPECT's main achievements/ results on strengthening ASEAN capacity to address transnational human security challenges and humanitarian crises? ✔ ✔ ✔ 3.6 Consistent with the TOC, what has PROSPECT done to contribute to improving ASEAN capacity to: a. Strengthens good governance and rules of law? b. Increase civil society participation in ASEAN? c. Expands rights and opportunities for women, youth, and other vulnerable and marginalized groups? ✔ 3.7 How has PROSPECT support impacted ASEAN’s capacity to: a. Engage with judicial, legislative, and executive branches? Give specific examples b. Increase civil society participation in ASEAN? Give specific examples ✔ ✔ ✔ 118 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION c. Expand rights and opportunities for women, youth, and other vulnerable and marginalized groups? Give specific examples 3.8 What are the main achievements/results of PROSPECT support to increase ASEAN capacity to promote good governance and advance the rule of law? ✔ ✔ ✔ 3.9 What critical factors (both from ASEAN and USAID internal and external sides) should (1) USAID, and (2) PROSPECT address in their TOC to strengthen project design, implementation and results? ✔ ✔ ✔ 3.10 What additional indicators should USAID use to measure the results of PROSPECT besides the existing indicators? ✔ ✔ ✔ ✔ PEQ4 What challenges has PROSPECT faced with implementing an assistance program that spans such a broad spectrum of issues? (SPECIFY THE SPECTRUM OF ISSUES) 4.1 What are the benefits of the broad spectrum approach of PROSPECT encompassing political, security, and socio-cultural issues? ✔ ✔ ✔ 4.2 What are the challenges/limitations experienced by PROSPECT with the implementation of a broad spectrum of issues targeted by the project? ✔ ✔ ✔ 4.3 Considering the different interventions of PROSPECT and the needs of ASEAN, which type of assistance should USAID/ASEAN prioritize and why? ✔ ✔ ✔ 4.4 What criteria should USAID/ASEAN use to decide or prioritize their assistance to ASEAN? a. Give specific suggestions and reasons for those suggestions b. What are the advantages and disadvantages to having an assistance program with a broad spectrum of issues (from political, security, socio￾cultural) such as PROSPECT? ✔ ✔ ✔ FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 119 API DATA COLLECTION TOOL Interviewee’s Title / Position Organization (ASEC/Sectoral Body/Entity/Others (please specify) Level ☐Regional ☐National Country Gender Support Area Date # Interview Questions Responding Participants’ Institution API IP (IBI) API COR (USAID) ASEC AMS AEQ1 What are the main successes of the API activity? And what challenges is the activity facing in achieving its intended objectives or vision? 1.1 What concrete needs of ASEAN has API identified regarding the development and implementation of ASEAN policies? a. What are your opinions about those needs? Probe for appropriateness and importance of those needs to the implementation of ASEAN policies? b. What activities have API undertaken to address ASEAN policy implementation needs? Probe for API interventions to: ● Support the achievement of the IPS vision of enhancing shared prosperity ● Unlocking private enterprise-led economic growth c. What are the main gaps in the support provided by API to meet ASEAN needs? ✔ ✔ ✔ ✔ 1.2 What Technical Assistance has API provided directly to the AMS to realize the API vision? a. Give examples of results achieved from that API Technical Assistance to AMS? ✔ ✔ ✔ ✔ 1.3 What are the main achievements/results of API at mid-term? ✔ ✔ ✔ ✔ 1.4 How has USAID/ASEAN supported API with the implementation of project activities a. Was that support effective? How? (Give specific examples) b. What additional actions/support should USAID/ASEAN provide to assist (1) ASEAN, (2) API with the implementation of ASEAN policies c. What difference could be anticipated if USAID/ASEAN provided this additional support to API (implementation effectiveness and results)? ✔ ✔ ✔ ✔ 120 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION 1.5 How has API supported the implementation of the following four areas of the ASEAN Framework on DDG e.g.: a. Data life cycle and ecosystem b. Cross-border data flows c. Digitalization and emerging technologies d. Legal, regulatory, and policy provisions ✔ ✔ ✔ ✔ 1.6 What progress has been realized from API’s support for the implementation of the Digital Data Governance (DDG) framework? ✔ ✔ ✔ ✔ 1.7 How has COVID-19 pandemic impacted API implementation? ✔ ✔ ✔ ✔ 1.8 What coping strategies has API applied to mitigate the challenges from COVID-19 pandemic? ✔ ✔ ✔ ✔ 1.9 What other challenges has API experienced with the activity implementation, and how has API coped? (identifying needs, technical assistance to ASEAN and AMS challenges etc.) ✔ ✔ ✔ ✔ AEQ2 In what ways did the ASEAN endorsement process of API impact the pace of project implementation? What interventions could or could not be undertaken while awaiting the endorsement process? 2.1 What were the experiences of API regarding the project endorsement process by ASEAN endorsement? a. What process was followed? b. What are your opinions about the endorsement process? c. How effective was the endorsement process? Give concrete reasons and examples. ✔ ✔ ✔ 2.2 How did the endorsement process impact the implementation of API activities? a. Probe for positive and negative impact b. What interventions could/could not be undertaken while awaiting the endorsement process? ✔ ✔ ✔ ✔ 2.3 What support did USAID/ ASEAN provide in support of API endorsement? a. Was that support effective in obtaining API endorsement? b. If yes/no, how/why? ✔ ✔ ✔ 2.4 How has the endorsement of API by ASEAN been beneficial in supporting the implementation of API activities? ✔ ✔ ✔ ✔ AEQ3 How should USAID/ASEAN adjust API’s theory model (development hypothesis, theory of change, results framework, or logic model), why? 3.1 Can you please share your perspectives about API Theory of Change (TOC)? a. Does it relate well to the design and implementation needs? How? (Relevance) ✔ ✔ FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 121 b. How effective is it in guiding the design and implementation of API activities? (Effectiveness/viability) c. What gaps, if any, exist in API TOC? How should API TOC be improved? (what should be removed, added or modified)? (Adequacy) d. What challenges has API experienced with applying the TOC to develop and implement interventions? (Viability/effectiveness) e. What has API done to address those challenges? f. What are the key lessons learnt from API’s theory of change? 3.2 Consistent with the API TOC, what has API done to continually a. Identify key opportunities through research and consultation b. Refine API approaches to those opportunities based on gathered evidence ✔ ✔ 3.3 What are the main achievements/results of API support to ASEAN regarding the implementation of policies under AEC? ASEAN Single Window (ASW) and Digital Data Governance (DDG), among others? ✔ ✔ ✔ ✔ 3.4 What critical factors should (1) USAID, and (2) API address in their TOC to strengthen project design, implementation and results? Probe for (something related to external and internal factors) ✔ ✔ ✔ 3.5 What additional indicators should USAID use to measure the results of API besides the existing indicators? ✔ ✔ ✔ AEQ4 What lessons have been learned about program elements (interventions or set of interventions) are most effective in the API design? And which program elements are least impactful? 4.1 What are the key lessons learnt from API implementation with regard to: a. The extent and value of aligning API strategies and interventions to ASEAN needs b. Best practices/strategies for enhancing policy development capacity building and results? c. Best practices for providing technical support to ASEAN and AMS d. Types and value of USAID/ASEAN support in the implementation of API performance e. API TOC and Result Framework ✔ ✔ ✔ ✔ 122 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION IGNITE DATA COLLECTION TOOL Interviewee’s Title / Position Organization (ASEC/Sectoral Body/Entity/Others (please specify) Level ☐Regional ☐National Country Gender Support Area Date # Interview Questions Responding Participants’ Institution USAID IGNITE IP ASEC Divisions IEQ1 In what ways is IGNITE helping narrow the development gap in ASEAN's EG sector? Which type of USAID ASEAN interventions under trade facilitation and digital economy group are most effective in meeting ASEAN's needs (provide examples)? And why? 1.1 What interventions is IGNITE currently undertaking to narrow the development gap in EG? Probe for: Trade facilitation a. ASEAN Single Windows (ASW) b. Logistics c. Harmonization of standards Digital Economy a. Regulatory reform for the digital economy b. Improved regulations for cybersecurity and data privacy c. Expanded broadband access ✔ ✔ 1.2 How has the implementation of the ASEAN Trade Facilitation Framework and the World Trade Organization (WTO) Trade Facilitation Agreement affected the time and cost of trade? a. Which aspects of trade were impacted by the framework and how? b. If it decreased the time, by how much? c. If it did not decrease, why? ✔ 1.3 Which interventions under the trade facilitation are most effective? Probe for: a. reduce trade cost for private firms b. support financial sector integration c. reduce logistical bottlenecks d. harmonizing standards ✔ ✔ ✔ 1.4 In your opinion, which interventions under trade facilitation are not effective? Why? ✔ ✔ ✔ 1.5 Which interventions under the digital economy are most effective in helping AMS to achieve: a. support e-commerce by SMEs, including digital finance b. strengthen cyber security and data privacy ✔ ✔ ✔ FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 123 c. making the internet more affordable and accessible Why? Please give specific examples. 1.6 In your opinion, which interventions under the digital economy are not effective? Why? ✔ ✔ ✔ IEQ2 Which type of IGNITE interventions are most effective in meeting ASEAN's needs and achieving deliverables in areas of shared priority (provide examples)? And why? 2.1 What are the main needs of ASEAN to meet expected results in the areas of shared priority? a. Science, Technology, and Innovation b. Gender c. MSMEs ✔ ✔ ✔ 2.2 What interventions does IGNITE implement to address those needs? ✔ ✔ ✔ 2.3 What has IGNITE done under Science, Technology, and Innovation (STI) to support the realization of cross cutting results? Probe for a. Enhancing the human capital in science, technology, engineering, and mathematics (STEM) b. Increased the mobility of regional research talent and collaboration and/or research talent mobility in STEM c. Increased women’s contribution and participation in STEM d. Increased SME access to finance, technology and e￾commerce ✔ ✔ ✔ 2.4 How effective is it in meeting ASEAN needs? Give specific examples in each intervention ✔ ✔ ✔ 2.5 In your opinion, which interventions are not effective? Why? Give specific examples. ✔ ✔ ✔ 2.6 Which interventions under gender and SMEs are most effective to increase the prosperity of women and small businesses in ASEAN? How? Which are not and why? ✔ ✔ ✔ IEQ3 How effective is IGNITE’s theory model in guiding the implementation of project activities towards achieving expected results (development hypothesis, theory of change, results framework, or logic model)? Why or why not? 3.1 Can you please share your perspectives about IGNITE Theory of Change (TOC): a. Does it relate well to the design and implementation needs? How? (Relevance) b. How effective is it in guiding the design and implementation of IGNITE activities? (Effectiveness/viability) c. What gaps, if any, exist in IGNITE TOC? How should IGNITE TOC be improved? (What should be removed, added or modified)? (Adequacy) d. What challenges has IGNITE experienced with applying the TOC to develop and implement interventions? (Viability/effectiveness) ✔ ✔ 124 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION e. What has IGNITE done to address those challenges? f. What are the key lessons learnt from IGNITE’s theory of change? 3.2 Consistent with the TOC, what has IGNITE done to improve ASEAN institutional capacity to: a. Mobilize sufficient political will to advance ASEAN priorities b. Reduce the time and cost of trade, particularly in key growth sectors (e.g. digital services and science, technology, and innovation) ✔ 3.3 What are the main achievements/results of IGNITE in helping ASEAN to achieve accelerated and inclusive growth among AMS? a. How has IGNITE support contributed to benefitting AMS consumers, firms and US exporters and workers? Give specific examples. (Probe for Outcome) ✔ ✔ 3.4 What critical factors (both from ASEAN and USAID internal and external sides) should (1) USAID, and (2) IGNITE address in their TOC to strengthen project design, implementation and results? ✔ ✔ 3.5 What additional indicators should USAID use to measure the results of IGNITE besides the existing indicators? ✔ IEQ4 In what ways has USAID/ ASEAN contributed to inclusive growth (provide examples)? What are the challenges and potential support provided by USAID to improve the results? 4.1 How has USAID/ASEAN supported the inclusive growth priorities of ASEAN? ✔ ✔ ✔ 4.2 What kind of support provided by USAID/ASEAN is most effective in facilitating inclusive growth in ASEAN in the areas of Innovation, Trade and E-Commerce? ✔ ✔ ✔ 4.3 How has USAID/ASEAN supported IGNITE with the implementation of project activities? Probe for support with diplomatic and administrative engagement a. Was that support effective? How? ✔ ✔ ✔ 4.4 What additional actions/support could USAID/ASEAN provide to (1) ASEAN, (2) IGNITE in their efforts to achieve inclusive growth in the AMS? a. What difference could be anticipated if USAID/ASEAN provided this additional support (Probe for difference in implementation, effectiveness and results)? ✔ ✔ ✔ 4.5 How has COVID-19 impacted IGNITE implementation? a. What coping strategies has IGNITE undertaken to mitigate the challenges posed by COVID-19? ✔ ✔ ✔ 4.6 What other challenges has IGNITE faced and how has it coped with those challenges? ✔ ✔ ✔ 4.7 What lessons have been learnt from the design and implementation of IGNITE? ✔ ✔ ✔ FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 125 ANNEX V: SOURCES OF INFORMATION LIST OF DOCUMENTS AND DATA SOURCE PROSPECT 1. ASEAN GMSF_Endorsed 2. ASEAN-USAID PROSPECT-final SOW 3. ASEAN-WPS-Study-01082021 4. PROSPECT AMEL Plan_12apr21CLEAN approved 5. PROSPECT AR 2019_30jan20 6. PROSPECT Annual Report_2020_29jan21_FINAL 7. PROSPECT_QR_Apr-Jun21_29july21 8. PROSPECT_QR_Jan-Mar21_17apr21 9. PROSPECT_QR12_Jul-Sept21_29oct21 10. Synthesis Report_20nov20 11. ToT Program to Address Disinformation and Promote Media Literacy_FINAL_2021 10 25 12. USAID PROSPECT CTIP Fact Sheet 2021 (external) 13. USAID PROSPECT Flash – CVE Bali WP Trifold.2021 (external) 14. USAID PROSPECT GM-WPS Fact Sheet 2021 (external) 15. USAID PROSPECT Policy Continuum CTIP.2021 (external) 16. USAID PROSPECT Policy Continuum CVE.2021 (external) 17. USAID PROSPECT Policy Continuum. GM-WPS.2021 (external) 18. USAID PROSPECT Policy Continuum.CWT.2021(external) API 1. 2020-06-25 Approved API AMEL Plan 2. 2020-10-30 API Annual Report FY2020 Draft 3. 2020-11-06 Summary of Discussion on the 2nd Knowledge Session v01 RR 4. 2021-01-13 ACDD Data Modelling Workshop 5. 2021-01-15 Data Modelling Workshop Highlights v02 6. 2021-07-22 Summary of Discussion on the 4th Knowledge Session v01 MA ET MA 7. 2021-07-26 National Consultant Report-Singapore FINAL 8. 2021-08-02 National Consultant Report-Indonesia FINAL 9. 2021-08-13 National Consultant Report-Vietnam FINAL 10. 2021-08-30 National Consultant Report-Brunei FINAL 11. 2021-08-31 National Consultant Report-Malaysia FINAL 12. 2021-09-10 National Consultant Report-Cambodia FINAL 13. 2021-09-10 National Consultant Report-Philippines FINAL 14. ASEAN Policy Implementation Phase 1 RFTOP Section C IGNITE 1. ASEAN-USAID IGNITE 1st Annual Report 2. IGNITE-RFP Containing SOO in Attachment 1 3. IGNITE Contract Containing PWS Reflecting SOW in Section C 4. IGNITE-21-017 AMEL Plan Version 3.1 (Submitted 8.27.21) 5. IGNITE-21R-02 2nd Annual Report 126 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION RESULT FRAMEWORKS PROSPECT RESULT FRAMEWORK ASEAN DEVELOPMENT COOPERATION GOAL: ASEAN and its Member States contribute to more sustainable and inclusive growth and encourage rules-based systems to support a prosperous Southeast Asia through partnership with USAID ASEAN DEVELOPMENT OBJECTIVE (DO1) / PROSPECT GOAL 1 (Component 1): Building ASEAN and AMS capacity to resolve regional and global challenges PROSPECT Work Stream 1: Non-Traditional Security PROSPECT Inputs: ● Provide technical support for improved cross-sectoral coordination on the implementation of the ACTIP and APA ● Support ASEAN’s efforts to counter violent extremism, including supporting the multi-sectoral coordination ● Provide technical capacity building to ASEAN to address non-traditional security threats such as cybercrime, wildlife trafficking, and other potential areas ● Strengthen ASEAN capacity to respond to emerging humanitarian crises Contribute to indicators: #1. Number of participants taking part in USG supported knowledge platforms in the Asia region #2. Number of women and youth with increased knowledge, leadership capabilities, and technical skills as a result of USG assistance #2.1. Number of women who participate in U.S. funded training for foreign nationals. #3. Number of youth trained in soft-skills/life skills through USG assisted programs #4. Number of U.S.-supported workshops and regional forums promoting greater social protections and human rights advocacy #4.1. Number of USG-supported activities designed to promote or strengthen the civic participation of women #5. Number of U.S.-supported events, workshops, or dialogues including participation of ASEAN and U.S. Government officials to strengthen mutual cooperation #7. Number of tools, laws, policies, and/or procedures strengthened and/or developed that promote ASEAN integration, including civil society engagement in decision-making; social protections; and gender inclusion #8. Number of CSOs and private sector contributing to ASEAN policies, procedures, and/or programs as a result of PROSPECT activities #9 Number of human rights defenders trained and supported (IPS-IPTI G5); (F Indicator DR.6.1-2) ASEAN DEVELOPMENT OBJECTIVE (DO3) /PROSPECT Goal 2 (Component 2): Support ASEAN and its Member States in advancing effective regional integration to enhance the rule of law and uphold human rights PROSPECT Work Stream 2 Responsive and Transparent Governance PROSPECT Work Stream 3 Human Rights and Opportunities for Marginalized and Vulnerable Groups PROSPECT Inputs: ● Build capacity of and strengthen linkages between the ASEAN Secretariat, CACJ, and AIPA to promote rule of law in ASEAN ● Support the process of transforming ASEAN into a rules￾based organization, including improving implementation of agreed upon ASEAN instruments PROSPECT Inputs: ● Support the AICHR and other regional actors to promote, protect, and mainstream human rights issues in ASEAN ● Engage youth to more frequently contribute to ASEAN processes and strengthen shared ASEAN identity ● Expand the rights and opportunities of women and other vulnerable and marginalized groups FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 127 ● Cultivate and strengthen civil society networks to support existing platforms and create new ones for effective engagement with ASEAN on critical issues Contribute to indicators: #1. Number of participants taking part in USG supported knowledge platforms in the Asia region #2. Number of women and youth with increased knowledge, leadership capabilities, and technical skills as a result of USG assistance #3. Number of youth trained in soft-skills/life skills through USG assisted programs #4. Number of U.S.-supported workshops and regional forums promoting greater social protections and human rights advocacy #4.1. Number of USG-supported activities designed to promote or strengthen the civic participation of women #5. Number of U.S.-supported events, workshops, or dialogues including participation of ASEAN and U.S. Government officials to strengthen mutual cooperation #7. Number of tools, laws, policies, and/or procedures strengthened and/or developed that promote ASEAN integration, including civil society engagement in decision￾making; social protections; and gender inclusion #8. Number of CSOs and private sector contributing to ASEAN policies, procedures, and/or programs as a result of PROSPECT activities Contribute to indicators: 1. Number of participants taking part in USG supported knowledge platforms in the Asia region #2. Number of women and youth with increased knowledge, leadership capabilities, and technical skills as a result of USG assistance #2.1. Number of women who participate in U.S. funded training for foreign nationals #3. Number of youth trained in soft-skills/life skills through USG assisted programs #4. Number of U.S.-supported workshops and regional forums promoting greater social protections and human rights advocacy #4.1. Number of USG-supported activities designed to promote or strengthen the civic participation of women #5. Number of U.S.-supported events, workshops, or dialogues including participation of ASEAN and U.S. Government officials to strengthen mutual cooperation #6. Number of USG funded workshops, seminars and/or trainings targeting ASEAN youth to raise awareness of ASEAN and/or strengthen a shared ASEAN identity #7. Number of tools, laws, policies, and/or procedures strengthened and/or developed that promote ASEAN integration, including civil society engagement in decision￾making; social protections; and gender inclusion #8. Number of CSOs and private sector contributing to ASEAN policies, procedures, and/or programs as a result of PROSPECT activities #9 Number of human rights defenders trained and supported (IPS-IPTI G5); (F Indicator DR.6.1-2) 128 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION API RESULT FRAMEWORK FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 129 IGNITE RESULT FRAMEWORK 130 | FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION KEY OBJECTIVES AND FOCUS AREA IGNITE KEY OBJECTIVES FINAL REPORT OF USAID/ASEAN JOINT MIDTERM PERFORMANCE EVALUATION | 131 API FUTURE SCOPE OF WORK FOCUS AREAS AND ITS RELATION TO PROSPECT AND IGNITE U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523