RETROSPECTIVE PERFORMANCE EVALUATION UNITED STATES ² ASEAN PROGRESS AND ACTI PROJECTS With a Focus on Select Policy Areas Contracted Under AID-497-C-16-00006 Indonesia Monitoring & Evaluation Support Project JUNE 2020 Prepared by: Dr. Channa Gunawardena Dr. Rona Chandran Dian Anshar Shinta Manurung i | RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV TABLE OF CONTENTS List of Figures..................................................................................................................................... ii List of Tables...................................................................................................................................... ii Abbreviations.................................................................................................................................... iii Executive Summary.......................................................................................................................... vi 1. Introduction................................................................................................................................... 1 1.1. Project Background............................................................................................................... 1 1.2. Evaluation Purpose and Audience....................................................................................... 2 1.3. Evaluation Scope and Questions......................................................................................... 3 2. Evaluation Methodology............................................................................................................... 3 2.1 Overall Approach................................................................................................................... 3 2.2 Evaluation Team...................................................................................................................... 4 2.3. Data Sources.......................................................................................................................... 5 2.4. Key Informants....................................................................................................................... 5 2.3. Data Analysis.......................................................................................................................... 6 2.4. Limitations.............................................................................................................................. 7 3. Findings, Conclusions and Recommendations.......................................................................... 8 3.1. Evaluation Question 1: What was the Development Path of the Targeted Policy Areas, from the Inception of the Policy at the ASEAN Level Through to its Implementation at the AMS Level?............................................................................................. 8 3.1.1. Findings.......................................................................................................................... 8 3.1.2. Conclusions.................................................................................................................17 3.1.3 Recommendations.......................................................................................................20 3.2. Evaluation Question 2: How did USAID Assistance Contribute to the Development of These Policies?........................................................................................................................21 3.2.1 Findings.........................................................................................................................21 3.2.2 Conclusions..................................................................................................................25 3.2.3 Recommendations.......................................................................................................25 3.3. Evaluation Question 3: Which Forms of Assistance Contributed the Most to the Development of These Policies? Which Contributed the Least?........................................26 3.3.1 Findings.........................................................................................................................26 3.3.2 Conclusions..................................................................................................................35 3.3.3 Recommendations.......................................................................................................36 3.4. Evaluation Question 4: What Role did Gender Play in the Design and Implementation of Activities? What Impact did This Role Have?....................................................................36 3.4.1. Findings........................................................................................................................37 USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | ii 3.4.2. Gender in the ACTI Project.....................................................................................42 3.4.3. Conclusions.................................................................................................................43 3.4.4. Recommendations......................................................................................................44 3.5. Evaluation Question 5: What Lessons Learned in Strategy and Approach can and should be Applied to Future Programming? ...........................................................................45 3.5.1 Findings.........................................................................................................................45 3.5.2 Recommendations for Future Programming...........................................................46 Annex 1. Scope of Work...............................................................................................................53 Annex 2. U.S.-ACTI Framework ..................................................................................................61 Annex 3. U.S.-PROGRESS Framework........................................................................................62 Annex 4. Evaluation Schedule .......................................................................................................63 Annex 5. Key Informant List .........................................................................................................64 Annex 6. Key Informant Interview Questions............................................................................67 Annex 7. FGD Tool ........................................................................................................................69 Annex 8. Enhanced Policy Continuum.........................................................................................70 LIST OF FIGURES Figure 1. Informants by Type of Organization................................................................................. 6 Figure 2. Informants by Type of Support Area................................................................................ 6 Figure 3. ASEAN Policy Development Path................................................................................... 18 Figure 4. ASEAN Policy Instruments............................................................................................... 20 Figure 5. Policy Development Path for Gender............................................................................ 38 LIST OF TABLES Table 1. Scope of Evaluation............................................................................................................... 3 Table 2. Summary of AHTN Policy Development Path................................................................. 8 Table 3. Summary of AMDD Policy Development Path ................................................................ 9 Table 4. Summary of TMHS Policy Development Path................................................................ 10 Table 5. Summary of Services Policy Development Path............................................................. 12 Table 6. Summary of SME Policy Development Path ................................................................... 13 Table 7. Summary of WEE Policy Development Path.................................................................. 14 Table 8. Summary of TIP Policy Development Path..................................................................... 15 Table 9. Summary of Youth Policy Development Path ................................................................ 16 Table 10. Summary of the Policy Development Path................................................................... 72 iii | RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV ABBREVIATIONS ABAC ASEAN Business Advisory Council ACCMSME ASEAN Coordinating Committee on MSME ACCSQ ASEAN Consultative Committee on Standard and Quality ACTI U.S.-ASEAN Connectivity through Trade and Investment ACTIP ASEAN Convention Against Trafficking in Persons, Especially Women and Children ACW ASEAN Committee on Women ACWC ASEAN Commission on the Promotion and Protection of the Rights of Women and Children ACWO ASEAN Confederation of Women·s Organizations AEC ASEAN Economic Community AEM ASEAN Economic Ministers AFAS ASEAN Framework Agreement on Services AFMAM ASEAN Foundation Model ASEAN Meeting AICHR ASEAN Intergovernmental Commission on Human Rights AHTN ASEAN Harmonized Tariff Nomenclature APSC ASEAN Political-Security Community AMDD ASEAN Medical Device Directive AMMSWD ASEAN Ministerial Meeting on Social Welfare and Development AMS ASEAN Member States AMWW ASEAN Ministerial Meeting on Women AMMY ASEAN Ministerial Meeting on Youth ASCC ASEAN Socio-cultural Community ASEAN Association of Southeast Asian Nations ASEC ASEAN Secretariat ASW ASEAN Single Window USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | iv ASY ASEAN Sub-Committee on Youth ASW ASEAN Sub-Committee on Women AWEN ASEAN Women Entrepreneurs· Network AYVP ASEAN Youth Volunteer Programme CEDAW Convention on the Elimination of All Forms of Discrimination Against Women CCS Coordinating Committee on Services DOC Department of Commerce ECF Equity Crowd Funding EU European Union FGD Focus Group Discussions GWAI Great Women in ASEAN Initiative GMP Good Manufacturing Practices HS Harmonized System (Harmonized Commodity Description and Coding System) IGNITE ASEAN-USAID Inclusive Growth in ASEAN Through Innovation, Trade and Ecommerce ICT Information and Communication Technologies KIIs Key Informant Interviews KPIs Key Performance Indicators MDPWG Medical Devices Product Working Group M&E Monitoring and Evaluation MSMEs Micro, Small and Medium Enterprises OECD Organization for Economic Cooperation and Development PWG Product Working Group PROGRESS ASEAN-U.S. Partnership for Good Governance, Equitable and Sustainable Development and Security PROSPECT ASEAN-U.S. Partnership for Regional Optimization with the Political-Security and Socio-Cultural Communities SAPSMED Strategic Action Plan for SME Development v | RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV SEOM Senior Economic Officials Meeting SME Small and Medium-Sized Enterprise SMEWG Small and Medium-Sized Enterprises Working Group SOP Standard Operating Procedures SOW Statement of Work SOMTC Senior Officials Meeting on Transnational Crime SOMY Senior Officials Meeting on Youth RPN Regional Policy Framework TF Task Force TIP Trafficking in Persons TMHS Traditional Medicine and Health and Supplements TOR Terms of Reference TOT Training of Trainers US The United States USAID United States Agency for International Development USTR U.S. Trade Representative WCO World Customs Organization WEE Women·s Economic Empowerment WG Working Group USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | vi EXECUTIVE SUMMARY EVALUATION SCOPE AND PURPOSE The United States (U.S.) and the Association of Southeast Asian Nations (ASEAN)1 have engaged in a cooperative partnership since 1977. The purpose of this final retrospective performance evaluation of the U.S.-ASEAN Connectivity through Trade and Investment (ACTI) and the ASEAN-U.S. Partnership for Good Governance, Equitable and Sustainable Development and Security (PROGRESS) projects was to: 1. Determine the degree to which ACTI and PROGRESS contributed to the development of ASEAN policies, why they did or did not do so, and how they contributed (i.e., factors influencing the achievement of results and challenges faced); and 2. Capture lessons learned around the development of key policies, including strategic recommendations, that may be applied to current and future USAID programming. The findings of this evaluation are intended to inform current USAID/ASEAN activities and to guide future program design. The main audience for this evaluation is USAID. The evaluation focused on four priority areas, including trade and investment facilitation and SME development for ACTI, and trafficking in persons (TIP) and youth engagement2 for PROGRESS. EVALUATION DESIGN AND METHODS The primary data used for the evaluation were qualitative and derived from key informant interviews (KIIs) and focus group discussions (FGDs) with representatives from USAID, U.S. government partners, implementing partners, ASEAN Secretariat (ASEC), country representatives from the ASEAN sectoral bodies, ASEAN entities, and youth beneficiaries. Interviews involved representatives from nine out of the 10 ASEAN Member States (AMS). A substantial review of documents was also undertaken. SUMMARY OF FINDINGS, CONCLUSIONS AND RECOMMENDATIONS Evaluation Question 1: What was the development path of the targeted policy areas, from the inception of the policy at the ASEAN level through its implementation at the AMS level? Findings The development path of ASEAN policy is driven by principles of consultation and consensus. The major factor driving the inception stage is high-level political commitment from ASEAN Leaders or ministerial bodies. These lead to the establishment of institutional arrangements including working groups (WGs), such as the Traditional Medicine and Health and Supplements (TMHS) product WG, and the Expert Group Meeting on the ASEAN Convention Against Trafficking in Persons, Especially 1 Brunei, Cambodia, Indonesia, Lao PDR, Malaysia, Myanmar, Singapore, the Philippines, Thailand and Vietnam. 2 There is no standard definition for youth in ASEAN. Among member states the age range for youth varies from between 15 to 35. vii | RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV Women and Children (ACTIP). During the development stage, for policy instruments in the ASEAN Economic Community (AEC) pillar, drafting was a technical process driven by appreciation of best practices, and a common understanding of issues and opportunities among the sectoral body members. In sensitive areas such as TIP, the drafting process was driven by political commitments. For policy instruments that do not have legal provisions, such as in youth engagement and SME development, development focuses on the drafting of work plans or strategic action plans. Following endorsement and signing of policy instruments, ratification takes place by the AMS for policy instruments with legal provisions, such as the ASEAN Medical Device Directive (AMDD) and the ACTIP. There is a finite period within which AMS should undertake ratification. Policies that do not have legal provisions, such as in SME development, youth and women·s empowerment, do not require ratification by the AMS. The main factors driving regional and national implementation stages, include resources from the AMS, and technical capacity to interpret and implement the ASEAN policy instruments. For policy instruments with legal provisions, such as AMDD, ACTIP and ASEAN Trade in Services Agreement (ATISA), implementation is based on the internal schedules of each AMS. Conclusions The development path for policy in ASEAN can be conceptualized in terms of the USAID/ASEAN Policy Continuum·s 11 distinct stages. The development of the stages outlined below are based on an analysis of ASEAN policies reviewed by this study. x Stage 1: Inception is catalyzed by political commitment, with instruments such as Community Blueprints providing the framework or starting principles for the path. x Stage 2: Institutional arrangements include the establishment of WGs and ToRs. x Stage 3: Definition includes prioritization of the target areas and task distribution. x Stage 4: Drafting of policy instruments focuses on the development of supporting text. x Stage 5: Review of draft policy instruments by the AMS and drafting of policy text. x Stage 6: Policy instrument finalization and approval by AMS at the WG level. x Stage 7: Endorsement by ASEAN sectoral bodies and signing by Ministers or Leaders. x Stage 8: Ratification by AMS of policy instruments with legal provisions. x Stage 9: Implementation at the regional level, including data collection and analysis. x Stage 10: Implementation in AMS, including training for government and business. x Stage 11: Monitoring and reporting, including compliance and outcome monitoring. Recommendations It is recommended that USAID/ASEAN considers revising its policy development continuum tool to better reflect the stages and details presented in the 11-stage framework in this report. The framework can serve to guide and help sequence future USAID/ASEAN policy support. The recognition of a range of ASEAN policy measures or instruments by USAID/ASEAN is important to enable the rationalization and targeting of support and can be used for monitoring and reporting on progress. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | viii Evaluation Question 2: How did USAID assistance contribute to the development of these policies? Findings Most USAID assistance provided through ACTI and PROGRESS was in response to demand-driven requests from ASEAN stakeholders. Much of the support provided was at stage 4 of the policy continuum (drafting of policy instruments), and regional implementation (stage 9). USAID strengthened the understanding and awareness of ASEAN stakeholders on international best practices and standards, leading to enhancements in draft policy instruments and the ability of stakeholders to review the policy instruments in relation to international standards. USAID adopted a strong participatory approach involving a range of stakeholders. The regional efforts increased the capacity of the sectoral body members from the AMS to develop and review the policies supported. USAID is perceived as a preferred and reliable dialogue partner in many support areas. Factors that contributed to this preference included responsiveness to ASEAN needs, the ability to provide credible and neutral experts, adoption of strong participatory approaches and the provision of long-term support. Conclusions USAID contributed to the development and implementation of ASEAN policy through capacity building, adoption of international best practices and participatory approaches. These contributions led to enhanced ASEAN policies and strengthened technical skills for policy review and implementation. USAID is perceived as a preferred dialogue partner, as it responded to ASEAN needs and provided long-term support Recommendations Support targeting stages 2 and 3 of the continuum should be demand-driven and sensitized to meet the needs of ASEAN stakeholders. As much of the support to-date has focused on stages 4 (drafting) and 9 (regional implementation), support should increasingly focus on capacity building for stage 10 (national implementation) and 11 (monitoring and reporting). Continued policy support by USAID is important for the Agency to be perceived as a reliable and strategic ASEAN dialogue partner. Evaluation Question 3: Which forms of assistance contributed the most to the development of these policies? Which contributed the least? Findings USAID·s provision of neutral and credible experts to facilitate dialogue among ASEAN stakeholders contributed across all areas. In AMDD, TMHS, AHTN, and services, which have instruments with legal provisions, USAID assistance that contributed the most included support for ASEAN meetings, and participatory approaches including cross-sectoral events, studies and capacity-building, all of which contributed to common understanding in developing and implementing policy instruments. In TIP, guidelines developed by USAID helped to address the handling of victims. In SME development and youth, which do not have legal provisions at the ASEAN level, USAID contributed to the development of supporting policy instruments through experts, and support for ASEAN meetings. For Women·s Economic Empowerment (WEE), USAID contributed capacity building, support for events and meetings, studies and inputs to strategy. Most USAID assistance contributed to the ix | RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV respective areas due to the close involvement of ASEAN stakeholders in designing activities. In terms of non-USAID assistance, the support by lead AMS to initiate ASEAN collaboration in specific sub-areas, such as TIP, TMHS, services, SME development, and WEE, were major contributions. Conclusions USAID assistance that brought together stakeholders to discuss and arrive at a common understanding, with facilitation provided by neutral experts, contributed the most across all areas. USAID support for ASEAN meetings, cross-sectoral events, studies and capacity-building efforts, contributed to both regional and national capacity building. For non-USAID assistance, the support by lead AMS to initiate collaboration contributed the most. Most USAID assistance contributed significantly to policy as the assistance was designed closely with ASEAN stakeholders. Recommendations Additional USAID/ASEAN assistance should focus on the development of important supporting policy instruments, such as dissemination mechanisms, and strengthening the monitoring of ASEAN policy initiatives as recommended by informants (refer EQ5). Evaluation Question 4: What role did gender play in the design and implementation of activities? What impact did this have? Findings Studies indicate that gender inequalities exist across ASEAN3 despite several policy instruments and institutional arrangements in place at regional and AMS levels. Gender mainstreaming in ASEAN is a challenge due to cross-sectoral coordination. However, gender played a significant role in TIP activities of PROGRESS in collaboration with the ASEAN Commission on the Promotion and Protection of the Rights of Women and Children (ACWC). This led to gender mainstreaming strategies for TIP, for example development of guidelines to manage women and child victims of TIP. In youth, efforts focused on supporting balanced gender representation. Gender efforts by ACTI were challenging, as its activities targeted the AEC pillar. The role of gender in the design and implementation of activities focused on encouraging women participants and trainers at workshops and events. Significant contributions were made by ACTI toward WEE. This led to the launch and operationalization of the ASEAN Women Entrepreneurs· Network (AWEN), capacity building of women entrepreneurs, and helped to create visibility for AWEN. Studies indicate that the establishment of AWEN is an important milestone in advancing WEE in the region4. 3 Projected Gender Impact of the ASEAN Economic Community, 2016, ASEAN Secretariat; ASEAN Support Project Appraisal: Gender Analysis, December 2017, USAID/MESP project, MSI Inc. 4 Strengthening Women·s Entrepreneurship in ASEAN: toward increasing women·s participation in economic activity, September 2017, OECD. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | x Conclusions At a project level, the role of a gender approach aimed to ensure balanced gender representation among participants and trainers at workshops, training events and meetings. Gender-mainstreaming approaches were also applied by supporting initiatives in WEE and TIP. These approaches achieved notable success and provide examples for ASEAN of cross-cutting gender-mainstreaming. Recommendations Specific USAID/ASEAN support could include the establishment of a WG on WEE, involving the ASEAN Coordinating Committee on MSME (ACCMSME) and AWEN, and facilitating policy dialogue events. Building on its track record in standards, USAID/ASEAN should facilitate collaboration between the ASEAN Consultative Committee on Standards and Quality (ACCSQ) and AWEN to harmonize standards for women-majority sub-sectors, such as coffee and textiles. Capacity-building should continue for AWEN in key areas such as Industry 4.0, digitalization and access to financial instruments such as crowd funding. Programming should consider facilitation for the collection of gender-disaggregated data in ASEAN in areas such as MSMEs and selected sub-sectors in services. Capacity-building on gender mainstreaming in all USAID/ASEAN programs is recommended. Evaluation Question 5: What lessons learned in strategy and approach can and should be applied to future programming? Findings A major strength of USAID was its ability to rapidly respond to demand-driven requests from ASEC and ASEAN sectoral bodies. Given ASEAN·s significant progress in developing core policy instruments in key areas, much of the attention will shift toward policy implementation in the AMS. Although the youth sector does not have ASEAN policy instruments with legal provisions, it nonetheless contains policy instruments such as five-year work plans, and the ASEAN Youth Development Index. Chapter 3 discusses examples where student beneficiaries have supported policy advocacy in their home countries through student projects initiated with USAID. USAID should consider supporting youth activities in support of priority policy areas. Recommendations Ownership of expert inputs can be enhanced by involving ASEC and the lead countries or chair of ASEAN sectoral bodies (if relevant) in the final approval of the experts. Programmatic lines of support should align closely with the specific sectoral body work plans or strategic action plans to ensure maximum ASEAN relevance. Targeted support for the AMS is required to ensure that USAID support provided for ASEAN policy development leads to successful national implementation (stage 10). This can be enhanced by having an ASEAN element in relevant USAID/bilateral support to help AMS fulfil their ASEAN commitments in each sector. Supporting tools can include the establishment of a USAID/ASEAN-bilateral mission coordination and monitoring mechanism. Support to strengthen monitoring and reporting (stage 11) by sectoral bodies would enable the assessment of policy effectiveness and identification of remedial action. The best entry point for USAID assistance to ASEAN remains ASEC, and specific capacity-building support to ASEC is recommended as part of a long-term, strategic approach. Other recommendations include greater use of cross-cutting measures, such as youth, gender and social governance across USAID/ASEAN. Further recommendations are provided in Chapter 3. 1 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV 1. INTRODUCTION The United States (U.S.) and the Association of Southeast Asian Nations (ASEAN)5 have worked together to build a cooperative partnership since 1977. The collaborations are carried out through U.S.-ASEAN Plans of Action, which are developed to identify priorities and necessary actions by both parties. In particular, the Plan of Action 2011-2015 had two objectives: to enhance and elevate the U.S.-ASEAN partnership to a strategic level in a broad-based and mutually beneficial manner; and to address emerging regional and global challenges over its five-year duration. USAID/ASEAN, based in Jakarta, builds the capacity of ASEAN and its Member States (AMS) to contribute to sustainable and inclusive growth, and encourage rules-based systems to support a prosperous Southeast Asia. U.S.-ASEAN Connectivity through Trade and Investment (ACTI) and the ASEAN-U.S. Partnership for Good Governance, Equitable and Sustainable Development and Security (PROGRESS) are two USAID/ASEAN projects that were implemented under the U.S.-ASEAN Plan of Action 2011-2015. ACTI provided support to accelerate selected components of the ASEAN Economic Community (AEC), while PROGRESS supported ASEAN·s Political-Security and Social￾Cultural Communities through targeted support to key ASEAN sectoral bodies. 1.1. PROJECT BACKGROUND USAID/ASEAN funded two projects: (i) ACTI, a $20.9 million project that ran from June 2013 to December 2018, which was implemented by Nathan Associates, Inc.; and (ii) PROGRESS, a $14.5 million project that ran from September 2013 to September 2018, which was implemented by DAI. The ACTI Project ACTI was organized around the following four work streams: 1. Technical and legal development of the ASEAN Single Window and outreach; 2. Trade and Investment Facilitation: Standards harmonization, trade transparency, services trade liberalization, and investment facilitation; 3. SME Development: with a special focus on resources for women and young entrepreneurs, and firms in Cambodia, Lao PDR, Myanmar, and Vietnam; and 4. ICTs for Development: Assistance to ASEAN in implementing its ICT Master Plan. The results framework for ACTI (Annex 2) was developed based on the ACTI contract·s Statement of Work (SOW) and priority outputs described in the annual reports (see Annex 2). The results framework focuses on the two priority work streams to be evaluated. This evaluation focused on ACTI Work Stream 2: Trade and Investment, and Work Stream 3: SME Development. ACTI worked with the following ASEAN stakeholders in these two work streams: the ASEAN Secretariat (ASEC) and sectoral bodies including the ASEAN Consultative Committee on Standard and Quality (ACCSQ), the Coordinating Committee on Services (CCS), Directors General of Customs, the ASEAN Coordinating Committee on MSME (ACCMSME), the ASEAN Women 5 Brunei, Cambodia, Indonesia, Lao PDR, Malaysia, Myanmar, Singapore, the Philippines, Thailand and Vietnam. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 2 Entrepreneurs· Network (AWEN), and the Great Women in ASEAN Initiative (GWAI). ACTI also worked other U.S. government agencies, the private sector, and other stakeholders. The PROGRESS Project PROGRESS focused on seven work streams, as follows: 1. Transnational crime and non-traditional security threats, with a focus on trafficking in persons (TIP); 2. Human rights; 3. Executive, judicial and legislative networks (including support to the ASEAN Chair); 4. Science-based policymaking; 5. Disaster risk reduction and management; 6. Women·s and children·s rights; and 7. Public outreach, youth engagement, and strengthening ASEAN institutions. The PROGRESS results framework (see Annex 3) was developed based on the PROGRESS SOW and priority outputs described in the annual reports. The results framework focuses on the two priority work streams to be evaluated. PROGRESS supported capacity building within ASEAN by working with the following stakeholders from the two evaluation priority work streams 1 and 7: ASEC, the ASEAN Intergovernmental Commission on Human Rights (AICHR), the ASEAN Commission on the Promotion and Protection of the Rights of Women and Children (ACWC), the Senior Officials Meeting on Transnational Crime (SOMTC), and the Senior Official Meeting on Youth (SOMY). 1.2. EVALUATION PURPOSE AND AUDIENCE The purpose of this retrospective performance evaluation of USAID·s ACTI and PROGRESS projects is to: 1. Determine the degree to which ACTI and PROGRESS contributed to the development of ASEAN policies, why or why not, and how (i.e., factors influencing the achievement of results and challenges faced); and 2. Capture lessons around the development of key policies, including strategic recommendations, that may be applied to current and future USAID programming. Findings and learning from the evaluation will inform USAID as the primary audience. Other audiences may include: U.S. government partners in design and programming of future ASEAN support; x ASEC; x ASEAN sectoral bodies, ASEAN sectoral committees and working groups, including representatives from AMS in designing and reviewing future USAID/ASEAN support; x Representatives to ASEAN sectoral committees/working groups from AMS and other relevant AMS stakeholders; x Implementing partners of USAID/ASEAN support; and x Possibly other ASEAN donors and development partners 3 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV 1.3. EVALUATION SCOPE AND QUESTIONS For the purpose of this evaluation, USAID/ASEAN selected specific policy areas under ACTI and PROGRESS for targeted evaluation, which are summarized in Table 1. Table 1. Scope of Evaluation Project Work Stream Focus ACTI Trade and facilitation ● ASEAN Harmonized Tariff Nomenclature (AHTN) ● ASEAN Medical Device Directive (AMDD) ● Traditional Medicine and Health and Supplement (TMHS) ● Trade in services ACTI SME development ASEAN SME Academy, US-ASEAN Business Alliance Women·s Economic Empowerment (WEE) PROGRESS Trafficking in persons ACTIP and supporting policy instruments PROGRESS Youth activities The evaluation focused on the following broad evaluation questions: 1. What was the development path of the four targeted policy areas, from the inception of the policy at the ASEAN level through its implementation at the Member States level, before the assistance of USAID? 2. How did USAID assistance contribute to the development of these policies? 3. What forms of assistance contributed the most to the development of these policies? Which contributed the least? 4. What role did gender play in the design and implementation of activities? What impact has this role had? 5. What are the lessons learned in strategy and approach that can and should be applied to future programming? USAID/ASEAN also provided a list of illustrative sub-questions that are included as Appendix 1 to the Evaluation SOW. 2. EVALUATION METHODOLOGY 2.1 OVERALL APPROACH MSI·s Evaluation Standard Operating Procedures (SOPs) were applied to this evaluation to ensure an effective start for the evaluation team and to focus on analytical issues from the outset. The SOPs identify the actions to be completed prior to the mobilization of the evaluation team, including a background literature review, the identification of key stakeholders, and the development of an evaluation design. The evaluation design includes a list of focus activities to be evaluated, budget allocations, and a geographic profile of the fieldwork. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 4 The overall evaluation approach of the team was guided by the following key principles: 1. Ensuring the externality and independence of the evaluation team. 2. Ensuring informants who provide feedback and information on the program remain anonymous. 3. Adoption of a mixed-methods and evidence-based findings approach for the answering of the evaluation questions in line with international best practices. This includes the use of multiple sources of data to answer the evaluation questions through a process of triangulation. 4. Integration of gender considerations in the evaluation process including gathering data and meta-data on male and female informants participating in the interviews. 5. A structured approach to the collection and analysis of primary qualitative data to ensure rigor and comparability of findings and consistency in approach. The evaluation commenced on October 17, 2019 and was completed on June 17, 2020. 2.2 EVALUATION TEAM The team comprised four independent evaluators with a diversity of relevant skills and experience. A short biograph of each team member follows. Dr. Chana Gunawardena - Team Leader and Subject Matter Specialist in Trade Facilitation and Economic Growth. Dr. Gunawardena coordinated efforts of the evaluation team and led the development of major outputs. Dr. Gunawardena has over 21 years of experience and 50 consultancy assignments in development cooperation. Much of his experience has focused on M&E, policy support and ICT for development in Asia. He has worked on several ASEAN assignments and has substantial knowledge of M&E procedures of the EU, the UN system, USAID, DFID, World Bank and ADB. Dr. Gunawardena has served as Team leader for several major evaluations, including for USAID. He holds a PhD in Management Sciences focused on management, monitoring and evaluation of development projects in Asia from Lancaster, UK. Dr. Rona Chandran - Evaluation methodologist/ Technical specialist. Dr. Chandran led the evaluation·s methodological design, managed the qualitative data collection and analysis process and provided technical expertise on youth and gender. She holds a PhD in social sciences and has over 18 years of experience in qualitative data analysis, research and policy analysis. Her expertise in M&E includes structured qualitative evaluation methods, design of data collection instruments, data gathering protocols, data management and qualitative data analysis, including prior M&E experience for USAID, EU and UK government. She is an experienced anthropologist with substantial experience in identifying and integrating socio-cultural values to enhance development. Dian Anshar - Trafficking in Persons (TIP) Specialist. Ms. Anshar played a lead role in the TIP area of the evaluation including data collection and development of findings. She has 14 years of experience with ASEAN including the ASEAN Secretariat, human rights groups, academia and research. She served as adviser to the Representative of Indonesia to AICHR and several ASEAN projects supported by the EU, Australia, and US in the area of TIP. She holds a Masters in Labour and Trade Union Studies. She is a Chevening scholar of the UK Foreign and Commonwealth Office and awardee of the Australia Award fellowship for Women, Peace and Security in Southeast Asia. Shinta Manurung ² Research Assistant. Ms. Shinta undertook research, scheduling of interviews and logistical coordination to support the evaluation. She has 25 years of working experience working as a project administrator, program management specialist and researcher. She served in a 5 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV liaison capacity, inter-facing with the ASEAN Secretariat and Indonesian Government on several projects including those by USAID. She has a B.A. in public relations. MESP Support Ms. Lis Nurhayati served as MESP Task Manager and supported the evaluation team in coordination and logistics. She supported the team in data collection in Thailand, Philippines and Indonesia and contributed to the final evaluation report including quality assurance. Mr. Peter Milne edited the evaluation report. Technical support and direction for the evaluation was provided by MESP Chief of Party Mr. David Callihan. Ms. Kusumastuti Soepardjo provided overall MESP personnel and management oversight. 2.3. DATA SOURCES Data sources included primary data and secondary data. The primary data were qualitative in nature and gathered from key informant interviews (KIIs) using a structured KII guide, and focus group discussions (FGDs) using a structured FGD guide. Quantitative primary data were not collected due to the specific qualitative focus of the evaluation questions. Secondary data were gathered by reviewing documents, including M&E plans, project progress and final reports, annual performance management progress reports (for ACTI), and data on performance indicator results and feedback data from project events (included in annual reports for PROGRESS), studies including white papers generated by ACTI and PROGRESS. A comprehensive set of ASEAN policy documents were reviewed including ASEAN Community Blueprints, declarations, sectoral agreements, legal protocols, technical annexes, training modules, sectoral work plans, sectoral strategic action plans and Terms of Reference for sectoral bodies. 2.4. KEY INFORMANTS Key informants for the evaluation priority work streams of ACTI and PROGRESS were identified, and arrangements made for the evaluation team to conduct KIIs (see Annex 5 for KII list) and FGDs. Substantial efforts were made to ensure that representatives from a range of AMS were engaged, including from the ´CLMVµ group of countries (Cambodia, Lao PDR, Myanmar and Vietnam), and the more advanced ASEAN-6 countries (Indonesia, Malaysia, the Philippines, Singapore and Thailand), to ensure a strong and credible ASEAN-wide perspective. For PROGRESS, a total of 30 KIIs were carried involving 40 informants, and one FGD was carried out involving three informants. For ACTI, a total of 43 KIIs were carried out involving 65 informants and two FGDS involving 20 informants. Figure 1 below provides a breakdown of informants according to type of organization. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 6 2.3. DATA ANALYSIS To ensure rigor and triangulation of findings, a structured approach to the analysis of data was used. This involved a first stage of analysis focusing on the analysis of each individual dataset and a second stage of analysis focusing on the triangulation of findings between datasets. Qualitative data analysis was conducted in two stages. First, data were formatted and imported into NVivo qualitative data management software, followed by coding, which included: topic coding, open coding, and axial coding. The second stage of analysis focused on triangulating the findings from the Figure 1. Informants by Type of Organization Figure 2. Informants by Type of Support Area 7 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV different data sources. The data sources for triangulation included different types of KII informants, data from FGDs, and secondary data from the documentation review. 2.4. LIMITATIONS This evaluation was complex due to its multi-project, multi-country and retrospective features. It is therefore natural for such a complex evaluation to involve several challenges and limitations. The main challenges and limitations are summarized below: 1. Multi-country nature: The multi-country nature of the evaluation posed challenges in terms of managing meetings and timing visits to each country based on the availability of key informants within a limited timeframe. The number of key informants that the evaluation team was able to interview for AHTN (three informants) and services (two informants) was less than ideal. The evaluation team reviewed the available documents related to these two areas to supplement data shortages. 2. Recall Bias: As ACTI and PROGRESS activities were launched in 2013 and finished in 2018, some informants may have found it difficult to remember specific project-related events, activities, or interventions. As a result, their opinions or comments may not necessarily be an accurate reflection of what actually occurred. 3. Halo Bias: There is a known tendency among informants to under-report socially undesirable answers and alter their responses to approximate what they perceive as the social norm (halo bias). The extent to which informants are prepared to reveal their true opinions may vary for some questions that call upon the informants to assess the performance of their colleagues or people on whom they depend for the provision of services. To mitigate this limitation, the evaluation team provided the informants with confidentiality and anonymity guarantees. FGDs were conducted among peer groups to encourage comfortable expression. 4. Translation: Most of KIIs and FGDs were conducted in English. However, two KIIs in Lao PDR were conducted in the local language, which was translated into English by a member of staff from the respective Ministry. The evaluation team tried to mitigate any translation risks by ensuring that the staff member conducting the translation understood the line of questioning. It should also be noted that the evaluation was not tasked with verifying project data as reported in project Annual Reports, Quarterly Progress Reports and PMP reports. The evaluation team assumed that (historic) figures recorded in these reports were accurate. The evaluation reviewed the project data-gathering processes and procedures to determine whether there were any anomalies that needed to be addressed to improve the veracity of the data gathered. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 8 3. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS 3.1. EVALUATION QUESTION 1: WHAT WAS THE DEVELOPMENT PATH OF THE TARGETED POLICY AREAS, FROM THE INCEPTION OF THE POLICY AT THE ASEAN LEVEL THROUGH TO ITS IMPLEMENTATION AT THE AMS LEVEL? 3.1.1. FINDINGS As many of the targeted policy areas go back over 10 years since their inception, the findings in this report are based on evidence provided by informants with substantial historical knowledge of each policy area, from both ASEC and the respective ASEAN sectoral bodies made up of representatives from the AMS. These findings have also been triangulated with a comprehensive set of ASEAN documentation, including ASEAN Community Blueprints, The ASEAN Charter, ToR for specific sectoral bodies, policy directives, legal protocols, strategic action plans and work plans. TARGETED POLICY AREAS UNDER ACTI PROJECT ASEAN HARMONISATION OF TARIFF NOMENCLATURE The policy development path for AHTN is summarized in the table below. Table 2. Summary of AHTN Policy Development Path Year Major policy development stages since inception Mid-1990s Deliberations among the AMS to develop an ASEAN harmonized commodity nomenclature that conforms to the international standards of the Harmonized System (HS) of the World Customs Organization (WCO) to facilitate trade in ASEAN 1997 First policy instrument developed: Article 4 in the ASEAN Agreement on Customs, which stipulates that the AMS will use a common tariff nomenclature at the 8-digit level 2003 Finalized and endorsed the definitions and the Protocol Governing the Implementation of the ASEAN Harmonised Tariff Nomenclature (AHTN) by the ASEAN Finance Ministers (AFM) Synthesis The development path of ASEAN policy is driven by the principles of consultation and consensus among the AMS. It can be conceptualized in terms of 11 stages along a policy continuum, according to the findings from the evaluation team. The major factor driving the progress during the initial stages is high-level political commitment originating from ASEAN Leaders or ASEAN sectoral/ministerial bodies. These commitments are reflected in ASEAN-wide policy instruments, such as a blueprint, declaration or a sector-wide agreement, which become the main reference for ASEAN cooperation in specific sectors and areas. During the drafting and review stages, major factors contributing to progress include the emergence of champions from the AMS to lead specific components, appropriate levels of technical know-how and capacity to facilitate decision-making, and a common interpretation of the respective draft policy instrument. During the implementation stages, the main factors contributing to progress include resources from the AMS, and the technical capacity to interpret and implement the ASEAN policy instruments. 9 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV 2004 Adopted by AMS following WCO-HS (Harmonized System) 2002 2007 AMS updated nomenclature following WCO-HS 2007 2012 AMS updated nomenclature following WCO-HS 2012 2017 AMS updated nomenclature following WCO-HS 2017 with support from USAID ACTI According to informants, the main entities involved in AHTN are the Directors General of Customs, the Trade Facilitation Division of ASEC, and the AHTN Task Force, which has recently been renamed the Technical Sub-Working Group on Classification (TSWGC). According to key informants, following the adoption of AHTN revisions by the 10 AMS, the nomenclature is transposed by the customs entities in each AMS to determine the product classification and tariff application. The challenge for most countries when updating the nomenclature in line with the HS is that many customs procedures need to also be updated, such as nomenclatures in free trade agreement negotiations, rules-of-origin tariffs, taxation, computer systems, and sometimes finance regulations. The U.S. assistance on AHTN began in the early 2000s under the U.S.-ASEAN Trade Facility Project. ASEAN MEDICAL DEVICE DIRECTIVE Health care is one of the priority sectors of the AEC. The policy development path for medical devices is summarized in Table 3. Table 3. Summary of AMDD Policy Development Path Years Major policy development stages since inception 2003 ASEAN Leaders agreed to realize an AEC and identified 12 sectors for cooperation, including healthcare resulting in the Declaration of ASEAN Concord II (Bali Concord II) 2004 Signed the ASEAN Sectoral Integration Protocol for Healthcare and the ASEAN Framework Agreement for Integration of Priority Sectors as part of the ASEAN Framework Agreement on Services (AFAS) 2005 Started working on harmonizing standards for medical devices through the Medical Devices Product Working Group (MDPWG) chaired by Malaysia until 20156 2007 Adoption of the AEC Blueprint (2008-2015) by ASEAN Leaders, which specifies the implementation of the ASEAN Common Technical Dossiers (ACTD) and the ASEAN Common Submission Dossier Templates (ACSDT) for medical devices as priority actions7 2007-2010 MDPWG conducted consultations followed by technical consultations which included detailed review of specific areas, followed by workshops to agree upon definitions 2010-2014 MDPWG drafted the text in the AMDD, including guidelines in annexes. Draft text underwent several rounds of review, including discussion, negotiation, and revision. This was followed by finalization and agreement in principle among all members of the MDPWG prior to submission for endorsement by the ASEAN Leaders in 2014 2015 ASEAN Leaders endorsed and signed the AMDD as a legal instrument. ASEAN Medical Device Committee (AMDC) formed to implement AMDD 2016-2019 Singapore, Malaysia and Indonesia have ratified AMDD by implementing national laws in compliance with AMDD 6 Source: Interview and the ASEAN Medical Device Directive (2015), ASEAN Secretariat. 7 ASEAN Economic Community (AEC) Blueprint (2008-2015). USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 10 Cambodia, Thailand, Lao PDR and Myanmar have partially ratified the AMDD 2019 Other AMS in the process of preparing documents for ratification 2016 onward MDPWG undertakes monitoring and reporting to the ACCSQ on an annual basis according to the AEC 2025 Monitoring and Evaluation (M&E) Framework The AMDC comes under the ASEAN Consultative Committee for Standards and Quality (ACCSQ), which in turn is under the purview of the Senior Economic Officials Meeting (SEOM) and the ASEAN Economic Ministers (AEM). Other entities involved are the Standards and Conformance Division of ASEC, which coordinates activities. At the time that ACTI commenced in 2013, AMDD was in its stages of finalization. Prior to the AMDD being finalized, several AMS had already developed their own medical device directives, including Singapore, the Philippines, Malaysia and Thailand. Singapore became the first AMS to fully implement the directive, as the country·s medical device regulations are similar to the AMDD. The AEC 2025 M&E Framework8 guides AEC sectoral bodies in developing and operating M&E mechanisms to ensure consistency and coherence in the reporting and monitoring of AEC progress. The AEC M&E framework stipulates three levels: (i) Compliance Monitoring of activities and outputs according to annual priorities of the sectoral work plans (on a quarterly basis); (ii) Outcome Monitoring against Key Performance Indicators (KPIs) set by the sectoral work plan (annual basis); and (iii) an Impact Assessment, usually undertaken at the mid-term of the AEC Blueprint and at the end of the AEC Blueprint (every five years). TRADITIONAL MEDICINES AND HEALTH SUPPLEMENTS Work commenced in this area in 2008, driven by specific priority actions in the AEC Blueprint (2008-2015) for harmonization of technical requirements in TMHS as part of removing technical trade barriers. A summary of the policy development path for TMHS is provided in Table 4. Table 4. Summary of TMHS Policy Development Path Years Major policy development stages since inception 2003 ASEAN Leaders agreed to realize an AEC and identified 12 sectors for the removal of technical barriers to trade, including healthcare resulting in the Declaration of ASEAN Concord II 2004 Signed the ASEAN Sectoral Integration Protocol for Healthcare and the ASEAN Framework Agreement for Integration of Priority Sectors as part of the ASEAN Framework Agreement on Services (AFAS) 2007 Adoption of the AEC Blueprint (2008-2015) by ASEAN Leaders, which specifies the identification of areas for harmonization of technical requirements in TMHS as a priority action. 2008 Formation of the Traditional Medicines and Health Supplements Product Working Group (TMHSPWG) by ACCSQ. Terms of reference for the TMHSPWG developed and approved. 2008 TMHSPWG discussions to define priority areas for TMHS harmonization including good manufacturing practices (GMP), labeling requirements, stability and shelf-life, claims and claims substantiation and limits of contaminants 8 AEC Monitoring website: aecmonitoring,asean.org. 11 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV Years Major policy development stages since inception 2008 Task forces (TF) for each of priority area established, with AMS taking lead roles in task forces. Malaysia took the lead in the GMP TF. GMP TF identified priority instruments required including Guidelines on GMP for manufacturers and importers of TMHS; and Training Programs for regulators and industry 2008-2012 GMP guidelines were drafted and, a review process including assessment by AMS, discussion, negotiation, and revision, followed by finalization and agreement in principle by the TF9 2013 Endorsement of GMP guidelines by TMHSPWG10 2015 Endorsement of a total of ten guidelines for health supplements and nine for traditional medicines 2015 Training-of-master-trainers program for GMP supported by USAID 2016 Development of GMP training modules by TF members from each AMS11 2016 onward TMHSPWG undertakes monitoring and reporting to the ACCSQ on an annual basis12 2017-2018 GMP regional training of trainers by AMS 2017-2019 GMP training of industries in each AMS 2016-2018 Drafting of ASEAN Agreement on Regulatory Framework for Traditional Medicine and the ASEAN Agreement on Regulatory Framework for Health Supplements. 2019 to date Review and negotiation of ASEAN Agreement on Regulatory Framework for Traditional Medicines and the ASEAN Agreement on Regulatory Framework for Health Supplements13 The Traditional Medicines and Health Supplements Product Working Group (TMHSPWG) comes under the ACCSQ. Other entities involved are the Standards and Conformance Division of ASEC, which coordinates activities. Some AMS already had regulations in place prior to the USAID support. Informants mentioned that one of the challenges was that some AMS had regulations for traditional medicines and health supplements separately, such as Vietnam, Myanmar and the Philippines, whereby the former is regulated under pharmaceutical law and the latter under food safety law, while in other AMS such as Malaysia and Singapore TMHS is regulated as one category, according to informants from five countries. Singapore, Malaysia and Indonesia were members of the Pharmaceuticals Inspection Co￾operation Scheme as of 2012 and had established international standards. The informants also stated that manufacturers in AMS have different levels of capacity to harmonize TMHS. SERVICES ASEAN began working on liberalization of services in the early 1990s driven by the interests of the AEM. A summary of the policy development path for Services is provided in Table 5. 9 According to informants from two interviews and one FGD. 10 ASEAN Guidelines for GMP for Traditional Medicines and Health Supplements. 11 One FGD and presentation material shared by FGD members. 12 AEC 2025 Monitoring and Evaluation (M&E) Framework. 13 ASEC informants. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 12 Table 5. Summary of Services Policy Development Path Years Major policy development stages since inception 1992-1995 Following interests of the AEM, discussions on liberalization of services in ASEAN followed by drafting and finalization of ASEAN Framework Agreement on Services (AFAS) 1995 AFAS endorsed by AMS 1995-1997 Protocol to Implement the First Package of Commitments under AFAS drafted and agreed upon following several rounds of negotiations 1997 The Protocol to Implement the First Package of Commitments under AFAS was signed by all AMS 2000 Ratification of Protocol to Implement the First Package of Commitments under AFAS by each AMS 1997-1998 Protocol to Implement the Second Package of Commitments under AFAS drafted, agreed and signed by the Ministers of Trade and Industry 2001 Protocol to Implement the Third Package of Commitments under AFAS drafted, agreed and signed 2004 ASEAN Framework Agreement for Integration of Priority Sectors signed by ASEAN Leaders 2006 Roadmaps for the Integration of Priority Sectors targeting air travel, healthcare, tourism and e-ASEAN signed by ASEAN Leaders 2004 -2014 Protocols to Implement the Fourth to the Ninth Packages of Commitments under AFAS signed 2007 Adoption of the AEC Blueprint (2008-2015) by ASEAN Leaders, which included further liberalization toward a free flow of services as bi-annual targets 2015 Adoption of the AEC Blueprint 2025 (2015-2025) by ASEAN Leaders, which provided policy intent to further deepen and broaden services integration including the negotiation and implementation of a new ASEAN agreement on services 2016 onward CCS undertakes monitoring and reporting to SEOM and AEM on an annual basis according to the AEC 2025 Monitoring and Evaluation (M&E) Framework 2016-2018 Drafting, review, negotiation and finalization of the ASEAN Trade in Services Agreement (ATISA) using a negative-list approach 2019 Endorsement and signing of ATISA by AMS To operationalize the AFAS, instruments in the form of legally binding protocols were developed, each protocol tasked to implement a package of commitments for the AMS agreed upon following rounds of negotiations. A total of 10 such protocols and packages of commitments were developed between 1995 and 2018 under AFAS. Each protocol was endorsed and signed by all 10 AMS. Once the protocols are signed by all AMS, the AMS have three years to ratify the protocol through a process of transposing and completing their internal liberalization schedule according to the evidence14. The main entities include the CCS and the Services and Investment Division of ASEC. 14 Interview with key informant; ASEAN Integration in Services (2015); Protocol to Implement the Fourth Package of Commitments under the ASEAN Framework Agreement on Services (2004). Protocol to Implement the Fifth Package of Commitments under the ASEAN Framework Agreement on Services (2006). Protocol to Implement the Sixth Package of Commitments under the ASEAN Framework Agreement on Services (2007). 13 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV SMALL AND MEDIUM ENTERPRISES ASEAN formally began working on SMEs in April 1995. From this starting point the Small and Medium-Sized Enterprises Working Group (SMEWG) explored areas for SME development at the ASEAN level based on data and inputs provided by AMS. Table 6 provides an overview of the policy development path for the SME development area. Table 6. Summary of SME Policy Development Path Years Major policy development stages since inception 1995 AEM initiates cooperation on SMEs. Establishment ASEAN SMEWG and first meeting of SMEWG comprising representatives from all SME agencies in the AMS 1995-2003 Drafting of the ASEAN Policy Blueprint for SME Development the first major policy instrument for SMEs SMEWG meetings lead to discussion, revision and finalization of the ASEAN Policy Blueprint for SME Development in a highly consultative manner with inputs from all AMS 2004 Endorsement of the ASEAN Policy Blueprint for SME Development (APBSD) 2004-2014 by the AEM 2007 Adoption of the AEC Blueprint (2008-2015) by ASEAN Leaders, which included SME development as a priority action 2009 ASEAN Strategic Action Plan for SME Development 2010-2015 (SAPSMED 2015) drafted by the SMEWG with inputs from the High-Level Task Force on Economic Integration (HLTF-EI) and endorsed by AEC Council 2011-2012 The ASEAN Common Curriculum for Entrepreneurship established 2015 Adoption of the AEC Blueprint 2025 (2015-2025) by ASEAN Leaders, which provided policy intent for development of micro, small and medium enterprises (MSMEs) and recognition of MSMEs as a strategic sector (or Blueprint) 2016 SMEWG elevated to the ASEAN Coordinating Committee on Micro, Small and Medium Enterprise (ACCMSME) 2016 Strategic Action Plan for SME Development 2016-2025 (SAPSMED 2025) drafted and finalized 2016 onward ACCMSME undertakes monitoring and reporting to SEOM and AEM on an annual basis according to the AEC 2025 M&E Framework The strategic schedule in the SAPSMED 2015 and 2025 drives policy implementation, which provides for lead responsibilities for implementation at the regional level by ASEC and ACCMSME. Each AMS is responsible for the implementation of national-level initiatives. Countries such as Singapore, Indonesia, Malaysia, the Philippines and Thailand have comprehensive SME policy frameworks and they often help inform the ASEAN policy development path. The national SME policy frameworks Protocol to Implement the Seventh Package of Commitments under the ASEAN Framework Agreement on Services (2009). Protocol to Implement the Eight Package of Commitments under the ASEAN Framework Agreement on Services (2010). Protocol to Implement the Ninth Package of Commitments under the ASEAN Framework Agreement on Services (2014). USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 14 are always complementary to ASEAN initiatives, and the ASEAN initiatives are integrated as and when required, with support from ACCMSME and ASEC. :OMEN·6 ECONOMIC EMPO:E5MEN7 The work on WEE commenced in 2007 when Vietnam was the Chair of ASEAN. Table 7 provides an overview of the development path for AWEN. Table 7. Summary of WEE Policy Development Path Years Major policy development stages since inception 2007 Concept of AWEN presented by Vietnam during their ASEAN Chairman year, as a cooperation between the ASEAN Committee on Women (ACW) and the ASEAN Confederation of Women·s Organizations (ACWO) 2007-2012 Exploration, definition and drafting of the Terms of Reference (ToR) for AWEN led by the Vietnam Women Entrepreneur Council 2010 The formation of AWEN included as an activity in the ASEAN Socio-Cultural Community (ASCC) Blueprint 2011-2015 and the ACW Work Plan of 2011-2015 2012 Finalized and adopted Terms of Reference of AWEN in the 11th meeting of ACW by all 10 AMS. The ToR includes a vision, set of purposes, regulations, functions, and structure 2014 AWEN was launched through a workshop supported by ACTI 2015 First GREAT Women in ASEAN Initiative launched at trade booth during ASEAN SME Summit supported by Malaysia and ACTI 2015-2018 Conducted a series of activities and meetings to implement AWEN priority agenda 2019 AWEN Bangkok Declaration launched during Thailand·s ASEAN Chairman year The chairmanship of AWEN changes annually and there are usually three focal points in each AMS who are representatives from women·s associations. AWEN established networks with women entrepreneur agencies in each AMS, such as the Indonesian Women Entrepreneurs· Association (IWAPI) and the Philippine Women Entrepreneurs· Network (Phil-WEN). As it is not an official ASEAN body, AWEN does not receive an ASEAN budget line and must be either self-funded or source funds through dialogue partners. It currently has no medium-term work plan. A set of annual priorities are identified during each country·s year developed by the AWEN Chair into an implementation plan. Each of the AWEN priority agenda meetings was chaired by one AMS. TARGETED POLICY AREAS UNDER PROGRESS PROJECT TRAFFICKING IN PERSONS The policy development path for trafficking in persons (TIP) in ASEAN began in the late 1990s. Table 8 summarizes the development path of policy in TIP15. 15 Interviews with four key informants with historical knowledge in the area. 15 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV Table 8. Summary of TIP Policy Development Path Years Major policy development stages since inception 1997 Adoption of the ASEAN Vision 2020 by ASEAN Leaders. This was ASEAN·s earliest political commitment to commence regional cooperation in TIP 1997 Adoption of the ASEAN Declaration on Transnational Crime by the ASEAN Ministerial Meeting of Transnational Crime (AMMTC), as follow up to the agreement among ASEAN Leaders 2004 Adoption of ASEAN Declaration Against Trafficking in Persons Particularly Women and Children by ASEAN Leaders provided a policy statement of intent to focus the direction of TIP efforts 2007 ASEAN Practitioner Guidelines on Effective Criminal Justice Responses to Trafficking in Persons developed by Senior Officials Meeting of Transnational Crime SOMTC as a supporting policy instrument to operationalize the ASEAN Declaration 2009 Adoption of the ASEAN Political-Security Community (APSC) Blueprint 2015, by ASEAN Leaders, which includes actions for TIP 2010 ASEAN handbook on International Legal Cooperation developed by SOMTC as a supporting policy instrument to operationalize the ASEAN Declaration 2011 ASEAN Leaders tasked the AMMTC to develop an ASEAN Convention on TIP 2011-2015 The development of the ACTIP by SOMTC 2015 ACTIP endorsed and signed by ASEAN Leaders as the first policy instrument in TIP with legally binding provisions 2016 Development of the Regional Review on Law, Policies and Practices within ASEAN Relating to the Identification, Management and Treatment of Victims of TIP Especially Women and Children by ACWC supported by PROGRESS 2017 ACTIP entered into force after ratification by six AMS. As of today, nine AMS except Brunei have ratified ACTIP 2017 Adoption of the Guidelines and Procedures to Address the Needs of Victims of TIP by AMS, developed by ACWC and supported by PROGRESS 2017 Adoption of the of Bohol TIP work plan 2017-2020, led by the SOMTC and involving nine ASEAN sectoral bodies ASEAN·s approach to TIP has evolved from a security approach led by SOMTC to a multi-sectoral approach involving nine ASEAN sectoral bodies within the framework of the Bohol TIP Work Plan. SOMTC remains the principal body for combating TIP and is under the APSC Pillar. SOMTC also has the Working Group on TIP, chaired by the Lead Shepherd on TIP, which is currently the Philippines. an Expert Group Meeting on ACTIP was established to draft the ACTIP and this since ceased operation. Other ASEAN sectoral bodies dealing with TIP include AICHR, the ASEAN Commission on the Promotion and Protection of the Rights of Women and Children (ACWC), and the ASEAN Committee on the Implementation of the ASEAN Declaration on the Protection and Promotion of USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 16 the Rights of Migrant Workers (ACMW). AICHR is the ASEAN overarching human rights body, with a mandate to mainstream a human rights-based approach to TIP across all ASEAN Communities. ACWC promotes gender-sensitive and children·s rights-based approaches, and rehabilitation of TIP victims. ACMW has conducted activities that relate to the nexus between trafficking and migrant workers. Other ASEAN bodies such as ASEAN Directors-General of Immigration Departments, Heads of Consular Affairs Divisions of the Ministries of Foreign Affairs, Senior Law Officials Meeting, Senior Officials Meeting on Education, Senior Labour Officials Meeting and Senior Officials Meeting on Social Welfare and Development have also played supporting roles in the ASEAN TIP agenda within the framework of the Bohol TIP Work Plan 2017-2020. Prior to ACTIP, many AMS had a mixture of disparate policy instruments for TIP16. For example, Malaysia had legislation against TIP and smuggling of migrants, and Thailand had a policy on anti￾trafficking, and a process of victim identification. Many AMS have established inter-agency bodies or task forces to coordinate TIP efforts across government agencies dealing with education, law and human rights, health, and women and children. These include Cambodia (1999), the Philippines (2003), Myanmar (2006), Malaysia (2007), Indonesia and Thailand (2008), Singapore (2010), Brunei (2011), and Lao PDR more recently in 2016. However, limited mechanisms for the sharing of information between AMS existed prior to ACTIP. YOUTH The policy development path for youth was initiated following the acknowledgement among ASEAN Leaders of the importance of active participation of youth in the development of ASEAN, through the Declaration of Principles to Strengthening ASEAN Collaboration on Youth (1983). Several additional policy instruments followed, which are summarized in the table below17. The policy instruments do not have legal provisions and therefore do not require ratification or the development of legal instruments in the AMS. Table 9. Summary of Youth Policy Development Path Years Major policy development stages since inception 1983 Declaration of Principles to Strengthening ASEAN Collaboration on Youth by ASEAN Leaders 1992 Establishment of ASEAN Ministerial Meeting on Youth (AMMY) through the declaration of AMMY 1997 Establishment of ASEAN Sub-Committee on Youth (ASY) to operationalize the ASEAN cooperation on youth 2002 Transformed ASY to Senior Officials Meeting on Youth (SOMY) 2000 Yangon 2000 Declaration on Preparing ASEAN Youth for the Challenges of Globalisation 2003 Manila Declaration on Preparing ASEAN Youth for the Challenges of Globalisation 16 Interview with four key informants. 17 Interview with three informants; https://asean.org/storage/2018/02/ADOPTED-TOR-of-SOMY_19Jul19.pdf 17 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV Years Major policy development stages since inception 2009 Adoption of the ASEAN Socio-cultural Community Blueprint 2015, by ASEAN Leaders which included youth priorities 2011 Hanoi Initiative to strengthen future generations of leaders and entrepreneurs 2013 Bandar Seri Bagawan Declaration on Youth Volunteerism, Entrepreneurship and Employment 2015 Adoption of the ASEAN Socio-cultural Community Blueprint (2016-2025), by ASEAN Leaders which included youth priorities 2016 ASEAN Work plan on youth 2016-2020 These policy instruments followed the policy continuum stages. Other instruments that were established included the ASEAN Youth Programme Fund, which was supported by the Singapore￾ASEAN Youth Fund; the ASEAN Youth Volunteer Programme (AYVP), which was established by Malaysia; and the ASEAN Young Entrepreneurs Seminar. Since 1992, the main sectoral entities have been the AMMY, and the ASY which was renamed to SOMY in 2002. The Education, Youth and Sports Division of ASEC coordinates activities. In addition, The ASEAN Foundation was established in 1998 as an entity responsible for implementing youth activities. The ASEAN University Network (AUN) supports youth activities in higher education. In terms of implementation, each AMS typically submits proposals on areas that it would like to implement from the five-year work plan, or new activities on an annual basis at the SOMY meetings for agreement. Many youth initiatives at the ASEAN level are activity based. Some ASEAN countries, such as Singapore, Malaysia and Thailand, have developed a more comprehensive national policy framework for youth, which feeds into the ASEAN process through the SOMY and AMMY meetings, and to the ASEAN work plans on youth. 3.1.2. CONCLUSIONS According to the evidence, there is no single documented process for the policy development path in ASEAN. Since 2008, the ASEAN Charter articulates the institutional framework for ASEAN, including nine ASEAN Organs, their composition and functions. However, policy development does not receive explicit coverage. Article 21 of The ASEAN Charter provides autonomy for the three ASEAN Community Councils, to prescribe their own rules of implementation and procedure. Consequently, variations exist in the policy development paths of different sectoral bodies, as illustrated in the findings. Based on an examination of the findings in each of the priority policy areas of this evaluation, conclusions are provided below for a generalized development path for policy in ASEAN. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 18 The development path for policy in ASEAN can be conceptualized using the USAID/ASEAN Policy Continuum as described in terms of four major phases and a total of 11 distinct stages. Phase I: Commitment and Institutional Arrangements x Stage 1: Inception. Catalyzed by political commitment from ASEAN Leaders. Reflected in instruments such as the ASEAN Community Blueprints. This mandates Sectoral Ministerial Bodies and Senior Officials Meetings to pursue the development of a given policy area. x Stage 2: Institutional arrangements. This includes establishment of a task force (TF), a product working group (PWG), a working group (WG) or an expert group made up of representatives from the relevant sectoral entities in each AMS. This is followed by formulation of terms of reference for these entities, which are approved by the AMS at a Senior Officials· Meeting or at the Ministerial level. x Stage 3: Definition. Further clarification and policy definition take place by prioritizing the target areas and task distribution, and identification of lead AMS for specific areas. Phase II: Policy Development x Stage 4: Drafting of policy instruments. This focuses on drafting of text for the policy instrument. For policy instruments in the AEC pillar, it was observed that drafting is a technical process driven by an appreciation of best practices, and a common understanding Figure 3. ASEAN Policy Development Path 19 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV of issues and opportunities among the sectoral body members from AMS. In sensitive areas such as TIP, the drafting process was driven by political commitments. Some agreements, such as AFAS, require additional instruments, such as protocols to operationalize subsequent commitments by the AMS. For policy instruments that do not have legal provisions, such as those in youth engagement, SME development and WEE, Stage 4 focuses on the drafting of work plans or strategic action plans. x Stage 5: Review of draft policy instruments. Assessment and review by the AMS of the draft policy text, which includes discussion, negotiation and revision. It can often take several rounds of meetings at the TF, PWG or WG levels, until a common interpretation of the policy instruments is achieved among the AMS. x Stage 6: Policy instrument finalization and approval. Finalization and agreement in principle by all AMS at the respective TF, PWG or WG levels. It is then passed on to the senior bodies, such as committees or Senior Officials· Meetings, for approval. Phase III: Member State approval x Stage 7: Endorsement by AMS. This culminates in endorsement by senior bodies and the signing of the policy instruments by the respective Ministers (for instruments without legal provisions) or ASEAN Leaders (for instruments with legal provisions). x Stage 8: Ratification by AMS. This is relevant for policy instruments with legal provisions, such as the AMDD, and ensures that the legal framework is in place in each AMS to implement the ASEAN policy instrument. This requires a process to transpose the policy into the legal and institutional context of each AMS. There is usually a finite period within which each AMS should ratify these protocols with additional time sometimes provided to the CLMV countries. Phase IV: Implementation x Stage 9: Implementation at the regional level. These include additional activities, data collection, and sometimes the development of supporting instruments, such as platforms, partnerships, indexes or the development of legal protocols and technical annexes. The development of additional policy instruments follows the activities described from Stages 4 to 8. x Stage 10: Implementation in the AMS. This includes implementation of activities from action plans, the provision of training for government and businesses, and outreach by AMS. For policy instruments with legal provisions, implementation is done based on an internal schedule developed by each AMS. Additional capacity building is often required for this stage. x Stage 11: Monitoring and reporting. This includes compliance monitoring, outcome monitoring and impact assessment, as guided by the Monitoring and Evaluation (M&E) Framework of a given ASEAN Community Blueprint. The findings revealed that a range of ASEAN policy instruments are developed. The diagram below provides a summary of the different types of policy instruments and examples. Core policy instruments include: Framework principles from a Community Blueprint, sectoral agreement, or an ASEAN Leaders Joint Statement; Planning instruments such as strategic action plans, work plans, guidelines or terms of references, which guide implementation; and Legislative instruments, which are policies with legal provisions such as directives, conventions or agreements with legal protocols. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 20 Supporting policy instruments include: Economic instruments, such as tariffs, taxes, incentives, and specific funds; Informational instruments, such as policy indexes, online platforms and outreach tools; Capacity-building instruments, such as training modules, curricula, training of trainers, or regular training programs; and Partnership instruments, such as networks, business advisory councils, and partnerships with civil society or youth. 3.1.3 RECOMMENDATIONS 1. It is recommended that USAID/ASEAN considers revising its policy development continuum tool to better reflect the stages and details presented in the 11-step framework in this chapter. This framework, summarized in Figure 3, can also serve as a basis to guide and sequence technical assistance policy support. Figure 4. ASEAN Policy Instruments 21 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV 2. Furthermore, the evaluation team recommends the recognition of a range of ASEAN policy measures or instruments as provided in Figure 4 above. This will enable the rationalization and targeting of support to areas such as youth and SMEs, which are sometimes not thought of as having policy dimensions. 3.2. EVALUATION QUESTION 2: HOW DID USAID ASSISTANCE CONTRIBUTE TO THE DEVELOPMENT OF THESE POLICIES? Synthesis USAID assistance contributed to the development and implementation of ASEAN policy in three major ways: (i) capacity building of stakeholders to further develop or to implement the policy instruments at the regional and national levels; (ii) the adoption of international best practices and standards; and (iii) stakeholder engagement from a range of ASEAN bodies and sectors, as well as types of stakeholders (e.g., government, CSOs, and the private sector). These contributions strengthened technical skills and led to a deeper understanding of the policies, improving the likelihood of the effective adoption of the policies. Moreover, USAID is perceived as a preferred dialogue partner, as it responded to ASEAN needs, was not perceived as having its own agenda, adopted strong participatory approaches, was a reliable long-term partner and fielded experts who were neutral, without a specific AMS bias, and was responsive with relevant international and regional experience. 3.2.1 FINDINGS HOW ACTI CONTRIBUTED TO POLICY DEVELOPMENT IN ASEAN ACTI contributed to the development and implementation of AEC policies based on demand-driven requests from ASEC and ASEAN sectoral bodies to support specific policy areas. For each area ACTI developed a concept note or proposal based on the request from ASEAN providing options for support. This concept note or proposal was then reviewed, followed by the selection of priority options and their endorsement by the respective ASEAN sectoral body. Much of the support provided by ACTI was in Stage 4 (drafting) of the policy continuum, such as for the AMDD, the TMHS GMP training guidelines, the ATISA, the ASEAN Strategic Action Plan for SME Development (SAPSMED) 2025 and the AWEN Strategy. ACTI also supported regional and national implementation (Stages 9 and 10 of the policy continuum process) of policy instruments such as the AHTN 2012 and AHTN 2017 revisions, the AMDD, the SAPSMED 2015, the SAPSMED 2025 and the AWEN Strategy. The support was in the form of events (including ASEAN meetings, workshops and conferences), travel and subsistence for participation by ASEAN officials and other stakeholders, experts and speakers, and studies and dissemination campaigns, and in the case of the SME development area an online platform. The first major outcome from the ACTI support was capacity building for the further development of the policy instruments in cases such as AHTN, TMHS and ATISA. For example, in AHTN, the capacity building contributed to: USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 22 1. Interpreting the World Customs Organization (WCO) revisions and revising the AHTN in 2017 in line with the WCO-HS updates of 2012 and 2017, and the development of new AHTN classification structures. 2. Improved AMS skills in identification and classification of products that have not yet been classified by national customs officials and to determine the tariff application. In the case of services, the capacity building contributed to a change in policy direction by the CCS from a positive-list approach to a more liberalized negative-list approach, which is reflected in some of the provisions of the draft ATISA. The capacity building also enabled the implementation of the policy instruments at regional and national levels in the case of AHTN, AMDD, TMHS, SME development and WEE. For example, USAID capacity building led to better understanding among regulators of the regulatory principles of the AMDD and how these should be transposed at the national level. Examples include:18 1. In Cambodia, Lao PDR, and Vietnam the assistance from ACTI helped government agencies to develop national regulations on how to classify medical devices and undertake risk classification, according to informants from these countries. 2. In Indonesia capacities in risk classification, grouping and quality management systems were enhanced, and risk classification was used to revise the national guidelines. 3. In Malaysia, Singapore, and the Philippines government staff were provided with the necessary capacity to regulate medical devices to implement their national programs, according to representatives from these countries. In the area of TMHS, informants mentioned that increased capacity and knowledge in Good Manufacturing Practice (GMP) provided by ACTI led to the following: 1. Following the USAID workshops, a cohort of trainers were established across the AMS. These trainers developed training modules with the responsibility for developing chapters in the training modules shared across AMS. These modules are being used to provide training to industry every two years as part of the licensing process. 2. In the Philippines and Lao PDR, the training modules were used to cascade training within the government across different departments and eventually to industry, according to representatives from these countries. 3. In Vietnam, the Government decided to make GMP mandatory in health supplements and promulgated a regulation, Circular 18 of July 2019, according to representatives from Vietnam. Vietnam has also developed a National Standard for Health Supplements covering the maximum level of contents, labels and claims as a result of the training received from ACTI and the ASEAN Harmonization process. Vietnam expects to implement all 10 ASEAN guidelines on health supplements well ahead of the deadline. 4. In Cambodia, the training was used to evaluate the dossiers of companies exporting TMHS to Cambodia. 18 Interviews with AMS, ASEC and USG. 23 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV In the area of SME development, examples of how USAID assistance contributed to policy implementation include the following: 1. Following a workshop in Lao PDR led by ACTI on ePayments and digital economy government, Laotian businesses gained understanding about the digital opportunities and how to deal with payment gateways. This led to the development of an eMarketplace platform, www.plaosme.com, funded by the Asian Development Bank. This helped address previous gaps in the eCommerce ecosystem for Laotian businesses. 2. A workshop in Lao PDR on equity crowdfunding (ECF) led to an increase in the number of ECF providers in the country from 8 to 12. According to informants, the ACTI approach of having a senior government official or business leader as the moderator, together with experts with specific knowledge of market issues and an understanding of ASEAN cultural norms, contributed to the effectiveness of the workshops. The second major outcome of ACTI support was the bringing together of stakeholders such as ASEAN entities, AMS government entities, and representatives of the ASEAN and U.S. private sector, to create a platform to develop a common understanding of issues. For example, in TMHS, the participatory approach led to industry and government coming together to interpret and understand the GMP guidelines and create a common understanding to help companies in exporting to all AMS. In AHTN, the participatory approach led to improved quality of dialogue and interpretation of WCO revisions in the taskforce meetings, as the technical expert acted as a neutral facilitator. In the case of WEE, through its focus on participatory approaches ACTI helped create a network involving AWEN focal points, women entrepreneurs, business leaders, and sectoral bodies from the AEC. ´ACTI helped AWEN to make a strategic plan«SEOM [Senior Economic Officials Meeting] was happy with that presentation and agreed to meet us annually afterwards.µ A third major outcome was the adoption of international standards in the case of AHTN, AMDD, and TMHS, and international best practices in the case of services, SME development, and WEE. For example, in TMHS, ACTI support led to the acceptance of international standards by ASEAN, as opposed to the development of a separate ASEAN certification system. These international standards in turn helped to enhance the ability of ASEAN companies to export globally and for international companies to undertake trade within the AEC. HOW PROGRESS CONTRIBUTED TO POLICY DEVELOPMENT IN ASEAN In the area of TIP, PROGRESS contributed to policy development at Stage 3 (definition), Stage 4 (drafting of policy instruments), Stage 5 (review), Stage 6 (finalization), Stage 7 (endorsement) and Stage 9 (implementation at regional levels). For youth, PROGRESS contributed at Stage 4 and Stage 9. The support was in the form of events, including ASEAN meetings, workshops and conferences; travel and subsistence for ASEAN officials and other stakeholders, experts and speakers, and; studies and dissemination campaigns. A major outcome of PROGRESS support was the development of guidelines and stakeholder capacity building in TIP and youth to implement policy instruments at regional and national levels. For example, under TIP, the ACWC Regional Guidelines and Procedures to Address the Needs of Victims of TIP, supported by PROGRESS, were officially endorsed by the ASEAN Senior Officials USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 24 Meeting on Social Welfare Development (SOMSWD). These are now included an ASEAN document providing comprehensive guidelines to the AMS on handling TIP victims. According to informants: ´ASEAN does not need to produce more documents, but to take actions by implementing these Guidelines. There should be implementation plans for the Guidelines in line with the AMS national action plan in areas of TIP, and AMS local cultures and customs.µ ´Some countries came out with policies on protection of children from trafficking. I would say this was guided to some extent by the USAID reports.µ The consultations in the eight AMS on the ASEAN Convention Against Trafficking in Persons, Especially Women and Children (ACTIP), done as a collaboration between PROGRESS and the International Organization for Migration (IOM), contributed to increased awareness of ACTIP among AMS stakeholders (PROGRESS Final Report). A second major outcome was bringing together representatives from different ASEAN sectoral bodies, AMS governments, practitioners of human rights, and CSOs and NGOs to discuss issues, develop common understandings and establish partnerships. These included, for example, bringing together multiple sectoral bodies covering AICHR, SOMTC, and ACWC to address TIP, which enabled the development of a shared understanding of the issues. The stakeholders included government front-liners, such as immigration and police officers dealing with law enforcement, agencies and practitioners dealing with women·s and child rights, human rights commissions, CSOs and NGOs. According to informants: ´The value of an external partner like PROGRESS is providing the spaces to speak and engage more freely, such spaces provide opportunity to discuss openly, it is a partnership involving many stakeholders.µ ´Through this kind of workshop each AMS can talk about issues together without being accused of breaching the non-intervention principles. It also provides space in which everyone is comfortable to talk about issues without blaming, naming, and anything sensitive for their country. Representative from each country can voluntarily present their domestic problem in terms of TIP, such as how they address the problem, challenges, and gaps.µ As a result of participatory workshops involving a range of stakeholders, the promotion of human rights-based approaches in TIP moved higher up the ASEAN agenda and have been mainstreamed across the three pillars of the ASEAN Community. In the area of youth, the participatory approaches for stakeholder engagement led to the creation of awareness and understanding among ASEAN youth on an ASEAN identity. This led to a network of ASEAN youth who have returned to their AMS as ASEAN advocates and can now help to promote the ASEAN Community, according to informants. ´It also helped with building the ASEAN identity. I got to meet other ASEAN youth and understood their problems and issues. Common issues help with building a common identity.µ 25 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV 3.2.2 CONCLUSIONS Most of the USAID assistance from ACTI and PROGRESS was in response to demand-driven requests from ASEAN stakeholders. Much of the support provided was at Stage 4 (the drafting of policy instruments) of the policy continuum, followed by regional implementation of policy (Stage 9 of the policy continuum). The type of support provided was in the form of events (including meetings, workshops and conferences), travel and subsistence for participation by ASEAN officials and other stakeholders, experts and speakers, and studies and dissemination campaigns. The manner in which this USAID assistance contributed to the development of policies was due to the three characteristics of the assistance provided. First, USAID assistance strengthened the capacity of ASEAN stakeholders in the drafting and reviewing of policy instruments, and in regional implementation. This contributed to improved capacity for implementation at the national level too. Second, USAID contributed to the adoption of international best practices and standards, leading to enhancements when drafting ASEAN policy instruments, such as the AMDD, GMP training modules for TMHS and ATISA. Third, USAID assistance adopted a strong participatory approach and engaged a range of stakeholders as part of the consultation process, including ASEAN sectoral bodies across a number of sectors (e.g., youth, AEC bodies, human rights bodies), CSOs, and the private sector. USAID, through ACTI and PROGRESS, is perceived as a preferred dialogue partner in most of the support areas, according to the informants. Factors that contributed to this preference included: (i) being responsive to ASEAN needs; (ii) the relatively fast approval times for the mobilization of resources; (iii) not being perceived as having its own agenda, but instead being supportive of ASEAN·s agenda; (iv) the provision of experts who were neutral, with no specific country bias, be it to any AMS or other countries, and being responsive with the relevant international and regional experience; and (v) the adoption of strong participatory approaches in the design and implementation of activities. The long-term support for specific policy areas contributed to positive perceptions among ASEC and the sectoral bodies that USAID was a reliable dialogue partner. USAID, through its ACTI and PROGRESS projects, commenced its assistance at various policy development stages, including Stages 4 to 6 (policy formulation), Stages 7 and 8 (endorsement), and Stage 9 (regional implementation). The types of assistance provided by the two projects were similar, with all assistance bringing about improvements in the capacity of ASEAN stakeholders, allowing for wider engagement, and promoting better adoption of international best practices and standards. 3.2.3 RECOMMENDATIONS 1. Support measures targeting Stages 2 (institutional arrangements) and 3 (definition) of the policy continuum should be demand-driven and carefully sensitized to meet the needs of ASEAN stakeholders. 2. Given that much of the support has focused on Stages 4 (drafting) and 9 (regional implementation), support can increasingly focus on Stage 10 (national implementation). Experts hired should have a neutral perspective, be responsive and sensitive to ASEAN needs, and have a combination of international and national experience. Continued support by USAID is important to be perceived as a long-term and reliable ASEAN dialogue partner, something which takes many years to cultivate. Specific recommendations regarding the content of each support area are provided under evaluation question 5. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 26 3.3. EVALUATION QUESTION 3: WHICH FORMS OF ASSISTANCE CONTRIBUTED THE MOST TO THE DEVELOPMENT OF THESE POLICIES? WHICH CONTRIBUTED THE LEAST? Synthesis USAID assistance that contributed to the development of the policies included the bringing together of stakeholders to discuss issues and arrive at a common understanding, facilitation provided by neutral and experienced international experts and capacity building, were major success factors across all areas for the policy development process. ASEAN meetings, cross￾sectoral events, studies and capacity-building efforts, including workshops, training and the training-of-trainers, contributed to both regional and national capacity building. The main forms of non-USAID assistance that contributed to the development of policies were leadership (including champions) and support by AMS in specific sub-areas. The majority of USAID assistance made important contributions to the respective policy areas as they were designed in close partnership with the relevant ASEAN stakeholders. Some forms of assistance that contributed to a lesser degree included certain studies that could have been rectified through a stronger participatory approach. 3.3.1 FINDINGS ACTI PROJECT ASEAN HARMONIZATION OF TARIFF NOMENCLATURE USAID assistance that contributed the most to ASEAN policy According to the informants, the major dialogue partner in this area for the 2007, 2012 and 2017 AHTN revisions was the United States. The provision of international experts who were former customs officials with knowledge and experience of the WCO-HS process, and who were from neutral countries outside ASEAN, was the major form of assistance that contributed to the development of the capacity of the AHTN task force. The meetings facilitated by the experts were able to ´smoothenµ discussions, support increasingly closer interaction between stakeholders, enabled common understanding and helped align the updating of the AHTN and product classification with practical realities. The capacities and common understanding gained from the regional meetings also contributed to implementation of the policy updates in the AMS. ´The expert has helped AMS, not just Indonesia, to have better processes in reviewing and amending AHTN to be in line with the international HS changes.µ Non-USAID assistance that contributed the most to ASEAN policy When USAID stopped its support for the next AHTN revision, ASEC facilitated the support of another dialogue partner using the same consultant that had been previously hired by USAID. According to one informant: 27 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV ´When the United States stopped supporting the AHTN revision, ASEC searched for other sponsors, and now the AHTN consultant is sponsored by Japan.µ USAID assistance that contributed the least to AHTN According to the informants, all forms of assistance were useful contributions designed closely with ASEC and sectoral bodies. Most significant AHTN achievements due to USAID assistance The most significant achievement was the increased policy-making capacity of the AHTN task force to implement the AHTN revisions in line with the WCO-HS revisions, the development of new AHTN classification structures, and their implementation across all AMS. A second major achievement was the bringing together of stakeholders to discuss issues and arrive at a common understanding in an open manner. ASEAN MEDICAL DEVICE DIRECTIVE USAID assistance that contributed the most to ASEAN policy The major form of assistance that contributed to the development of the AMDD and supporting policy instruments was the ASEAN meetings and workshops that were supported by ACTI. This assistance also helped the AMS to interpret the AMDD and annexes, and transpose these at the AMS level through revisions or new legislation. According to AMS representatives: ´USAID has provided more specific training in this area than other donors. If we had not received the USAID training, we would have been slower to transpose the legislation.µ Non-USAID assistance that contributed the most to ASEAN policy Much of the drafting process for the AMDD was supported by the AMS through several rounds of meetings. According to informants the most significant achievement from this support was the harmonization of requirements for medical devices across AMS, in line with international standards, through the signing of the AMDD, and the development of supporting instruments and procedures, such as the post-market document and evaluation process. Other dialogue partners have recently started to support ASEAN. According to an ASEC informant: ´The other partner that recently started providing knowledge sharing and training is Japan. The advantage of Japan·s approach is that the experts are regulators, not from industry, and they share their experiences with ASEAN regulators, which is very relevant. But Japan does not provide financial assistance for participants to attend the training.µ USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 28 USAID assistance that contributed the least to AMDD According to some AMS representatives, the regional training was not very effective, as it provided a limited number of slots for participants from a given AMS to create scale, and because the training topics were too general. Most significant AMDD achievements due to USAID assistance The major achievement from USAID assistance was the development of capacity among AMS to better interpret the AMDD and its annexes, and transpose these through revisions of existing legislation, such as in the Philippines and Indonesia, or the development of new legislation, such as in Lao PDR and Cambodia, to regulate medical devices according to the AMDD. USAID in Myanmar provided specific support to Myanmar for transposing the AMDD. Seven out of 10 AMS have now transposed the AMDD into national legislation and ratified the AMDD, which is a major achievement according to these informants. Countries such as Lao PDR were able to ratify the AMDD ahead of some of the more advanced AMS. TRADITIONAL MEDICINES AND HEALTH SUPPLEMENTS USAID assistance that contributed the most to ASEAN policy ACTI·s training-of-trainers (TOT) program in GMP increased capacity of the GMP task force, which led to the development of training modules by the TF with the sponsorship of the AMS. The ACTI￾supported TOT program, the modules developed by sectoral body representatives with AMS sponsorship, and the support from AMS to cascade the training all contributed to policy implementation. According to one representative: ´Although we had the GMP guidelines in place, awareness among industry was low. Based on the USAID TOT, ASEAN developed the GMP training modules and we delivered training in 2017 to industry. This has enabled industry to comply better with the regulatory requirements. The modules are the same as those used for government training, and hence industry and regulators are on the same page and harmonized to the extent possible with regards to interpreting the guidelines.µ Non-USAID assistance that contributed the most to ASEAN policy The TMHSPWG meetings and coordination by ASEC were the most significant contributions to the development of the TMHS guidelines, as there were no other dialogue partners involved. According to the informants, the most significant policy achievement in the area was the harmonization of requirements on TMHS across all 10 AMS, through the development of guidelines that now facilitate trade between the AMS as a result of the support from the AMS and ASEC. These guidelines have also facilitated transparency for the sector. USAID assistance that contributed the least to TMHS According to the informants, all areas of assistance were effective and important due to the strong consultation process between ACTI, ASEC, the ASEAN sectoral bodies and the experts provided. 29 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV The evaluation did not hear of, or identify, any forms of assistance that were considered ´least effective.µ Most significant TMHS achievements due to ACTI assistance The major achievement was the development of capacity to implement the guidelines in the AMS, particularly by regulators through training programs, for example in Malaysia, the Philippines and Lao PDR. As a result, the sector was able to interpret and implement the guidelines in a manner similar to the regulators. In Vietnam, due to the ACTI training, the Government promulgated a regulation, making GMP mandatory for health supplements. Vietnam also developed a National Standard for Health Supplements covering the maximum level of contents, labels, and claims, as a result of the training received from ACTI, according to informants from Vietnam. SERVICES USAID assistance that contributed the most to ASEAN policy Assistance from USAID in the form of a study and workshops contributed greater awareness and knowledge for a new policy direction to be adopted by the sectoral body moving from a positive-list approach to a negative-list approach during the formulation of the new services agreement -- ATISA. The negative-list approach is a more liberalized approach as it only places restrictions on those sectors and modes of service supply that are not liberalized, opening all other sectors and modes of supply for entry and investment from ASEAN-based firms. Non-USAID assistance that contributed the most to ASEAN policy The AMS supported all the meetings during the policy development process for both the old agreement, AFAS, and the new agreement ATISA, which contributed to the policy development process. According to AMS informants, the old agreement involved the development of several packages of commitments, and these have over the years led to the substantial improvement in the logistics services sector in AMS. In terms of implementation, countries such as Singapore and Malaysia can implement policies using their own resources. For the other AMS, training and technical expertise at the regional level were usually the major forms of assistance that contributed to AMS policy implementation, according to an informant from an ASEAN sectoral body, who added: ´The training is very useful as we gain from experts and also exchange with peers from the region. As a result, some of my thinking has become similar to representatives from AMS, so this helps [with discussions].µ USAID assistance that contributed the least to services All assistance provided by ACTI was relevant as it was designed in close collaboration with ASEC and the sectoral body. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 30 Most significant services achievements due to USAID assistance The most significant achievements in the area of services were the improved capacity of the stakeholders to adopt a new policy direction liberalizing the trade in services from a positive-list approach to a negative-list approach. SMALL AND MEDIUM ENTERPRISES DEVELOPMENT USAID assistance that contributed the most to ASEAN policy ACTI provided training to over 5,000 entrepreneurs in eight AMS through training events delivered through the U.S.-ASEAN Business Alliance for Competitive SMEs19. ACTI also supported studies in areas such as facilitating equity crowdfunding (ECF). These studies provided information and analysis to help decision-making by ACCMSME. ACTI provided substantial support to establish the ASEAN SME Academy, including expertise, platform development, and training on the usage and development of training modules by major U.S. companies. Subsequent support from the Philippines helped to create greater awareness of the Academy and continued its operation. Non-USAID assistance that contributed the most to ASEAN policy According to the informants, the most important achievement in the area was moving the MSME agenda higher up the list of ASEAN priorities, resulting in the SMEWG being renamed as a committee, namely the ASEAN Coordinating Committee on Micro, Small, and Medium Enterprise (ACCMSME). The main form of assistance that contributed to the increased prioritization of the MSME agenda in ASEAN was the leadership and championing by certain AMS such as Indonesia at the outset. A second major achievement was the development and implementation of the SAPSMED 2025. This is currently the main policy instrument guiding SME development in ASEAN. This was by supported by regular ASEAN sectoral body meetings, which were sponsored by the AMS. Other inputs which supported the development of SAPSMED 2025 were the Regional Policy Network (RPN), which was supported by OECD. The RPN covers topics such as good regulatory practice and M&E for SME policy implementation. OECD and the Economic Research Institute for ASEAN and East Asia (ERIA) conduct policy studies on SMEs every three years and provide recommendations. AMS also contributed to the development of supporting policy instruments included the ASEAN SME Policy Index, the ASEAN Guidelines on One-Village One-Product (OVOP), and a regional program for promoting internship schemes. USAID assistance that contributed the least to SME Development According to one informant, the Handbook for MSME Access to Alternative Sources of Finance in ASEAN was not very useful for policymakers but was a useful resource for SMEs. This could have been rectified at the design stage with closer involvement of the sectoral body. According to 19 ACTI Final Report, 2018; ACTI Annual Performance Management Progress Report, 2018. 31 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV another informant, the ASEAN SME Academy failed to contribute in a major way to SME development as an ASEAN instrument for building capacity among SMEs. Many agencies in the AMS are unaware of the Academy, the modules were not updated, and language was an issue for micro enterprises, as many owners were not fluent in English. Further recommendations are provided under evaluation question 5. Most significant SME Development achievements due to USAID assistance Some informants felt the establishment of the ASEAN SME Academy was an achievement since it provided an ASEAN policy instrument in the form of a training platform for ASEAN MSMEs to access training materials. The ASEAN SME Academy established is sustainable and is hosted by the Government of the Philippines. Other achievements include contributing to one of the priorities of the SAPSMED 2025 to provide access to information and resources for ASEAN MSMEs to be competitive through capacity development provided by the U.S.-ASEAN Business Alliance. :OMEN·6 ECONOMIC EMPO:E5MEN7 USAID assistance that contributed the most to ASEAN policy The major forms of ACTI assistance that contributed to policy development of AWEN included capacity building, support for events and meetings, and inputs to the strategy and work plans. ACTI also supported cross-sectoral events, and a study that contributed to the AWEN strategy. Non-USAID assistance that contributed the most to ASEAN policy According to five AWEN informants, the most significant achievement was the establishment of AWEN and subsequent development of the AWEN Strategy in the form of the ´AWEN Bangkok Declaration,µ uniting women entrepreneurs under one platform to create business opportunities for women. The major form of assistance that contributed to the establishment of AWEN was the lead taken by Vietnam during its year as ASEAN Chair to champion the formation of AWEN. The subsequent Chairs from the Philippines and Thailand also played a significant role in AWEN. Initiatives such as GWAI started during the time of the Philippines Chair. When Thailand was Chair, AWEN was awarded a seat on the ASEAN Business Advisory Council (ABAC) and received high￾level endorsement from the Government and the Royal Family, which supported policy development. AWEN collaborates with the Ministry of Social Security and Human Development and works with National Bank to finance SME projects in Thailand. The AWEN focal points, who are unpaid volunteers, and the support provided by AMS governments, contributed substantially to AMS policy implementation efforts. USAID assistance that contributed the least to WEE According to the informants, all forms of USAID assistance made important contributions as they were designed in close partnership with the relevant ASEAN stakeholders. Most significant WEE achievements due to USAID assistance According to five AWEN informants, ACTI was one of the major contributors to the formal launch and operation of AWEN. ACTI·s support to AWEN led to clear impacts in the sustainable operation of AWEN, such as the launch of AWEN, the development of AWEN·s strategy, the conduct of AWEN meetings, the setting-up of AWEN focal points in the AMS, and training and capacity building USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 32 to encourage women entrepreneurs and MSME awards. ACTI also created greater awareness and visibility on women entrepreneurs and AWEN among ASEAN sectoral bodies, particularly in the AEC. This led to AWEN having a seat in the ABAC. This helped bring the AEC pillar closer to AWEN and women entrepreneurs who are traditionally placed (or mis-placed) under the ASCC. ACTI also contributed to the Action Agenda on Mainstreaming WEE in ASEAN, through recommendations. This was reviewed by the AEM and approved by the ASEAN Leaders in 2017, gaining visibility among the political leadership of the AMS. ACTI enabled GWAI to expand from a national to a regional initiative with a network of ASEAN entrepreneurs through support for its regional launch. At the AMS level there were varying levels of achievement, according to AWEN representatives from each country. In Malaysia, AWEN is active in policy advocacy, working with government to create financial assistance, promote ECF services, and access to angel financing. In Vietnam, a Women Entrepreneurs Forum takes place annually. In the Philippines, there is the Magna Carta for supporting women that works well with AWEN. In Myanmar, Indonesia and Singapore, AWEN was established, comprising women entrepreneurs and governments. PROGRESS Project TRAFFICKING IN PERSONS USAID assistance that contributed the most to ASEAN policy The technical expertise and support for meetings provided by PROGRESS contributed the most to the development of the ACWC Guidelines and the increased prominence of human rights-based approaches in the ASEAN agenda. PROGRESS was also able to support the development of platform for engagement and collaboration between the most challenging bodies within ASEAN to have dialogue about TIP and human rights, such as the AICHR-SOMTC Joint Workshops during 2015- 2017. PROGRESS facilitated the initial cross-sectoral engagement between SOMTC and AICHR in their joint workshop and activities. That collaboration process contributed to creating the multi￾sectoral platform that lead to the Bohol TIP Work Plan. The main contribution to the development of the ACWC Guidelines and the increased prominence of human rights-based approaches in the ASEAN agenda was from the PROGRESS project in the form of expertise, ASEAN-level meetings, national consultations and cross-sectoral events, such as the AICHR youth debate on human rights, all driven by a strong participatory approach involving a range of ASEAN sectoral bodies. The development of key policy instruments such as the ACWC Regional Guidelines and Procedures to Address the Needs of Victims of TIP supported by PROGRESS, and regular cross-sectoral meetings, such as the AICHR-SOMTC joint workshops contributed to the gradual adoption of a multi-sectoral approach to TIP across ASEAN. The implementation of policy in the AMS was catalyzed by the strong political commitment in each AMS and the collaborative efforts at addressing TIP. PROGRESS was the key party that promoted a collaborative approach at a regional level. According to informants: ´At the regional level, PROGRESS has been able to support engagement and collaboration with most difficult bodies within ASEAN. PROGRESS supported the collaboration process.µ 33 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV ´The collaboration between the United States and ASEAN is characterized by: a stable developed partner, in the form of the United States which has provided continued support to the agenda, the lead from AMS, the facilitation from ASEC.µ Non-USAID assistance that contributed the most to ASEAN policy ACTIP is regarded as the major achievement in the area of TIP with the major assistance coming from the AMS. The initial ideas for the establishment of ACTIP were tabled by some champions in SOMTC, such as the Philippines and Indonesia, and this was subsequently agreed by all AMS. The SOMTC Lead Shepherd on TIP of the Philippines led the process of the development of the ACTIP within the SOMTC mechanism. ACTIP is the first legally binding document on TIP in ASEAN and is acknowledged as a comprehensive convention that complies with the international standards. It is considered by some as better than the international convention on TIP, the Palermo Protocol. The efforts to come up with a legally binding document in ASEAN required significant political will and tangible commitment from the AMS. This is because ASEAN has for decades been reluctant to have binding documents in the area of TIP. In 2015, ACTIP was endorsed by the AMS, and it has now been ratified by nine out of 10 AMS. Some AMS are also in the process of amending their national TIP coordination mechanisms to align with ACTIP, such as the Philippines with its Inter-Agency Council Against Trafficking (IACAT). Following the establishment of ACTIP, AMS mainly contributed to the development of the first ever ASEAN multi-sectoral work-plan on TIP, the Bohol TIP Workplan 2017-2020. The Bohol TIP Work Plan is seen as another milestone for ASEAN, as it succeeded in engaging at least nine ASEAN sectoral bodies from all three ASEAN Community Pillars. The Bohol TIP Work Plan has shifted the ASEAN approach from its ¶traditional· security-based approach by the SOMTC that has guided TIP in previous years, toward a multi-faceted and comprehensive approach to combating TIP. With the Bohol TIP Work Plan in place, there have been gradual efforts and initiatives from ASEAN sectoral bodies to collaborate and work with one another. USAID assistance that contributed the least to TIP According to informants, the Victim Restitution Study in Thailand is considered to have contributed the least due to the lack of ownership from Thailand to extend the study to become an ASEAN agenda item. After the study was completed, no follow-up was undertaken on how to utilize and disseminate the study in Thailand or at the regional level. Most significant TIP achievements due to USAID assistance According to informants, the most significant achievement due to PROGRESS assistance was the adoption of the ACWC Regional Guidelines and Procedures to Address the Needs of Victims of TIP. These Guidelines provide a standard for ASEAN for both the sending and receiving countries for trafficking on how to deal with TIP victims. It has become a regional reference providing guidance and recommendations to AMS front-liners, such as immigration officers, social workers, and health workers, on how to assist TIP victims and what kind of services TIP victims need. A second major achievement due to PROGRESS assistance was that human rights-based approaches moved higher up the ASEAN agenda in TIP specifically and across the three ASEAN Community USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 34 Pillars. This led to the engagement of various ASEAN sectoral bodies from all three ASEAN pillars and a move away from the security-based approach that had guided TIP in previous years toward a multi-sectoral approach. These achievements were achieved in collaboration with the EU-funded Regional ASEAN-EU Dialogue Instrument Human Rights Facility (READI-HRF). YOUTH USAID assistance that contributed the most to ASEAN policy PROGRESS assistance that contributed the most included expertise for developing guidelines and initial financial assistance, including logistics, subsistence, and travel for youth participants, and expert speakers. This assistance was important to launch the youth initiatives of the ASEAN Foundation Model ASEAN Meeting (AFMAM), the ASEAN Youth Video Contest (AYVC), and the ASEAN Youth Social Journalist Contest (ACJC). These enabled the AMS sectoral body to build capacity and experience in running the programs. When USAID phased out its funding, The ASEAN Foundation was able to secure further funding from other sources to continue the implementation of the programs. Non-USAID assistance that contributed the most to ASEAN policy The ASEAN Youth Development Index (YDI) was developed using expertise from the United Nations Population Fund (UNFPA) based on a global index, with support from Malaysia, Indonesia and the Philippines. Other assistance has come from the International Federation of the Red Cross and UNICEF. The AMS, particularly Singapore, have also provided support to the ASEAN Youth Fund. USAID assistance that contributed the least to youth According to the informants, all forms of USAID assistance made important contributions to the area, as assistance was designed in close partnership with the relevant ASEAN stakeholders. Most significant youth achievements due to PROGRESS assistance According to the informants, the most significant achievement from PROGRESS assistance was the strengthening of the ASEAN Youth Agenda. This involved the development of supporting ASEAN￾level instruments, such as the AFMAM, the AYVC, the ACJC, and the ASEAN Youth Volunteers Programme (AYVP), enabling youth to engage in ASEAN issues and create awareness among their peers. Another major achievement was the establishment of a network of ASEAN youth who could network with other ASEAN youth to develop a common understanding on ASEAN issues, share ideas, highlight ASEAN values, advocate for ASEAN, and contribute toward ASEAN integration. According to one informant: ´USAID support helped us in the Flagship initiative of the ASEAN Foundation Model ASEAN Meeting. Students have developed a sense and understanding of ASEAN. Not many of the dialogue partners support work in this area.µ 35 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV 3.3.2 CONCLUSIONS USAID·s provision of neutral and international experts to facilitate dialogue and build understanding among various ASEAN stakeholders contributed to policy development and implementation across all areas. In the areas of AMDD, TMHS, AHTN, and ATISA, which have ASEAN policy instruments with legal provisions, the main forms of USAID assistance that contributed to the development and implementation of these policy instruments were ASEAN meetings, cross-sectoral events, studies and capacity-building efforts, including workshops, training and the training of trainers³all of which contributed in various ways to building political commitment. Many of these meetings and training activities were supported by USAID through the ACTI and PROGRESS projects, and involved experienced and neutral experts, and the use of strong participatory approaches. In the areas of SME development and youth³sectors not regulated by legal provisions at the ASEAN level³ACTI and PROGRESS, respectively, assisted in the development of many supporting instruments including the development of platforms through the provision of experts and support for ASEAN meetings and events. The main forms of non-USAID assistance that contributed to the development of policies were leadership from certain AMS as champions and implementation support from governments. Both USAID and non-USAID assistance contributed to the development of a 10-year strategic action plan for SME development and a five-year workplan for youth, with several supporting instruments for implementation at the ASEAN level, which elevated these areas higher up the ASEAN agenda. For WEE, the main USAID assistance that contributed to policy development included capacity building, support for events and meetings, and inputs to the strategy and workplans through experts and studies. The main non-USAID assistance that contributed included leadership from specific AMS, support from governments and the efforts of the national focal points. This assistance contributed to the establishment and operation of AWEN, the AWEN Strategy in the form of the ´AWEN Bangkok Declarationµ, and other supporting measures. In terms of non-USAID assistance, the leadership and support by lead AMS to initiate collaboration and support by all AMS in the drafting of policy in specific sub-areas such as TIP, TMHS, services and SME development, were important. Some AMS that had little or no legislation in some areas were able to ratify the ASEAN legal provisions faster than some of those AMS that already had legislation in place. In AMDD, TMHS, AHTN, trade in services and TIP, both USAID and non-USAID assistance contributed to the establishment of more comprehensive policy frameworks consisting of core and supporting policy instruments. For AMDD, TMHS, AHTN and trade in services, these developments enhance regional economic integration and trade through the harmonization of requirements, standards and the liberalization of priority services sub-sectors. In the area of TIP, there is a comprehensive policy framework for the AMS to counter TIP developed with non-USAID assistance. Guidelines developed by USAID help to address key issues, such as the handling of the victims of TIP. According to the findings, nearly all forms of USAID assistance contributed to the development and/or implementation of policy instruments in ASEAN, as they were identified based on specific requests and the needs from the ASEAN sectoral bodies and were designed in a participatory USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 36 manner. Forms of assistance that contributed to a lesser degree included certain studies that were not designed with a strong participatory approach. 3.3.3 RECOMMENDATIONS 1. According to the findings, many of the core policy instruments have been developed and implemented at the ASEAN level, while some supporting policy instruments have been developed in the priority areas of the evaluation. The evaluation team recommends that additional USAID/ASEAN assistance should focus on the development and implementation of important supporting policy instruments in order to achieve the right mix of policy instruments to be in place, which can enable effective ASEAN integration in the target sectors. Examples of supporting policy instruments include dissemination mechanisms, and mechanisms for strengthening the monitoring and reporting of ASEAN policy initiatives according to feedback from the informants. 2. The evaluation team recommends that USAID projects engage ASEC and ASEAN sectoral bodies in the approval process for experts that have been identified by the project. The CVs of experts should be shared with ASEC, sectoral body chairs or lead countries, with the latter facilitated by ASEC if needed for final approval, following best practices adopted by other dialogue partners for maximum buy-in. 3. Where a more programmatic line of support is required, for example to commit funds to a specific area, the evaluation team recommends that USAID/ASEAN programming aligns closely with the specific sectoral body work plans or strategic action plans to ensure maximum relevance and ownership by ASEAN stakeholders. 3.4. EVALUATION QUESTION 4: WHAT ROLE DID GENDER PLAY IN THE DESIGN AND IMPLEMENTATION OF ACTIVITIES? WHAT IMPACT DID THIS ROLE HAVE? Synthesis At a project level, the role of gender aimed to ensure balanced gender representation among participants and trainers at workshops, training events and meetings. Gender-mainstreaming approaches were also applied by supporting initiatives in WEE and TIP. These approaches achieved notable success that can support ASEAN in moving toward a more integrated and cross￾cutting gender mainstreaming approach. This section discusses the design and implementation of gender approaches in the PROGRESS and ACTI projects. The discussion starts with the main policy instruments and institutional arrangements to better contextualize gender initiatives in ASEAN. It then looks at its impact through USAID support to ASEAN, followed by an analysis of how gender was mainstreamed in the two projects, together with recommendations for the future. 37 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV 3.4.1. FINDINGS POLICY DEVELOPMENT PATH OF GENDER IN ASEAN The diagram below illustrates the policy development path for gender, from its inception in 1975 through to implementation. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 38 Figure 5. Policy Development Path for Gender 39 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV The promotion of gender equality in ASEAN began in 1975 with the establishment of the ASEAN Sub-Committee on Women (ASW). The ASW established the ASEAN Confederation of Women·s Organizations (ACWO) to strengthen the role of women in development20. The first major ASEAN instrument on gender, the Declaration on the Advancement of Women in the ASEAN Region (1988), was endorsed by the AMS Foreign Ministers. At the national level, the Philippines, Lao PDR, Vietnam, Indonesia and Thailand ratified the UN-sponsored Convention on the Elimination of All Forms of Discrimination Against Women (CEDAW) in 1981, followed by Cambodia and Myanmar21. While Brunei has signed the CEDAW, it has yet to ratify it. CEDAW is recognized by ASEAN as the ´basic and foundational framework for gender equalityµ22. In 2001, the ASW was elevated to a committee, the ASEAN Committee on Women (ACW). The ACW led the development of several ASEAN policy instruments, including the Declaration on the Elimination of Violence Against Women in the ASEAN Region (2004) and other instruments that are summarized in Figure 6. In 2011, the ASEAN Ministerial Meeting on Women (AMMW) was established under the ASCC, consisting of the ministers responsible for issues affecting women and girls across the AMS. The AMMW has a mandate to set strategic policy direction on ASEAN·s regional cooperation on women. Other entities dealing with gender equality include the ACWC established in 2010, with a mandate23 to promote and protect human rights, fundamental freedom, well-being, development, empowerment and participation of women and children. The ACWC reports to the ASEAN Ministerial Meeting on Social Welfare and Development (AMMSWD). The ACW coordinates efforts and synergies with the ACWC through the ACW-ACWC Consultation Meeting and a joint ad-hoc working group on gender mainstreaming involving the ACW and ACWC. The Poverty Eradication and Gender Division at ASEC acts as the secretariat for gender efforts, particularly those under the ACW. STATUS OF GENDER EQUALITY ACROSS AMS An official ASEAN study24 on the AEC found that: x Although both intra-ASEAN and extra-ASEAN trade have been expanding, the value of women’s share of exports has remained constant for the past 20 years; x The share of high-skilled sectors in ASEAN economies has been increasing, whereas sectors in which women dominate (such as agriculture) have remained stagnant; 20 Overview ASEAN Ministerial Meeting on Women, ASEAN Secretariat ² asean.org 21 The Status of CEDAW Implementation in ASEAN Countries and Selected Muslim Countries, 2004, International Women·s Rights Action Watch Asia Pacific. 22 ASEAN Declaration on the Gender-responsive Implementation of ASEAN Community Vision 2025, issued in 2017. 23 ACWC Terms of Reference, 2010. 24 Projected Gender Impact of the ASEAN Economic Community, 2016, ASEAN Secretariat. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 40 x More women are employed in lower-skilled and lower-paying jobs than men, resulting in a persistent and high gender wage gap; x The women labor force participation rate is persistently lower than that for men across all AMS; and x Although the AEC is creating growth opportunities for MSMEs, cultural and financial barriers persist for women business owners. The study highlighted that one barrier to women·s participation in AEC growth opportunities is the limited effectiveness of gender-mainstreaming policies and laws due to cross-sectoral coordination issues, a lack of capacity among implementers, and silo-based approaches to gender. A more recent study indicates that gender inequalities continue to exist across ASEAN25. The study found that the women labor participation at 56.5 percent rate is much lower than that of men, at 82.0 percent. Women are paid about 30 to 40 percent less than men, and 63 percent of women across the region work in jobs that are classified as ´vulnerableµ26. Women have less access to land and financial services, with only 16 percent of women having ever borrowed money from a financial institution, while women also have more limited access to technology. In recognition of the gender inequalities, cross-sectoral coordination issues and silo-based approaches to gender, the ASEAN Declaration on the Gender-responsive Implementation of ASEAN Community Vision 2025 and the Sustainable Development Goals (2017) was signed by Leaders of all AMS. This policy instrument provides nine commitments toward the ´gender-responsive implementation of the ASEAN Community Vision 2025µ. This is a significant policy milestone, which goes beyond the ASCC and targets all three ASEAN Community Blueprints (pillars), namely the ASCC, the AEC and the APSC. Most AMS stipulate constitutional equality between men and women. Furthermore, several ASEAN policy instruments and institutional arrangements are in place to support the gender agenda. However, according to many informants interviewed by the evaluation team, gender efforts largely remain within the domain of a few sectoral bodies, such as the ACW and ACWC, with limited gender mainstreaming across other bodies or in other pillars, particularly the AEC pillar. A review by the evaluation team of the AEC 2025 Consolidated Strategic Action Plan (2018, ASEC) for all AEC sectors from 2016 to 2025, revealed gender aspects in only two measures out of 153. These challenges are reflected in the findings below and the scope for gender mainstreaming that could be addressed by the USAID projects evaluated. 25 ASEAN Support Project Appraisal: Gender Analysis, December 2017, Management Systems International under the USAID/MESP project. 26 ILO defines vulnerable workers as those employed under precarious circumstances who are likely have low wages, no access to social protection, and lack decent working conditions. 41 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV Gender in the PROGRESS Project The evaluation found that the PROGRESS project supported work on gender mainstreaming in both the TIP and youth areas. Gender was a major component during the design and implementation of TIP activities. This was accomplished mainly through a partnership formed between PROGRESS and the ACWC. The ACWC was responsible for specific work plan activities focusing on gender equality and children27. PROGRESS also worked in partnership with the AICHR and SOMTC. PROGRESS· work in this area resulted in the outcomes summarized below28. x The Regional Review on Laws, Policies and Practices on the Identification, Management and Treatment of Victims of Trafficking contributed to a greater understanding of the gaps in the legal frameworks that deal with women victims of TIP. x The development of the ACWC Regional Guidelines and Procedures to Address the Needs of Victims of TIP provides comprehensive guidelines to the AMS on handling TIP victims, especially women and children. For example, in the Philippines, the Inter-Agency Council Against Trafficking issued a memo for the use of these guidelines by the police, social workers, and prosecutors. The support provided by PROGRESS helped to ensure that gender played a major role in the design and implementation of TIP activities by a range of sectoral bodies29. PROGRESS incorporated some gender perspective in its support of youth activities. These efforts focused on promoting gender balance in the team composition for the ASEAN Foundation Model ASEAN Meeting (AFMAM), and in the final selection of participants for the ASEAN Youth Video Programme and the ASEAN Youth Citizen Journalism Contest. By the end of the project, PROGRESS had supported a total of 2,380 women (over 65 percent of them women youth) and provided them with increased knowledge, leadership capabilities and technical skills30. Informants indicated that the results illustrated above could be improved if USAID were to broaden its mandate on gender mainstreaming in future projects. They suggested that a comprehensive capacity-building program, such as gender-mainstreaming training, would be important, as many ASEC and ASEAN sectoral body staff are not fully aware of gender issues and approaches, given that they view gender mainstreaming simply as an exercise in the equal representation of men and women in activities or events. This capacity building could adopt a training-of-trainers model and can cover all parties involved, including the implementing partners, ASEC and ASEAN sectoral bodies, to ensure coherence and consistency in gender mainstreaming. Particularly on TIP, training could be given to specific frontline staff, such as women police officers, for improving the treatment and handling of women TIP victims. To improve gender equality in youth initiatives, the informants mentioned that the gender strategy could, for example, be better aligned with the UN·s gender strategy. 27 Interviews with ASEC, SOMTC and AIHCR. 28 Interviews with ASEC, SOMTC and AIHCR. 29 Interviews and PROGRESS Final Report 2018. 30 PROGRESS Final Report 2018. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 42 3.4.2. GENDER IN THE ACTI PROJECT Most of the informants involved in trade facilitation and services indicated that, to their knowledge, gender had not played a major role in these areas. They felt that these areas were highly technical and gender neutral, thus narrowing the scope of gender. Gender-mainstreaming efforts by the ACTI project faced additional challenges as its activities targeted the AEC pillar, whereas gender efforts in ASEAN are anchored around the ASCC pillar. As a result of these challenges and the limited scope for gender mainstreaming in these technical areas, ACTI focused its efforts on encouraging women participants and trainers at workshops, training events and official ASEAN meetings. This approach had some impact, as ASEAN sectoral bodies have since started to adopt this approach at their own events. However, the evaluation found that gender can play a more significant role in both the trade facilitation and services areas. Informants from CLMV countries31 felt that gender was an important consideration when it came to the labeling requirements for TMHS, especially for pregnant and lactating women. One informant felt that job creation and WEE should be integrated as a priority in the services area, as the sector includes many sub-sectors that could provide employment and entrepreneurship opportunities for women. In the area of SME development supported by ACTI, gender played a significant role, particularly in WEE. This included: support for two white papers on WEE (one on the digital economy in ASEAN and the other on empowering grassroots women in Myanmar); a women·s science prize; a Gender Mainstreaming Study in the AEC; support for the establishment and operation of the AWEN; and support for the expansion of the GWAI. Moreover, the studies and events supported by ACTI contributed toward greater awareness of the importance of WEE in SME development, which in turn led to the inclusion of WEE as an objective and strategic action in the SAPSMED 2016-2025 of the ACCMSME. ACTI produced significant results by supporting AWEN and GWAI32, as described in Box 1. Despite these efforts and results, informants felt that additional work was needed to mainstream the gender lens across ASEAN sectoral bodies, particularly in the economics pillar, the AEC. Several informants recommended awareness and capacity building through a gender ´101µ program on the fundamentals of gender and the opportunities for gender-differentiated approaches. According to the informants, the Gender Mainstreaming Study in the AEC, supported by ACTI, failed to have a significant impact, as it did not provide useful recommendations on how to address gender mainstreaming across the various sectors of the AEC. The informants recommended that, in order to develop a strong gender mainstreaming strategy, more work was needed to generate gender￾disaggregated data in ASEAN, especially in core areas such as MSMEs and the services sector. This would help to differentiate and highlight issues that are faced only by women, as opposed to those issues that are faced by both men and women and enable the development of more targeted interventions. 31 Cambodia, Lao PDR, Myanmar and Vietnam. 32 Interviews with AWEN informants and ACTI Final Report 2018. 43 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV Box 1. Gender and Women Empowerment through the ACTI Project 3.4.3. CONCLUSIONS Despite the development of several policy instruments and institutional arrangements for gender at the ASEAN and AMS levels, gender mainstreaming initiatives remain a challenge due to issues of cross-sectoral coordination and silo-based approaches. In the case of the PROGRESS project, gender played a significant role in the design and implementation of TIP activities, driven by collaboration with the ACWC, leading to visible results of gender mainstreaming in TIP efforts. In youth, efforts aimed at achieving balanced gender representation in youth activities were also put in place. In the ACTI project gender played a significant role in the design and implementation of ACTI support to WEE in the SME development area. This led to the launch and sustainable operationalization of AWEN and direct capacity building of women entrepreneurs, and helped to bring the AEC pillar closer to AWEN, which is usually placed (or mis-placed) under the ASCC pillar. It also enabled the GWAI to become an ASEAN initiative, and created awareness of women entrepreneurs among ASEAN Economics Ministers (AEM) and ASEAN Leaders. Third-party studies indicate that the establishment of AWEN was an important milestone in the advancement of WEE across the region, given the platform it provides and its policy advocacy role. ASEAN Women Entrepreneurs Network (AWEN) ACTI·s support to AWEN led to clear impacts in the sustainable operationali]ation of AWEN, such as the launch of AWEN, the development of AWEN·s strateg\, the conduct of AWEN meetings, the setting-up of AWEN focal points in the AMS, and training and capacity building to encourage women entrepreneurs and MSME awards. ACTI also created greater awareness and visibility on women entrepreneurs and AWEN among ASEAN Sectoral Bodies, particularly in the AEC. This led to AWEN having a seat in the ASEAN Business Advisory Council (ABAC). This helped bring the AEC pillar closer to AWEN and women entrepreneurs who are traditionally placed under the ASCC. ACTI also contributed to the Action Agenda on Mainstreaming WEE in ASEAN, through recommendations. This was reviewed by the AEM and approved by the ASEAN Leaders in 2017, gaining visibility among the political leadership of the AMS. ACTI contributed to the development of the AWEN Bangkok Declaration, its main strategy, which was presented to the ASEAN Chair, the Prime Minister of Thailand. A study by OECD* concludes that the launch and establishment of AWEN was an important step forward in the region for women entrepreneurship, as it serves as a platform for the promotion, development and empowerment of women entrepreneurs in ASEAN. The study also states that AWEN plays an important policy advocacy role in creating a favorable business environment to support women·s entrepreneurship in ASEAN and in the AMS through public-private policy dialogue. Great Women in ASEAN Initiative (GWAI) ACTI enabled GWAI to expand from a national to a regional initiative with a network of ASEAN entrepreneurs, and it enabled women entrepreneurs to meet and discuss challenges and opportunities in specific areas, such as coffee, as part of an AWEN initiative. *Strengthening Women·s Entrepreneurship in ASEAN: toward increasing women·s participant in economic activity, September 2017, OECD. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 44 3.4.4. RECOMMENDATIONS Based on the findings, the evaluation team recommends the following: 1. USAID should continue its support to AWEN as it provides the strongest platform available for WEE in ASEAN. Specific support measures can include support for the establishment of an ad-hoc or permanent working group on WEE, involving ACCSMSE and AWEN, and the conduct of policy dialogue events that would strengthen the ability of AWEN to contribute toward MSME policy development, which in turn can support WEE. 2. USAID should leverage its strong track record in supporting the development of harmonized standards in ASEAN to support collaboration between ACCSQ and AWEN to harmonize ASEAN standards for women-majority sub-sectors, such as coffee and textiles. Gender opportunities in other standards work should also be explored. 3. Capacity-building programs should continue for AWEN in areas such as Industry 4.0, company restructuring, digitalization and access to financial instruments such crowdfunding. Other support should be provided to enable AWEN to become recognized as an official ASEAN sectoral body. 4. Future USAID programming should explore facilitation for the collection and analysis of gender-disaggregated data in ASEAN for areas such as MSMEs and selected sub-sectors in services. This could take the form of collaboration with ASEAN Statistics, the ACCMSME, the CCS and AWEN. Such data could be used to generate analytical reports or white papers to support informed decision-making in ASEAN. This is a long-term process that can be catalyzed by USAID, through support for a series of initial cross-sectoral meetings or policy dialogue events focusing on the collection and analysis of gender-disaggregated data for the priority ASEAN integration areas with the involvement of other dialogue partners. Additional support can include expertise and guidelines for the AMS to gather gender￾disaggregated data. 5. USAID/ASEAN programs can also support capacity building for gender mainstreaming as part of their design. This should include training of all implementing partner staff (IGNITE/PROSPECT) in gender mainstreaming, the training of trainers, and the establishment of gender-mainstreaming champions among the implementing partners, ASEC and ASEAN sectoral bodies. All workstreams should include awareness sessions on gender￾mainstreaming principles when conducting workshops and relevant events, to strengthen awareness among staff from ASEC, ASEAN sectoral bodies, the AMS, and youth and business leaders. The ASEAN Declaration on the Gender-responsive Implementation provides a policy instrument to support gender mainstreaming across ASEAN sectors. Short-term technical advisor (STTA) and long-term technical advisor (LTTA) reporting templates should include a section on gender mainstreaming (needs, opportunities and efforts). 6. Gender considerations should also be considered in future support for medical devices such as IUDs to better suit the physiology of Southeast Asian women, according to recommendations from the respondents from these countries. 45 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV 3.5. EVALUATION QUESTION 5: WHAT LESSONS LEARNED IN STRATEGY AND APPROACH CAN AND SHOULD BE APPLIED TO FUTURE PROGRAMMING? Synthesis The ability of USAID projects to respond to demand-driven requests is viewed as successful but improvements can be made in involving ASEC and the ASEAN sectoral body chair or lead country in the final approval of experts selected. Programmatic lines of support should align closely with the specific sectoral body work plans or strategic action plans to ensure maximum ASEAN relevance. At a program level, measures to improve the effectiveness and value of USAID assistance are recommended. To ensure that the USAID support provided for ASEAN policy development leads to successful implementation, targeted support for the AMS is required. This can be enhanced by mainstreaming an ASEAN element in relevant USAID support to the AMS provided by the bilateral missions. Other support tools can include the establishment of a USAID regional-bilateral coordination and monitoring mechanism. Overall, the best entry point for USAID assistance to ASEAN remains ASEC and specific capacity-building support to ASEC is recommended as part of USAID·s strategic and long-term approach to ASEAN. Other recommendations include greater use of cross-cutting measures, such as youth, gender and social governance across USAID/ASEAN support where relevant to support the goals of the USAID￾ASEAN Development Cooperation Framework and the ASEAN Vision 2025. Specific sectoral recommendations are provided. Specifically, in regard to TIP, sectoral bodies, lead countries or champions may often provide the best entry point. 3.5.1 FINDINGS Demand-Driven and Program-Based Assistance The findings from many of the informants confirmed that a major strength of USAID support to ASEAN was its ability to respond to demand-driven requests from ASEC and ASEAN sectoral bodies relatively rapidly. Future USAID/ASEAN programming should continue to incorporate these demand-driven features in the design of programs. Improvements can be made through a more participatory process of approving experts. Many informants from ASEC and the sectoral bodies felt that, in some cases, the experts identified by the PROGRESS and ACTI projects had general knowledge but insufficient expertise to match the specific needs of ASEAN. This limited the effectiveness of the assistance. Most informants felt that it was important for ASEC and either the chairs of the sectoral bodies or the lead countries to review and approve experts identified by PROGRESS and ACTI. Balancing Regional- and National-Level Support As ASEAN and its dialogue partners, such as the United States, have developed core policy instruments in key areas, much of the attention will shift to policy implementation. The analysis in evaluation questions 1, 2 and 3 revealed that much of the implementation efforts are carried out at the AMS level. Thus, to ensure the implementation of ASEAN policies, the AMS need to be provided with targeted technical assistance when needed according to the informants. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 46 Value of Youth Support Initiatives The findings reveal that although the youth sector does not have ASEAN policy instruments with legal provisions, it nonetheless contains policy instruments such as five-year work plans and the ASEAN Youth Development Index. The youth sector has sound institutional arrangements in place to enable the effective engagement of a range of stakeholders, including AMS governments, education institutes and the private sector. Lessons on how youth initiatives can contribute to policy implementation are provided in Box 2 for one of the activities carried out in the youth work stream in the Philippines, and the acknowledgement received from the AMS government. USAID should consider supporting youth activities in ways that help to advance priority policy support areas. Box 2. ASEAN Citizen Journalism Contest 3.5.2 RECOMMENDATIONS FOR FUTURE PROGRAMMING PROGRAMME LEVEL RECOMMENDATIONS Balancing Regional- and National-Level Support To accelerate the implementation of ASEAN policies in the AMS, support for national-level capacity￾building initiatives and specific technical assistance to CLMV countries should be further programmed, where feasible. This can build on the experience from the dedicated U.S.-ACTI Economic Reform and ASEAN Integration (U.S.-ACTI-ERA) program for Myanmar. The support could be channeled by the USAID bilateral missions. One approach would be to examine how AMS ASEAN policy implementation links to the development objectives of USAID ´We were one of the finalists of the ASEAN Citi]en Journalism Contest and attended a USAID supported workshop. We saved up money from our workshop allowance to fund the costs for our project in the Philippines including travel and FB ads to promote our project and printed our materials. We also had volunteers. We wanted to create awareness about transparency in government poverty alleviation programs, to track child welfare social disbursement based on their school attendance. We were approached by the Philippines government (Department of Social Welfare) to find out more about our project. We were also approached for an interview, facilitated by the Department of Foreign Affairs, to promote our achievements among the netizens. The department of Social Welfare also signed an MoU with us based on the work we were doing and the impact were creating. They offered to provide us with additional training. We were also interviewed by the Philippines News Agency and the Ministry of Foreign Affairs to share with other netizens on the work we were doing. We have over 70,000 likes and 2.1 million people following us. Our FB page has turned into a help desk. We have 56 pages of list of people who have reached out to get help. We went out to the communities. We provided support to families who wanted to be part of the programme. We conducted interviews.µ ASEAN Citizen Journalism Contest Participant Philippines 47 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV bilateral support, so as to develop a coordinated approach to assistance. Further, all USAID bilateral support to the AMS can include an ASEAN element in relevant areas such economic growth, good governance and human rights, including specific support elements to help the AMS fulfil their ASEAN requirements in priority areas. There are numerous benefits of this approach, including supporting AMS integration within the ASEAN Community, enhancing coherence of USAID support, and supporting alignment with the U.S.-Indo-Pacific Strategy. These benefits can be communicated to the bilateral missions through the meetings of the ASEAN Champions. The ASEAN Champions mechanism, established by USAID/ASEAN, can be used to strengthen coordination in USAID support to the AMS and ASEAN. Other support tools can include the establishment of a USAID regional-bilateral coordination and monitoring mechanism, building upon lessons from other dialogue partners such as the European Union (EU), which would help consolidate the support provided at the regional and AMS levels, and support reporting and learning through the ASEAN Champions initiative of USAID/ASEAN. Institutional Entry Points Based on the ASEAN institutional arrangements in place, the relevant division from ASEC remains the most suitable entry point for USAID assistance in each sector. Where feasible, engagement with sectoral bodies, particularly the Chair or the lead country of the body is recommended to enable the USAID implementing partners to better understand the needs and constraints of the entities that formulate and implement policies. A continued and permanent presence of all USAID/ASEAN projects in ASEC is recommended, to improve the visibility and position of USAID/ASEAN projects as an ´internalµ resource of ASEC. A more permanent presence can be implemented through the establishment of an ´ASEAN-USAID Cooperation Officeµ space for USAID projects, building on the practices of other dialogue partners. Capacity Building Support for the ASEAN Secretariat ASEC plays the key coordination role across all ASEAN activities. This includes the convening of regular sectoral body meetings, the development of meeting agendas, the consolidation of reporting and monitoring efforts by sectoral bodies and coordinating efforts between dialogue partners. One of the challenges facing ASEC, according to the informants, is the relatively high staff turn-over and the limited number of staff available. Major dialogue partners such as the EU have addressed this challenge33 by establishing a specific capacity-building grant for ASEC targeting human resource development and operational continuity. This provides financial support for the employment of ASEC staff in key areas where there is cooperation between the EU and ASEAN, and where there are recognized capacity gaps. Future USAID/ASEAN programming should consider the integration of such a support mechanism in targeting human resource development and operational continuity in 33 Annual Action Programme 2016 (Part III) in favour of the Asian region to be financed from the general budget of the European Union and Asia Regional Multi-annual Indicative Programme 2014-2020, European Union. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 48 ASEC in specific areas as part of USAID·s strategic and long-term approach to ASEAN and supporting alignment with the U.S.-Indo-Pacific Strategy. Results-Oriented Monitoring, Evaluation and Learning Project monitoring tools such as feedback forms should increasingly focus on relevance, contribution to ASEAN integration goals and sustainability measures, with less emphasis on logistics and administration. Project monitoring systems should include a compilation and categorization of all outputs generated by a project, to supplement the work done on activities and reflected in annual reports. Outcome assessments at the mid-term should support learning and adaptation by projects. Informants from services, SME development, WEE, Youth and TIP support areas mentioned that a prominent gap is the lack of adequate monitoring and reporting mechanisms and capacity to assess the effectiveness of ASEAN policies and to identify if remedial action is required. USAID/ASEAN should explore the provision of assistance to sectoral bodies for Stage 11 of the policy continuum, ´Monitoring and Reportingµ. Support can include the development and implementation of robust M&E mechanisms and appropriate capacity building of staff from the sectoral body and relevant ASEC division. Youth Mainstreaming USAID/ASEAN projects should explore the use of youth as an enabler for outreach and advocacy in areas such as human rights, TIP, good governance, SME development, ICT and green technologies. This can build on practices from activities such as the Youth Human Rights Debate (supported by PROGRESS). SECTORAL LEVEL RECOMMENDATIONS AHTN-SPECIFIC RECOMMENDATIONS Continued support by USAID for the revisions to the AHTN should be considered in specific areas given the strategic nature of this work dealing with customs procedures, rules-of-origin tariffs and sectors such as industry and trade, health, food and drugs. According to the informants, ASEAN is strengthening its dispute settlement mechanism, and this is an area in which USAID could provide support. An additional area for support requested by informants is for capacity building to understand product development characteristics, which will help customs authorities to classify these in the coding system. AMDD-SPECIFIC RECOMMENDATIONS Specific topics requiring additional support identified by the informants include the classification of medical devices, safety evaluation procedures, and how to regulate new devices such as software devices and in-vitro diagnostic (IVD) devices. The trainers should ideally be former regulators and can include classroom-based training supplemented by site visits to medical device companies. Cost￾sharing of such support was indicated as a possibility by some AMS representatives. TMHS-SPECIFIC RECOMMENDATIONS Building on the success from the capacity-building support provided by ACTI for two out of 10 technical annexes (GMP and Safety Substantiation), USAID through IGNITE should explore the 49 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV provision of support for the implementation of remaining eight technical annexes, where gaps remain, to complement the support provided by the AMS. SERVICES-SPECIFIC RECOMMENDATIONS AFAS commenced implementation in 1997 and is due for complete implementation in 2021. The experience from AFAS indicates that although there was high level endorsement, it was not effectively implemented at the AMS level according to informants. Specific areas of support that USAID may consider, to contribute to more effective implementation of the new agreement ATISA, includes capacity building and strengthening of the legal framework in areas such as eCommerce particularly for supporting services such as registration of online businesses, order fulfilment, distance selling regulations and continued support on eBanking. Other areas for support requested include strengthening of public outreach instruments on services including the development of case studies and leaflets to create more awareness. SME-SPECIFIC RECOMMENDATIONS The ASEAN SME Academy (Academy) has not fulfilled its potential due to a relatively low number of active users. Building on the commitment by the Philippines to host the Academy and to justify the initial investment by USAID ACTI, the evaluation team recommends additional USAID support to create outreach for the Academy. This should include the development of a dissemination and socialization strategy to be implemented by the 10 AMS covering all agencies involved in SME development, as well as key private sector players and education institutes. The support can include facilitation to establish a strategic partnership between the Academy, and relevant SME education and training institutes in each AMS, such as INSKEN in Malaysia, and the SME Centre in Lao PDR and private sector institutes. The update of the training modules should also be facilitated, and the Academy can serve as a platform for other training modules developed by ASEAN stakeholders, such as the ASEAN Tourism Curriculum. According to the findings, a major gap to support the policy development path in this area is the lack of good quality data on MSMEs in ASEAN. A priority area for support should be the provision of expert advice to develop a database of better-quality data on MSMEs in ASEAN, including the use of annual surveys on MSMEs. These can be undertaken with the support of the ASEAN Statistics Division and with AMS representatives from ACCMSME and the statistics agencies. The availability of better-quality data will enable improved analysis of MSME needs, challenges and opportunities, which can drive the design of targeted MSME support in the ACCMSME workplans. Such data can also help USAID projects such as IGNITE to be more demand-driven and identify priority areas that require support including in WEE. Additional areas of capacity-building support requested by sectoral body representatives include: branding for micro-enterprises, the engagement of youth in digital economy start-ups, the engagement of youth in green economy start-ups, and the development of cooperatives for women and youth entrepreneurs. TIP-SPECIFIC RECOMMENDATIONS One very well-placed informant suggested that providing capacity building on the use of regional guidelines in all agencies might be a constructive way to build a closer relationship with SOMTC and ACWC. The evaluation team recommends this option be explored by channeling the assistance USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 50 through the SOMTC Lead Shepherd on TIP. This could help SOMTC to pursue and register this as an accomplishment toward the implementation of Article 24 of ACTIP. Assistance would need to be developed to link with SOMTC·s work plan and mandate. Also, as the Bohol TIP Work Plan is a SOMTC priority through 2020, this could be used as an entry point to work with sectoral bodies. USAID could approach AMS to gauge support for such support and, if present, could assist to develop a project proposal, which could be considered by SOMTC. Additional studies might also be useful, for example researching the challenges and needs regarding the prosecution of TIP cases, or a review of judicial systems to better understand prosecution rates according to some informants. Initiatives to support greater engagement of the private sector and CSOs are also recommended, such as support to establish platforms between ASEAN sectoral bodies dealing with TIP and AEC entities such as the ABAC and others. This would help to create awareness on TIP and could also be extended to include responsible governance in business. Ongoing training activities in PROSPECT targeting front-line stakeholders should particularly target women staff, including women police officers, as these are usually the staff dealing with TIP victims. A major gap identified by the informants is the lack of a suitable outcome monitoring mechanism in TIP, which would generate results on an annual basis and be assessed at the end of the current cycle of the ASEAN Community Blueprints 2025 for the ASCC and ASPC. The evaluation team recommends that PROSPECT explore the provision of capacity-building support to ASEC and the sectoral bodies involved in TIP to strengthen their monitoring mechanisms, particularly at the outcome monitoring and impact levels. This should include the development of outcome and impact indicators, data-collection templates and reporting templates for use by relevant sectoral bodies involved in the area to report on, and enable ASEC to aggregate the data through their sectoral monitoring divisions under the ASCC and the APSC pillars. Some of this support could feed into the development of the national representative monitoring and reporting system, which is currently being defined for ACTIP. Some informants highlighted that, with policy instruments such as ACTIP and the Bohol TIP Work Plan in place, there is a need for ASEAN to engage and collaborate with a broader set of stakeholders, including relevant sectoral bodies across the three pillars, CSOs and dialogue partners. The need for wider collaboration within the complexity of ASEAN·s organizational structure and mechanism requires capacity to be able to coordinate effectively. In this context, facilitation for ASEC·s role in coordinating multi-sectoral work on TIP is essential, building on USAID·s success and experience in adopting strong participatory and cross-sectoral approaches. YOUTH-SPECIFIC RECOMMENDATIONS ASEAN dialogue partners such as the EU, China, the Rep. of Korea, Australia and India see the value in the youth area and have regularly provided support in youth development. The goal of USAID·s Development Cooperation Framework is to help ASEAN contribute to more sustainable and inclusive growth in the region, along with helping to build supporting rules-based systems to support 51 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV a more prosperous Southeast Asia34. The evaluation team recommends that support for youth should focus efforts on nurturing ASEAN values, culture and identity, which should reflect these three goals. Efforts to better leverage on youth as advocates for ASEAN should be explored due to the potential of youth in creating awareness of ASEAN issues and opportunities among peers and at grass-roots levels, which in turn contributes to the broader goals of an ASEAN Community. It also provides an efficient channel to create significant visibility for USAID as a trusted and credible dialogue partner in ASEAN. Assistance to facilitate an active ASEAN Youth Alumni, as a platform to create awareness and advocacy, should be provided, building on the significant number of youth participants at USAID events. USAID should also consider supporting youth activities alongside priority policy support areas. 34 USAID-ASEAN Development Cooperation Framework, 2017, USAID/ASEAN Office. 53 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV ANNEX 1. SCOPE OF WORK Scope of Work Retrospective Performance Evaluation of U.S.-ASEAN ACTI and PROGRESS I. Introduction The United States Agency for International Development (USAID) intends to conduct a whole-of￾office retrospective performance evaluation for two completed programs in the ASEAN portfolio, namely U.S.-ASEAN Connectivity through Trade and Investment (ACTI) and ASEAN-U.S. Partnership for Good Governance, Equitable and Sustainable Development and Security (PROGRESS). ACTI provided support to accelerate selected components of the ASEAN Economic Community (AEC), while PROGRESS supported ASEAN·s Political-Security and Social-Cultural Communities through targeted support to key ASEAN sectoral bodies. USAID/ASEAN has tasked the USAID Indonesia Monitoring and Evaluation Support Project (MESP) to conduct the final retrospective evaluation of ACTI and PROGRESS. The goal of the final evaluation is to provide USAID/ASEAN with an independent review of the performance of the two Activities highlighting the achievement of expected results, factors influencing the achievement of these results and challenges faced by both programs over the course of their implementation. The final evaluation will also provide strategic recommendations for current programming and offer insights for the design of future programming. II. Background: U.S.-ASEAN Activities The Association of Southeast Asian Nations (ASEAN)35 is Southeast Asia·s primary multilateral institution for regional cooperation. Founded on the vision of a rules-based order, ASEAN provides a platform for economic cooperation, the peaceful arbitration of disputes, and the promotion of the rule of law to ensure a free and open Southeast Asia. USAID/ASEAN, based in Jakarta, builds the capacity of ASEAN and its Member States (AMS) to contribute to sustainable and inclusive growth and encourage rules-based systems to support a prosperous Southeast Asia. USAID/ASEAN previously funded two projects: (i) ACTI, a $20.9 million project that ran from June 2013 ² December 2018; and (ii) PROGRESS, a $14.5 million project, which ran from September 2013 ² September 2018. ACTI supported ASEAN in the areas of customs modernization, the ASEAN Single Window (ASW), trade and investment facilitation, small and medium-sized enterprise (SME) development, energy efficiency, and information and communication technologies (ICTs). The project also worked to deepen the ASEAN-U.S. economic relationship, as well as well provided direct assistance to AMS, particularly through the two-year US-ACTI Economic Reform and ASEAN Integration (US-ACTI￾ERA) program in Myanmar (2013 to 2016). ACTI was organized around the following four work streams: 1) ASW: technical and legal development of the ASW, along with public outreach. 35 Brunei, Cambodia, Indonesia, Lao PDR, Malaysia, Myanmar, Singapore, the Philippines, Thailand and Vietnam. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 54 2) Trade and Investment Facilitation: standards harmonization, trade transparency, services liberalization, and investment facilitation. 3) SME Development: policies to make SMEs more competitive by encouraging greater integration of SMEs into regional and global supply chains, providing access to information, and expanding access to finance, with a special focus on resources for women and young entrepreneurs and firms in Cambodia, Lao PDR, Myanmar, and Vietnam. 4) ICTs for Development: assistance to ASEAN in implementing the ICT Master Plan, focusing on technical and financial issues related to broadband expansion in primarily rural areas, as well as information security and e-commerce. ACTI worked with the ASEAN Secretariat, ASEAN Member States, U.S. government agencies, the private sector, and other stakeholders. It also worked with the following ASEAN bodies: x ASEAN Secretariat·s ASW Project Management Office x Senior Officials Meeting for ASEAN Ministers on Agriculture and Forestry (SOM-AMAF) x ASEAN Sanitary and Phyto-sanitary Contact Points (ASCP) x ASEAN Senior Officials on Rural Development and Poverty Eradication (SOMRDPE) x ASEAN Coordinating Committee on MSME (ACCMSME) x ASEAN Women Entrepreneurs· Network (AWEN) PROGRESS supported ASEAN·s Political-Security and Social-Cultural Communities through targeted support to key ASEAN sectoral bodies. PROGRESS was organized around three broad components: (i) advancing good governance and political-security cooperation; (ii) improving ASEAN·s capacity to promote equitable and sustainable human development; and (iii) strengthening ASEAN institutional capacity. Within this framework, PROGRESS focused on seven work streams: 1) Transnational crime and non-traditional security threats, with a focus on trafficking in persons (TIP) 2) Human rights 3) Executive, judicial, and legislative networks (including support to the ASEAN Chair) 4) Science-based policy making 5) Disaster risk reduction and management 6) Women·s and children·s rights 7) Public outreach, youth engagement, and strengthening ASEAN institutions PROGRESS supported capacity building within ASEAN by working with the following bodies: x ASEAN Secretariat (ASEC) x ASEAN Intergovernmental Commission on Human Rights (AICHR) x Senior Official Meeting on Transnational Crime (SOMTC) x ASEAN Commission on the Promotion and Protection of the Rights of Women and Children (ACWC) x Senior Official Meeting on Youth (SOM-Y) x ASEAN Humanitarian Assistance (AHA) Center x ASEAN Committee on Disaster Management (ACDM) x ASEAN Inter-Parliamentary Assembly (AIPA) x Council of ASEAN Chief Justices (CACJ) III. Purposes, Audience and Intended Use The Foreign Aid Transparency and Accountability Act of 2016 defines ¶¶evaluation·· as ´the systematic collection and analysis of information about the characteristics and outcomes of the program, including projects conducted under such program, as a basis for: (A) making judgments and 55 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV evaluations regarding the program; (B) improving program effectiveness; and (C) informing decisions about current and future programming.µ USAID·s Evaluation Policy establishes two goals for evaluations: accountability and learning. To support accountability, evaluations should measure project effectiveness, relevance and efficiency; match metrics to meaningful outputs and outcomes; and compare commitments and targets through valid measurement and credible analysis. To support learning, evaluations should test underlying assumptions and strategies to refine design and improve future efforts. The purposes of this evaluation are: (i) to determine the degree to which ACTI and PROGRESS achieved their stated goals, why or why not, and how? and (ii) to capture lessons around the development of key policies that may be applied to current and future programming. USAID is the primary audience for the evaluation. Other audiences may include other U.S. government partners, the ACTI and PROGRESS implementing partners, and ASEAN counterparts. Once the final report has been approved by USAID, it will be uploaded to the USAID Development Experience Clearinghouse. A workshop might be conducted to share the evaluation to inform USAID/ASEAN, its implementing partners and counterparts to improve the ongoing activities and to suggest and provide insights for future programming. IV. Scope and Questions For this evaluation, USAID/ASEAN has selected two key policy areas under each program for targeted examination. ACTI·s focus will be on trade facilitation and SME development, and PROGRESS· focus on trafficking in persons and youth engagement. The evaluation will focus on the following broad evaluation questions: 1) What was the development path of the four targeted policy areas, from the inception of the policy area at the ASEAN level through the possibilities for its implementation at the Member State level? 2) How did USAID assistance contribute to the development of these policy areas? 3) What forms of assistance most contributed to the development of these policy areas? Which contributed the least? a. The focus of the question is on USAID assistance, but the contribution of other development partners, and of ASEAN itself, should also be examined in terms most significant contribution. Insights from this analysis should be documented to inform current and future programming. 4) What role has gender played in the design and implementation of activities? What impact has this role had? 5) What lessons learned in strategy and approach can and should be applied to current and future programming? A list of illustrative sub-questions is available in Appendix 1. V. Methodology USIAD/ASEAN estimates that the evaluation will be conducted over a five-month period (September 2019 to January 2020). The evaluation team will propose the evaluation methodology·s, which will include a document review, key informant interviews, and limited site visits. A detailed evaluation methodology will be provided in an evaluation design and work plan (see Annex 1). The evaluation·s methodology will examine the sequence of support steps used to support the policy development process and associate these with policy accomplishments. In part, the methodology will USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 56 use a most significant change approach to determine which forms of support were most critical to the success of the policy development and implementation effort. VI. Deliverables and Schedule The evaluation will include the following minimum deliverables: a. Evaluation Work Plan The evaluation team will develop a work plan that includes the methodology (including a list of sub-evaluation questions and a discussion of methodological limitations), activities, milestones, associated deliverables, and timeline. The work plan will be submitted to USAID for review and approval 10 working days prior to the evaluation team convening to begin its work in Jakarta. b. Evaluation In-briefing Concurrent with approval of the work plan, the evaluation team will conduct an in-briefing with USAID/ASEAN to arrive at common understandings of the evaluation·s purpose and methodology prior to initiating stakeholder interviews. c. Evaluation Debriefings After the field work is completed, the evaluation team will conduct a debriefing with USAID to present a summary of findings, conclusions, and recommendations. Additional debriefings may be held with implementing partners, other U.S. government stakeholders, and ASEAN counterparts. The evaluation team will share a slide presentation of draft findings, conclusions, and recommendations at least two days before the debriefing. d. Report Submission The evaluation team will submit the draft report to USAID within four weeks after delivering its debriefing presentation. The report will reflect verbal feedback from USAID and other stakeholders received during the debriefings. Within 10 working days from submission, USAID will provide written input on the draft report. The evaluation team should resubmit a draft report within three weeks from receipt of USAID·s written comments. The revision cycle may occur more than once, with a targeted maximum of three revisions. USAID will provide final report approval once its comments have been sufficiently addressed. The evaluation report should comply with USAID Evaluation Policy guidance and ADS 203 requirements. A recommended report outline is provided in Annex 3. Report formatting should follow USAID·s branding guidelines. http://pdf.usaid.gov/pdf_docs/PNADB334.pdf. Deliverables Estimated Due Date 1. Draft of evaluation design and work plan September 2019 2. Approved evaluation design and work plan Mid to late September 3. Presentation and discussion of initial key findings, conclusion and recommendation November 2019 4. Draft evaluation report December 2019 57 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV VII. Team Composition The evaluation will be conducted by an independent evaluation team overseen by USAID. At a minimum, the evaluation team will consist of a team leader and two (2) technical experts. The team should include at least one expert with similar programs in the region. The evaluation must be independent from the ACTI and PROGRESS implementing partners, their staff, and subcontractors. All evaluation team members will be required to submit written conflict of interest disclosure forms. The evaluation team should possess the following qualifications: x Expertise and experience in the evaluation of large, complex regional projects, with a preference for familiarity with ASEAN. x Wide experience throughout ASEAN Member States. x Expertise in the technical areas targeted by the evaluation, namely, trade facilitation, SME development, trafficking in persons, and youth engagement. x Skills to successfully plan and conduct key informant interviews across multiple locations. x Regional language skills or the ability to engage interpreters to support interviews of local language documents and records where necessary. VIII. USAID Participation Regular communication between the evaluation team and the designated USAID Activity Manager will be essential to the successful execution of the Scholarship Landscape Assessment. The assessment team will keep USAID apprised of changes and developments that necessitate any significant decision-making or modification of the approved assessment design. MESP encourages participation of USAID staff in the evaluation process. However, USAID participation in stakeholder interviews, if desired, should be structured in consideration of reducing or avoiding interview bias. IX. Level of Effort and Work Plan GANTT Chart The estimated period of performance is September 2019 to January 2020, with an estimated total level of effort (LOE) for the Team Leader of approximately 74 days, as outlined in the table below. No. Activities/Tasks Estimated LOE Deliverables 1. Desk review and development of evaluation work plan 7 days Draft Final Evaluation Work Plan, including sub-questions 2. Work plan revision (after review by USAID) 3 days Final Work Plan (due within five working days after receiving USAID review) 3. Inception meeting, data collection and fieldwork 24 days List of interviews, list of interview questions 4. Analysis and debriefing 10 days Summary presentation of findings, 5. Final report submission (January 2020) Two weeks after receiving comments from USAID/ASEAN. Note: multiple reviews may be required, which could affect the timeline for final completion target of January 2020. 6. Utilization workshop/s (tentative) TBD - depending on USAID preferences and participants availability. Could be following initial draft report submission or following the completion of the report. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 58 conclusions, and recommendations 5. First draft of the evaluation report to be submitted in December 2019 16 days First draft of the report (due within four weeks of the team·s exit debriefing) 6. Compile and incorporate comments of first draft, submit second draft 6 days Second draft of report 7. Compile and incorporate comments of second draft, submit final report 4 days Final Report, no later than January 2020 The chart below presents an estimated timetable for this evaluation. Proposed Work Plan Schedule: Activities Jul Aug Sept Oct Nov Dec Jan 2020 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 SoW and Budget Approval x x x Recruitment x x x x x x Preliminary desk review (MESP and Evaluation team) x x x x x x x x Assessment design and Work Plan x x x Data Collection x x x Data Analysis x x Presentation of key findings x Report writing x x 1st draft submission x Revisions x x x x Finalizing the report x X. Budget MESP will submit to USAID an estimated budget to PRO once the SOW is agreed upon 59 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV Appendix 1. Illustrative Sub-evaluation Questions Evaluation Question Illustrative Sub-evaluation Question What was the development path of the four targeted policy areas, from the inception of the policy at the ASEAN level through its implementation at the Member State level? When did ASEAN begin working on the targeted policy area? What steps has it taken over this time? How did the policy area progress over the period of performance? What differences are seen across Member States? What accounts for these differences? How did USAID assistance contribute to the development of these policies? At what stage did USAID become involved in supporting the targeted policy areas? (In what stages has USAID been involved?) What was the rationale for this involvement? How do stakeholders perceive its impact? What forms of assistance most contributed to the development of these policies? Which contributed the least? In what ways has USAID been involved? What results arose from support to meetings and other events? What was the value of U.S. diplomatic engagement? What other forms of technical assistance (approach and methodology) might have had greater benefit? What role has gender played in the design and implementation of activities? What impact has this role had? To what degree did implementing partners consider gender in the design and implementation of activities? What was the rationale for these decisions? How were these efforts received by ASEAN? What lessons learned in strategy and approach can and should be applied to future programming? Did the assumptions underlying project design and implementation hold true? Which assumptions should be adjusted and why? Was the mix of activities the most effective ones? How can effectiveness be improved? What dynamics will be key moving forward (importance of ASEAN in the region, role of the ASEAN chair, relationships between Member States, evolving trade policies, etc.)? USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 60 Appendix II. Suggested Desk Review Documents Project Documents Annual work plans 1. Quarterly reports 2. Annual reports 3. Monitoring, evaluation and learning plans 4. Gender Analysis for PAD 5. Other project-produced products (list to be provided) 6. ASEAN Documents ASEAN Convention Against Trafficking in Persons, Especially Women and Children 7. ASEAN Plan of Action Against Trafficking in Persons, Especially Women and Children 8. ASEAN Youth Development Index 9. Agreement to Establish and Implement the ASEAN Single Window 10. Protocol on the Legal Framework to Implement the ASEAN Single Window 11. ASEAN Strategic Action Plan for SME Development 2010-2015 12. ASEAN Strategic Action Plan for SME Development 2016-2025 13. ASEAN 2025: Forging Ahead Together 14. Research Design Outline Research design should include, at least, the following components: 1. Background 2. Purposes/Objectives/Questions 3. Research methodology, including analytical framework 4. Information of request list 5. List of key informants 6. List of desk review documents 7. Reference Recommended Report Section and Content At a minimum, all reports should include the following sections: x Executive Summary (2 ² 5 pages) x Evaluation Purpose and Questions (1 page) x Project Background (1 page) x Methodology, including limitations (1-3 pages) x Findings, Conclusions, and Recommendations (15-25 pages) x Annexes 61 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV ANNEX 2. U.S.-ACTI FRAMEWORK USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 62 ANNEX 3. U.S.-PROGRESS FRAMEWORK 63 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV ANNEX 4. EVALUATION SCHEDULE Date Task 1-7 Oct 2019 Contract and mobilize evaluation team 09 Oct 2019 Conduct kick-off meeting 10 ² 22 Oct 2019 Review of documents and draft evaluation design and work plan 22-28 Oct 2019 Review and QA of draft evaluation design by MESP 29 Oct ² 4 Nov 2019 Review of draft evaluation design by USAID 23Oct ² 06 Nov 2019 Plan field work including developing Jakarta meeting schedule (IPs, USAID CORs, ASEC) 31 Oct ² 01 Nov 2019 Prepare in-brief presentation 4 Nov 2019 Conduct in-brief and planning meeting with USAID in Jakarta 5 ² 8 Nov 2019 Review, revise evaluation design and submit to USAID 5 ² 15 Nov 2019 Conduct field work ² Indonesia (ASEAN Secretariat, IPs, Sectoral Bodies) 11² 22 Nov 2019 Plan field work including set-up of meetings in ASEAN Member States (AMS) 19 Nov ² 13 Dec 2019 Conduct field work - ASEAN Member States (Philippines, Singapore, Thailand, Malaysia, Vietnam, Lao PDR, Cambodia, Indonesia) and conference calls (Myanmar, Brunei) 16 ² 18 Dec 2019 Prepare and present preliminary findings to USAID 19 Dec 2019 ² 16 Jan 2020 Conduct field work (continued) ² ASEAN Member States (Malaysia, Indonesia and conference calls incl. USG ² DC) 17 ² 21 Jan 2020 Formatting of primary data (KII and FGD scripts) 22 Jan ² 11 Feb 2020 Data analysis (of primary data and desk study) 05 ² 21 Feb 2020 Write 1st draft Evaluation Report (submit to MESP) 24 ² 28 Feb 2020 Review, edit and design report (by MESP) 28 Feb 2020 (COB) 1st draft Evaluation Report submission to USAID 02 ² 10 Mar 2020 Review of 1st draft Evaluation Report by USAID 11 ² 19 Mar 2020 (COB) Revise report (submit to MESP) 20 ² 24 Mar 2020 Review and edit report (by MESP) 24 Mar 2020 Submit 2nd draft Evaluation Report to USAID 24-25 Mar Prepare presentation for USAID 26/27 Mar 2020 Present draft final findings to USAID ² in Jakarta 30 Mar ² 06 April 2020 Review of 2nd draft Evaluation Report by USAID 07 ² 13 April Revise report based on feedback from USAID (submit to MESP) 14 April ² 20 April 2020 Format and QA of Final Evaluation Report (by MESP) 21 April 2020 Submit Final Evaluation Report to USAID Utilization Phase, if desired by USAID TBC May Prepare Presentation and Executive Brief for ASEC TBC May Review, edit Presentation and Executive Brief for ASEC (MESP) TBC May Review Presentation and Executive Brief for ASEC (USAID) TBC May Presentation /workshops to ASEC / Sectoral bodies USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 64 ANNEX 5. KEY INFORMANT LIST 65 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 66 67 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV ANNEX 6. KEY INFORMANT INTERVIEW QUESTIONS Meta Data Reference : Support Area : Organization : Level : Position : Name : Gender : Date of Interview : Q1. Development Path of Support Area 1.1. When did ASEAN begin working on the support area? 1.2. What steps (policy development path) have taken place at the ASEAN level from inception until ACTI/PROGRESS commenced in 2013? 1.3. What steps (policy development path) have taken place in the ASEAN Member State (AMS)/your country over this time? Q2. USAID contribution to policy development 2.1. At what step did PROGRESS/ACTI become involved in supporting the targeted policy areas? 2.2. What was the rationale for this involvement? 2.3. What specific assistance was provided by PROGRESS/ACTI? 2.4. What were the main outcomes or impact of PROGRESS/ACTI assistance? 2.5. Did the assistance of USAID contribute to certain AMS more than the others? Q3. Most and least contribution 3.1. What forms of assistance most contributed to the development of these policies AT ASEAN level? 3.2. What forms of assistance most contributed to the successful policy implementation by AMS? 3.3. Which forms of assistance contributed the least? 3.4. What have been the most significant achievements in the area overall? 3.5. What forms of assistance most contributed to these achievements? Q4. Role & Impact of Gender 4.1. How did PROGRESS/ACTI integrate gender in the design and implementation of activities? 4.2. What was the rationale for these decisions? 4.3. How did these gender efforts contribute to the gender equality agenda in ASEAN? 4.4. How can gender efforts be improved in design and implementation? USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 68 Q5. Lessons learned in strategy and approach 5.1.1.Have U.S.-ASEAN engagements built capacity for development of the ASEAN Community? 5.1.2.Do the AMS have the commitment and capacity to implement the policies in the selected areas? 5.2. How can effectiveness of future USAID-ASEAN assistance be improved (design / methodology /activities)? 5.3. What aspects (e.g. institutional dynamics, role of ASEAN Chairs, support to AMS etc.) will be key moving forward? 5.4. Is regional level assistance (ASEAN) more effective than national level assistance? Please explain 69 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV ANNEX 7. FGD TOOL Meta Data Reference : Support Area : Organization : Level : Position : Name : Gender : Date of Interview : Q2. USAID contribution to policy development 1.1. What specific assistance was provided by PROGRESS/ACTI? 1.2. What were the main outcomes or impact of PROGRESS/ACTI assistance? 1.3. Did the assistance of USAID contribute to certain AMS more than the others? Q3. Most and least contribution 1.1. What have been the most significant achievements in the area overall? Q4. Role & Impact of Gender 1.1. How can gender efforts be improved in design and implementation? Q5. Lessons learned in strategy and approach 1.1. How can effectiveness of future USAID-ASEAN assistance be improved (design / methodology /activities)? USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 70 ANNEX 8. ENHANCED POLICY CONTINUUM This annex provides further details to enhance the policy continuum for the ASEAN policy development path based on the findings of the evaluation team. Each sectoral body develops and implements policy according to guidelines in its organizational terms of reference (ToR), mandates from senior officials or ministerial bodies, and broad guidance from The ASEAN Charter. Phase I: Policy Commitment and Institutional Arrangements The three stages in Phase I are summarized below. x Stage 1: Inception. This is catalyzed by political commitment made by ASEAN Leaders. These are reflected in instruments endorsed by the ASEAN leaders, such as the Declaration of ASEAN Concord II, the ASEAN Community Blueprints, or ASEAN Leaders Joint Statements. These instruments provide the framework or starting principles for cooperation by the AMS in a given area led by the respective Sectoral Ministerial Body who issue a Ministerial Joint Statement. This mandates the respective Senior Officials Meeting to pursue the development of a given policy area. x Stage 2: Institutional arrangements. Following the inception stage, institutional roles are defined by the Senior Officials Meeting. This often includes the establishment of a high￾level task force, a product working group (PWG), a working group (WG), or in some cases a committee made up of representatives from the relevant sectoral entities in each AMS. This is followed by the formulation of organizational terms of reference for these entities, which are approved by the AMS at a Senior Officials· Meeting or at the Ministerial level. x Stage 3: Definition. Further clarification and policy definition takes place by prioritizing the target areas and task distribution, which includes the setting-up of task forces for specific sub-areas, including lead countries or champions for specific areas. Phase II: Policy Development The main stages in Phase II are as follows: x Stage 4: Drafting of policy instruments. This stage undertakes the drafting of text for the policy instrument. Examples of policy instruments include strategic action plans, work plans, guidelines, directives, conventions, and agreements with legal protocols. For policy instruments in the ASEAN Economic Community (AEC) pillar, it was observed that drafting was a technical process driven by an appreciation of best practices, and a common understanding of issues and opportunities among the sectoral body members. In sensitive areas such as Trafficking in Persons (TIP), the drafting process was driven by political commitments. Some agreements, such as the ASEAN Framework Agreement on Services (AFAS), require additional instruments, such as protocols to operationalize subsequent commitments by the AMS for a given sub-sector. These protocols become technical annexes to the original agreement. For policy instruments that do not have legal provisions, such as those in youth engagement, SME development and WEE, Stage 4 focuses on the drafting of work plans or strategic action plans. x Stage 5: Review of draft policy instruments. Assessment and review by the AMS of the draft policy text, which includes discussion, negotiation and revision. This review stage can often take several rounds of sectoral body meetings at the task force (TF), product working group (PWG) or working group (WG) level, until a common interpretation of the policy instruments is achieved among the AMS members of the sectoral body. 71 | RETROSPECTIVE EVALUATION REPORT OF U.S.-ASEAN ACTI & PROGRESS PROJECT USAID.GOV x Stage 6: Policy instrument finalization and approval. Finalization and agreement in principle by all AMS at the respective TF, PWG or WG levels. Once the policy instrument has been agreed in principle it is passed on to the senior bodies, such as committees or Senior Officials· Meetings for approval. Phase III: Member State approval The main stages in Phase III are as follows: x Stage 7: Endorsement by ASEAN sectoral bodies. This stage culminates in endorsement by senior bodies and the signing of the policy instruments by the respective Ministers (for instruments without legal provisions) or ASEAN Leaders (for instruments with legal provisions). x Stage 8: Ratification by ASEAN Member States. This stage is relevant for policy instruments with legal provisions, such as the ASEAN Medical Device Directive (AMDD) and the ASEAN Convention Against Trafficking in Persons (ACTIP) Especially for Women and Children, and ensures that the legal framework is in place in each AMS to implement the ASEAN policy instrument. This requires a process to transpose the policy into the legal and institutional context of each AMS. There is usually a finite period within which each AMS should ratify these protocols. Policies that do not require the development of legal instruments, such as SME development, youth engagement and women·s empowerment, do not require ratification by the AMS. Phase IV: Implementation The main stages in Phase IV are as follows: x Stage 9: Implementation at the regional level. Initiatives at the regional level include additional initiatives, data collection and sometimes the development of additional supporting instruments, such as platforms, partnerships, indexes or the development of legal protocols and technical annexes. The development of additional policy instruments follows the activities described from Stages 4 to 8. x Stage 10: Implementation in the AMS. The implementation of initiatives by individual AMS includes activities from action plans, the provision of training for government and businesses, and constructing outreach. For policy instruments with legal provisions, implementation is done based on an internal schedule developed by each AMS. Additional capacity building is often required to implement the agreement at the AMS level. x Stage 11: Monitoring and Reporting. This final stage includes compliance monitoring, outcome monitoring and an impact assessment, as guided by the Monitoring and Evaluation (M&E) Framework of a given ASEAN Community Blueprint. For example, sectors and areas under the AEC are guided by the AEC 2025 Monitoring and Evaluation Framework. The AEC 2025 M&E Framework stipulates monitoring and reporting at three levels: (i) Compliance Monitoring of activities and outputs according to annual priorities of the sectoral work plans (on a quarterly basis); (ii) Outcome Monitoring against key performance indicators (KPIs) set by the sectoral work plan (on an annual basis); and (iii) an Impact Assessment, which is usually undertaken through a combination of a macroeconomic assessment, and the use of ASEAN Statistics and third-party databases. The impact assessment is usually undertaken at the mid-term and the endpoint of a blueprint (after five years). The responsibility for compliance and outcome monitoring of a given sub-area or area resides with the specific PWGs, WGs, and TFs that are leading the specific activities. USAID.GOV RETROSPECTIVE PERFORMANCE EVALUATION OF U.S.-ASEAN ACTI & PROGRESS PROJECT | 72 They report progress at the committee level. The committee, together with ASEC, aggregates progress and reports at the Senior Officials· Meeting. According to informants from ASEC, the development of a binding agreement, is the ultimate end￾product expected from the ASEAN policy development path. A major factor that drives the development of a policy from the inception framework stage to a binding agreement is the establishment of capacity-building instruments such as regular training workshops, training modules or facilitation to ensure that all the AMS are comfortable with, and understand the requirements of the binding agreement. SUMMARY OF DEVELOPMENT PATH IN PRIORITY POLICY INITIATIVES A summary of the development path for each of the priority policy areas supported by USAID is provided in the Table below, together with the stages at which USAID provided support, which are highlighted in grey. Table 10. Summary of the Policy Development Path AHTN AMDD TMHS Services TIP Youth SME WEE Stages: Stage 1: Inception x x x x x x x x Stage 2: Institutional arrangements x x x x x x x x Stage 3: Tasks definition x x x x x x x x Stage 4: Drafting policy instruments x x x x x x x x Stage 5: Review x x x x x x x x Stage 6: Finalization x x x x x x x x Stage 7: Endorsement x x x x x x x x Stage 8: Ratification x x x x x Stage 9: Implementation at ASEAN level x x x x x x x Stage 10: Implementation at AMS x x x x x x x Stage 11: Monitoring and reporting x x x x x x