June 2016 This publication was produced at the request of the United States Agency for International Development (USAID). It was prepared independently by Nhat Nguyen Duc (Team Leader), Rajan Kapoor, Dung Trinh Tien, Hoang Tran Vu, Hong Hanh Dang Thi and Management Systems International. Final Evaluation of the Provincial Competitiveness Index Final Evaluation of the Provincial Competitiveness Index Performance Contracted under AID-440-C-14-00003 USAID/Vietnam Evaluation, Monitoring and Survey Services (VEMSS) Cover photo: The Dragon Bridge in Da Nang spans the River Hàn. (Photo by Nhat Nguyen Duc) DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation iii TABLE OF CONTENTS ACRONYMS AND ABBREVIATIONS .................................................................................................... VI ACKNOWLEDGMENTS .........................................................................................................................VII EXECUTIVE SUMMARY ............................................................................................................................ 1 EVALUATION PURPOSE ........................................................................................................................................................................ 1 CORE PCI OBJECTIVES AND ACHIEVEMENTS/FINDINGS ................................................................................................................. 1 THE PCI PROGRAM’S EFFORTS TO IMPROVE THE BUSINESS ENVIRONMENT ................................................................................ 4 OVERALL CONCLUSIONS .................................................................................................................................................................... 4 RECOMMENDATIONS ........................................................................................................................................................................... 6 LESSONS LEARNED................................................................................................................................................................................ 7 I. INTRODUCTION ............................................................................................................................ 9 PROVINCIAL COMPETITIVENESS INDEX.............................................................................................................................................. 9 II. EVALUATION PURPOSE AND METHODOLOGY .................................................................. 10 EVALUATION PURPOSE ......................................................................................................................................................................10 EVALUATION QUESTIONS ................................................................................................................................................................. 10 METHODOLOGY.................................................................................................................................................................................10 III. COUNTRY CONTEXT ................................................................................................................. 12 COUNTRY PERFORMANCE ................................................................................................................................................................12 GOVERNMENT AND ADMINISTRATION REFORM ...........................................................................................................................12 ECONOMIC CONTEXT OF PROVINCES VISITED..............................................................................................................................13 IV. CORE PCI OBJECTIVES AND ACHIEVEMENTS ...................................................................... 16 1. SURVEY IMPLEMENTATION ............................................................................................................................................................16 2. PROMOTION OF PUBLIC-PRIVATE DIALOGUE............................................................................................................................22 3. CAPACITY BUILDING .....................................................................................................................................................................26 4. SUSTAINABILITY..............................................................................................................................................................................31 5. CROSSCUTTING ISSUES..................................................................................................................................................................34 6. MONITORING EVALUATION AND LEARNING.............................................................................................................................36 IV. PCI PROGRAM CONTRIBUTIONS TO THE BUSINESS ENVIRONMENT ................................ 37 ADMINISTRATION AND REGULATORY REFORM.............................................................................................................................37 RELEVANCE OF SUB-INDEXES TO FIRM OPERATION......................................................................................................................40 IDENTIFICATION OF KEY DRIVERS OF REFORM ..............................................................................................................................40 V. OVERALL CONCLUSIONS ......................................................................................................... 43 SURVEY IMPLEMENTATION................................................................................................................................................................. 43 PROMOTING PUBLIC-PRIVATE DIALOGUE ......................................................................................................................................44 IMPROVING ECONOMIC GOVERNANCE CAPACITY.......................................................................................................................44 SUSTAINABILITY ..................................................................................................................................................................................44 CONTRIBUTING TO IMPROVING THE BUSINESS ENVIRONMENT ..................................................................................................44 CROSSCUTTING ACTIVITIES ..............................................................................................................................................................45 VI. RECOMMENDATIONS ................................................................................................................. 45 VII. LESSONS LEARNED ..................................................................................................................... 46 REFERENCES ............................................................................................................................................ 48 ANNEX 1: LIST OF STAKEHOLDERS ................................................................................................... 49 ANNEX 2: INTERVIEW GUIDE: PROVINCIAL STAKEHOLDERS .................................................... 58 Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation iv ANNEX 3: INTERVIEW GUIDE: PRIVATE ENTERPRISES ................................................................. 63 ANNEX 4: QUESTIONNAIRE: GOVERNMENT STAKEHOLDERS ................................................... 65 ANNEX 5: QUESTIONNAIRE: PRIVATE ENTERPRISES .................................................................... 67 Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation v LISTS OF FIGURES AND TABLES Figure 1: PCI Composite Scores by Province (2010–2014).................................................................... 15 Figure 2: The Median PCI Value............................................................................................................... 15 Figure 3: PCI Data Use by Government Stakeholders (Percentage).................................................... 19 Figure 4: PCI Graphic of Thai Nguyen, 2007–2014................................................................................ 21 Figure 5: Percentage of Private Firms Agreeing That Local Governments Try to Maintain PPD .... 24 Figure 6: Responses to Whether the PCI Program Helps Increase Local Capacity on Economic Governance ........................................................................................................................................ 29 Figure 7: PCI Economic Governance Improvement Indicators ............................................................ 36 Figure 8: Key Concerns of Business ......................................................................................................... 40 Figure 9: Administrative Reform by provinces ....................................................................................... 41 Figure 10: MOA Analysis Framework I.................................................................................................... 43 Table 1: Changes in Vietnam’s Economy from 2013 to 2015 ................................................................ 12 Table 2: Major GVN Legislation on Administrative Reform Since 1995 .............................................. 13 Table 3: Basic Macro-Economic Data by Province ................................................................................. 14 Table 4: PCI Launch Targets and Achievements, 2013–2015................................................................ 18 Table 5: Diagnostic Workshop Targets and Achievements................................................................... 24 Table 6: Number of Training Courses Conducted in Years 1 and 2 ..................................................... 27 Table 7: Action Plans Developed or Adopted with Influence by PCI .................................................... 27 Table 8: Evaluation of Provinces’ Action Plans ....................................................................................... 30 Table 9: Targets and Achievements Regarding Financial Sustainability Plans ..................................... 31 Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation vi ACRONYMS AND ABBREVIATIONS AOR Agreement Officer’s Representative ASEAN Association of Southeast Asian Nations BSM Benefit-Sharing Mechanism CAP Commune Action Plans COP Chief of Party CPC Commune People’s Committee CPMU Central Project Management Unit CSA Climate-Smart Agriculture CSO Civil Society Organization CSR Corporate Social Responsibility DAI Development Alternatives International DARD Department of Agriculture and Rural Development DCOP Deputy Chief of Party DDI Domestic Direct-Invested DoNRE Department of Natural Resources and Environment DoTI Department of Trade and Industry DPC District People’s Committee DPI Department of Planning and Investment DoT Department of Taxation FIE Foreign-Invested Enterprises FY Fiscal Year GDP Gross Domestic Product GTD General Tax Department GVN Government of Vietnam IFC International Finance Corporation ISED Institute for Socio-Economic Development (in Da Nang) M&E Monitoring and Evaluation MARD Ministry of Agriculture and Rural Development MCD Center for Marine Conservation and Development MOA Motivations, Opportunities and Analysis MOIC Ministry of Information and Communication MPI Ministry of Planning and Investment MSI Management Systems International NGO Non-Governmental Organization ODA Official Development Assistance PAR Public Administration Reform PAR-MP Public Administration Reform – Master Program PBA Provincial Business Association PC People’s Committee PCC People’s Committee Cabinet PCI Provincial Competitiveness Index PMEP Performance Monitoring and Evaluation Plan PPC People’s Provincial Committee PPD Public-Private Dialogue SEDP Socio-Economic Development Plans USAID United States Agency for International Development USG United States Government USD U.S. Dollars VAT Value-Added Tax VCCI Vietnam Chamber of Commerce and Industry VNCI Vietnam Competitiveness Initiative Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation vii ACKNOWLEDGMENTS This Provincial Competitiveness Index (PCI) performance evaluation is the consolidated work of the evaluation team commissioned by USAID/Vietnam. However, it would not have been possible without the kind support and help of many individuals and organizations. We would like to thank the members of the Vietnam Chamber of Commerce and Industry (VCCI) team, in particular Chairman Vu Tien Loc and Director Dau Anh Tuan from the Legal Department, for their time and insights on PCI’s development and impact on economic governance reform in Vietnam. We would also like to express our gratitude to Eddy Malesky, Pham Ngoc Thach and Nguyen Ngoc Lan and VCCI regional offices in Can Tho and Da Nang, who took the time to meet and share information with the evaluation team. We are thankful to the local partners who took time from their schedules to meet with the evaluation team. In particular, we would like to mention the chairman of the Provincial People Committee, leaders and officials from the People’s Committee Cabinet, staff from the departments of Planning and Investment; Taxation; Natural Resources and Environment; and Industries and Trade. The input provided by these stakeholders was critical to the evaluation. We are grateful for the constructive and critical discussions with representatives of all private enterprises who took part in interviews, as well as representatives from the various business associations. We would also like to thank them for completing the mini-poll on such short notice. The evaluation team would like to express our sincere appreciation to the many Government of Vietnam officials and academic experts, namely Mdm. Pham Chi Lan, Prof. Nguyen Van Thang from the National Economics University and faculty from the Institute of Socio-Economic Development and Provincial University of Da Nang. Importantly, our appreciation goes to the USAID Program Development Office and Economic Growth and Governance Office for their support in meeting with and assembling information for the evaluation team. In particular, we want to acknowledge Randolph Flay, Binh Thi Thanh Le and Michael Trueblood, as well as Ha Thi Nguyen and Emily Rupp, who provided input to the methodology development, joined the evaluation team during field visits and reviewed reports. Prompt comments and field notes provided by the USAID team have been informative and helpful to deepen data collection and analysis. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 1 EXECUTIVE SUMMARY USAID launched the Provincial Competitiveness Index (PCI) in 2005 under the Agency’s Vietnam Competitiveness Initiative (VNCI) program. From 2005 until 2013, prime contractor Development Alternatives Inc. (DAI), principal subcontractor The Asia Foundation and local partner the Vietnam Chamber of Commerce and Industry (VCCI) implemented the PCI program. Starting in 2013, USAID awarded a direct grant to VCCI to carry out the PCI program until 2016. PCI aims to promote private sector growth in Vietnam by supporting provincial governments to improve business-enabling environments for sustained growth through improved economic governance. To realize this goal, PCI has organized activities around three main objectives: 1) Promote public-private policy dialogue on economic governance through the PCI survey; (2) Increase understanding of economic governance based on the results of the PCI survey; and (3) Improve economic governance capacity in targeted provinces. PCI’s set of core activities includes implementation of the surveys of foreign direct-invested and domestic enterprises; launch events to disseminate survey results; and diagnostic workshops that the PCI team conducts to assist provincial officials in understanding the factors behind provincial survey scores, with the ultimate goal of improving their performance in local economic governance to promote the development of the private sector. In addition to the core PCI activities, the PCI team also conducts complementary activities that include maintenance of the PCI website, PCI training courses for business, PCI training courses for National Assembly members, development of a handbook on public-private dialogue and capacity building for VCCI regional staff to ensure sustainability of the program. Evaluation Purpose USAID/Vietnam requested that the Vietnam Evaluation, Monitoring and Survey Services (VEMSS) project assess key PCI program results and contributions to reform and improvement of local economic governance in selected provinces. The evaluation will serve as a base of understanding for ongoing and future technical assistance to the Government of Vietnam (GVN) to improve the country’s business-enabling environment. Key evaluation questions include the following:  What progress has been made in achieving the PCI program’s objectives?  How has the PCI program contributed to business environments, particularly in relation to administration reform? To answer the evaluation questions, the evaluation team collaborated with USAID to select seven provinces where the team would perform data collection. The selected provinces represented both high and low performers in terms of provincial economic governance (PCI composite score). Within each province, the evaluation team spoke with multiple stakeholders to triangulate data regarding PCI implementation and the program’s contribution to local economic governance improvements. Core PCI Objectives and Achievements/Findings Survey Implementation In general, the PCI program has met all targets, including high-quality surveys conducted; successful engagement of provincial leaders and business associations in the dissemination of the survey results; and improving the acceptance of PCI and the program’s results by provinces. Additionally, mass media attention has been achieved not only during the PCI launch event, but throughout the implementation of reform￾related activities by provinces. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 2 As a result of the program’s relevance, PCI data has been widely and frequently used by government stakeholders. A mini-poll conducted during the evaluation showed that 73 percent of government stakeholders frequently use PCI data in their daily work. The evaluation team found evidence suggesting that PCI helps improve the local understanding of economic governance issues, which has helped the program successfully mobilize commitment from local governments in the implementation of economic governance reform. The drivers of success of PCI program implementation include: 1) commitment and support from both USAID/Vietnam and VCCI; 2) the robust design of the program; 3) growing capacity of the VCCI/PCI team; and 4) effectiveness of international technical assistance. The Government of Vietnam and other international donors have highly valued the PCI program’s achievements. The current approach of implementing the survey via mail may be susceptible to nonresponse bias. Reported cheating and coercion are challenges that might negatively influence the quality of data. Also, the length and complexity of the questionnaire may affect a firm’s ability to complete the survey. Promotion of Public-Private Dialogue (PPD) PCI survey data is an efficient channel for the private sector to voice its opinions and concerns regarding economic governance. Diagnostic workshops were also effective venues for promoting PPD; they were well attended by provincial leaders, line departments and private sector representatives. However, the evaluation team notes that only about 50 percent of low-performing provinces diagnostic workshops with the PCI team. This largely results from the fact that the PCI team resources were limited. Additionally, the VCCI team conducted diagnostic workshops only in provinces and regions that invited the VCCI team. Some of the contributions the PCI program has made include the following:  Clear evidence shows that PCI contributes to the awareness of the value of PPD in economic governance reform. Provincial stakeholders have considered PPD to be a critical tool in guiding the reform process.  In addition to the diagnostic workshops, provinces directly sponsored many PPD events, which included business cafés, entrepreneur rendezvous, sectoral meetings hosted by provincial line departments and site visits to firms by provincial leaders.  The evaluation team has collected clear evidence that PCI has contributed to the establishment and development of provincial business associations (PBAs). Challenges identified include difficulties for small and medium enterprises (SME). In most cases, provincial business association members were larger business that had resources to actively participate in a PBA. In addition, provinces displayed significant variation in the quantity and quality of PPD events they implemented. The evaluation team also found evidence that the quality of province-sponsored PPD events may greatly affect the business community’s confidence in the local reform process. Capacity Building The capacity of VCCI/PCI team has improved significantly over time. Currently, VCCI is capable of conducting the PCI surveys and dissemination events independently. Government officials from the central and provincial levels recognize VCCI as a credible organization and a center of knowledge regarding the private sector in Vietnam. The Central Party even recognized the PCI program as “one of the 62 projects that have made an enormous contribution to the socio-economic development of the Country between 2011 and 2015.” Stakeholders’ understanding of PCI results and economic governance has improved to varying degrees. Some high-performing provinces are highly capable of proactively performing in-depth analysis to examine sub- Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 3 index scores. The analysis process has enabled these provinces to translate sub-index scores into a concrete understanding of local economic governance issues and begin working toward developing solutions. However, other provinces did not demonstrate this ability to conduct analysis. Findings from the evaluation team’s fieldwork indicate that PCI contributes to the overall economic governance capacity of provincial officials. All provincial stakeholders reported that the PCI survey is a useful tool in helping provincial authorities better understand specific issues of economic governance in their provinces. This finding is bolstered by the results of the evaluation’s mini-poll, with 99 percent of provincial stakeholders indicating that the “PCI program helps to increase local capacity on economic governance.” This is supported by the trend in the median PCI composite score, which has increased over the last several years, demonstrating the improvement in economic governance capacity throughout the country. All provinces in which the evaluation team conducted fieldwork have developed a strategic/action plan or issued legal documentation aimed at improving provincial competitiveness. However, the quality of the action plan (reflected by its comprehensiveness and practicality) varied across provinces. In addition, the ability of provinces to implement the action plans varied across the seven provinces that the evaluation team visited. Sustainability The evaluation team collected clear evidence that VCCI will sustain the technical capacity to implement the PCI program. However, the majority of knowledge regarding data collection and analysis resides with only a few individuals. Going forward, staff turnover may affect program implementation. Regarding financial sustainability, the PCI team has considered six possible funding models. These potential models include the following:  Licensing of diagnostic and consulting workshops to select firms;  Providing PCI data to users for a fee;  Selling space on the PCI survey;  Selling data to investment firms;  International donor funding; and  Provincial funding. However, the team has not finalized any formal documentation regarding cost-benefit analysis or pilot testing plans regarding each of the possible options. Crosscutting Issues Government, private sector and third-party stakeholders have all indicated that the PCI program has been effective in establishing a brand that is recognized and respected. PCI has adopted a simple management structure that has been complemented by targeted support from USAID and VCCI. Also, the partnership between VCCI and USAID has benefitted from leaders who are committed to the program. There is room for further enhancing the monitoring and evaluation activities of the PCI program. In the current PCI M&E plan, the indicators used to track progress toward the program’s objectives can track only program outputs. A review of the quarterly reporting documents and interviews with the PCI team reveal that, as presently designed, no indicators can track the quality of capacity building and PPD activities sponsored by local officials. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 4 The PCI Program’s Efforts to Improve the Business Environment In all provinces that the evaluation team visited, PCI has supported implementation of the GVN’s agenda on administration reform, as required by Resolution 30c by the GVN on the National Program for Administrative Reform from 2011 to 2020, which was enacted in 2011 at the provincial level. PCI has provided provincial leadership with a performance evaluation tool. Provincial officials, domestic researchers and international experts all report that PCI has provided a tremendous boost to the economic governance reform process at the provincial level by providing the PPC with a tool for monitoring progress. PCI provides a platform for stakeholder discussion around administration reform issues. In all visited provinces, the PPC and line department used the PCI annual score and ranking for internal review purposes. PCI sub-indexes highlight some common issues affecting private enterprises. Increased public awareness from mass media coverage has positively affected reform. The mass media often reports on annual PCI scores and rankings at both the national and provincial levels, which has created legitimate pressure on provincial governments to reform. PCI supports coordination between provincial departments for sectoral reform. In all visited provinces, line department representatives that the team interviewed confirmed that sectoral reform was possible only with the collective action from related agencies. In each visited province, the evaluation team found evidence of implementation of economic governance reform over the last several years. However, the level of reform and its effectiveness has varied across provinces. During fieldwork, the evaluation team did not encounter any instance where provincial leadership or officials within provincial line departments rejected PCI results or the idea of administrative reform. To better understand the variation in the level of administrative reform implemented between provinces, the evaluation team divided the provinces into three major groupings: 1) those that implemented limited or ineffective reform; 2) those that conducted rapid assessment of PCI annual scores, which resulted in a short￾term response; and 3) those that have taken long-term and strategic action. In high-performing provinces (those in group 3), key drivers of successful reform include the following:  Prioritizing improvement of provincial competitiveness;  Policy/program development that complements central government planning;  Active monitoring by provincial leaders;  Active engagement by the business community; and  Mass media engagement. In low-performing provinces (those in groups 1 and 2), the variation in reform at the provincial level can be attributed to key factors related to motivation and ability (MOA analysis framework), including the following:  Lack of local resources/capacity to reform, such as financial or human resources;  A weak connection with the business community, combined with an inability to effectively understand how local economic governance issues contribute to low performance; and  Provincial officials demonstrated weak motivation and commitment to reform, which resulted in infrequent monitoring of reform implementation and line departments’ lack of accountability to provincial leaders. Overall Conclusions The PCI program has supported provincial governments to understand and engage the business community to improve the business environment over the lifespan of the program. It has supported provincial Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 5 governments to improve local business environments in line with the national/provincial administrative reform agendas, promoted public-private dialogue and increased understanding of economic governance from all stakeholders. Survey Implementation  The PCI team has employed high standards of quality in the design and implementation of the PCI survey. The team has achieved this through the effective engagement with international and domestic experts, national and provincial stakeholders and other donors during the design and implementation phases of the PCI survey, as well as during the subsequent data dissemination and use activities.  Areas of survey implementation still need to be addressed on a continual basis; these include reducing the length and complexity of the questionnaire, developing procedures to prevent cheating and coercion and increasing stakeholders’ knowledge about survey methodology. Promoting Public-Private Dialogue  PCI has achieved its objectives of promoting increased public-private dialogue at the provincial level by bringing provincial officials and businesses together on various aspects of economic governance. In some provinces, PPD has facilitated direct coordination between the private sector and government agencies of action plan development and implementation.  The quality of PPD events varies greatly across provinces due to provincial-level capacity and resources.  SMEs’ ability to participate in PPD was often limited in the provinces the evaluation team visited.  Provincial diagnostic workshops are a useful tool in strengthening PPD and governance capacity; however, not all provinces were able to access technical assistance from the PCI team due to limited budgetary resources. This may result in challenges, especially for low-performing provinces. Improving Economic Governance Capacity  Understanding of economic governance issues and the capacity to implement reforms has improved at the provincial level. However, implementation and effectiveness of reform vary across provinces.  VCCI’s capacity and credibility have grown significantly as a result of PCI’s implementation and achievements. VCCI has proved to be a capable partner in the implementation of both PCI surveys and supporting activities. This has resulted from the long-term implementation of the program (10 years) with collaboration and support from USAID/Vietnam.  Due in large part to limited resources, PCI outreach activities have mainly focused on government officials at the provincial and national levels. Direct training to business associations has been limited in participation and geographical coverage.  Provinces’ capacity to independently perform in-depth analysis of PCI raw data is still low. While the VCCI/PCI team is willing to share, the provision of raw data to provincial stakeholders has been limited because of the PCI team’s data release policy and the limited capacity of local officials to request raw data. Sustainability  VCCI/PCI technical capacity is sustainable, although the number of VCCI (central and regional) staff who have technical capacity is still limited.  Beyond receiving direct funding from USAID/Vietnam, no established plan exists to sustain the PCI program financially. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 6 Contributing to Improving the Business Environment  PCI has contributed to creating momentum for stakeholders to coordinate around reform planning and implementation.  Key factors driving reform at the provincial level include provincial leaders’ prioritization of competitiveness improvement and commitment to actively monitoring the implementation of economic governance reform policies and programs by line departments. Additionally, an active and engaged business community that can help provincial officials identify key policy bottlenecks and provide input regarding a solution can contribute greatly to pushing through economic reforms.  Economic governance is one of many factors that influence the development of the domestic private sector. The PCI program’s overarching objective is to promote private sector growth in Vietnam. However, while PCI has made a contribution to improving economic governance in Vietnam, the total impact of the PCI program on private sector growth is still unclear. Crosscutting Activities  Management of the PCI program has been effective.  While it has successfully complied with the project M&E plan, the PCI program has been unable to systematically assess the quality of outreach activities sponsored by the PCI program (diagnostic workshops or training), which impacts the program’s ability to understand progress toward achieving objectives 2 and 3. Additionally, as the program is currently designed, it does not collect data on PPD events/activities (business cafés, entrepreneur rendezvous, etc.), which provinces initiated and sponsored. Understanding the quantity and effectiveness of such PPD events would be beneficial for provincial stakeholders in their efforts to maintain an effective and continuous PPD. Recommendations Future Implementation of the PCI Survey  USAID should ensure that the PCI program continues to focus significantly on its core activities, which include implementation of the Domestic Private Sector Survey, the Foreign Investment Survey and diagnostic workshops. These aspects remain the most critical to the PCI program and should continue to be implemented with the same technical rigor and efficiency. These core activities are still and will continue to be relevant for provincial stakeholders, especially low-performing provinces, over the next three to five years or until evidence of PCI composite score conversion (among all provinces) shows that the ranking is no longer relevant in relation to absolute score.  Local economic environments can change quickly; therefore, it is important for the PCI team to review the survey questionnaire annually to assess the relevance of the sub-indexes and the specific questions posed. Such a review should include a pilot test of the questionnaire using a diverse sample of the country’s business community. The team can also consider collecting public opinion regarding survey content through its website. In addition, the PCI team may consider providing more in-depth information on the survey methodology to provincial stakeholders, to avoid misconceptions and misunderstandings regarding PCI data. Promoting Public-Private Dialogue  To further strengthen the effectiveness of PPD, the PCI team should consider providing technical guidelines for provinces on developing and implementing effective PPD activities. The guidelines should include ways to involve SMEs in PPD events.  The PCI team should consider developing a criteria for the selection of provinces that will receive technical assistance and support through diagnostic workshops. Low-performing provinces that demonstrate a willingness to reform should be prioritized. Regional PCI workshops should also be Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 7 maintained or expanded, since they provide a greater number of provinces the opportunity to access technical assistance. Improving Economic Governance Capacity  USAID/Vietnam and the PCI team should consider increasing the number of trainings. Trainings should take place in all seven economic zones, with the primary beneficiaries being the local business community and newly appointed provincial leaders. By centering trainings in each of the economic zones, local private sector representatives can more easily access the trainings organized in their regions. Trainings should include a presentation of lessons learned and best practices from the region in regard to business association engagement and its impact on provincial reform. Trainings should also provide beneficiaries with a comprehensive analysis of PCI sub-indexes and its interpretation to the local reform agenda. Finally, trainings should provide guidance on how business associations can best develop and maintain linkages between each other, including often-marginalized SMEs.  The PCI team should review and consolidate common stakeholder questions into a detailed frequently asked questions (FAQ) section on the website. This FAQ section would provide an easily accessible and official source of information to the public. Sustainability of the PCI Program  The PCI team should develop a human resource plan for the PCI program. The plan should include a system for developing and utilizing additional well-trained permanent staff at both VCCI central and regional offices.  The PCI team and USAID should establish clear expectations regarding funding and sustainability of the PCI program. Next, representatives from the PCI team and USAID should work together to develop a clear and feasible financial sustainability plan that considers two or three of the most viable funding options, with forecasting and revenue models based on research and pilot testing. Better Monitoring Quality and Impacts of PCI Program Activities  The PCI team should employ in-depth monitoring of PPD and capacity-building activities. By doing this, the team can more systematically assess the quality and impact of such activities. This can be done by adding a section in the PCI survey regarding the quality of PPD or by the PCI team conducting a separate, smaller survey. To better monitor PPD and capacity-building activities that are directly part of the PCI program, such as diagnostic workshops and trainings, the PCI team can conduct pre- and post-tests during events to gauge knowledge gained by participants and assess the helpfulness of specific topics covered during the events. Lessons Learned  Reform is an evolutionary process that needs consistent support over time without compromising the rigor of related activities. Any program’s design and implementation should include a clearly defined, singular objective. This is especially true for programs supporting governance reform: The local stakeholders (both government and non-government partners) are engaged over a prolonged period. By following this strategy, programs can achieve their core objectives and produce positive externalities resulting from local reform.  Having the right partner is key to success. In an operating context of a transitional economy, it is necessary to choose a partner that is capable of learning and delivering on both technical outputs and outreach activities. In the case of Vietnam, VCCI’s legal department is a good example of an organization’s flexible structure allowing it to continuously adapt during a long learning process and then deliver on technical and outreach activities. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 8  Capacity building is a key factor that enables and motivates local partners. By learning, adapting and ultimately being capable of operating independently, local partners can solidify ownership over activities and programs. By having ownership, partners can also mobilize resources not directly related to the program to support program activities. Capacity building is a process, not an ad hoc activity. For the PCI program, it took several years with direct and indirect (through USAID/VNCI) assistance from USAID/Vietnam before the PCI team became technically independent.  Knowledge-sharing is key in supporting reform efforts. Information can help guide stakeholders on program rationale, activities and general outcomes. Informed stakeholders can help push reform agendas forward.  Independent evaluations are critical to ensure program alignment and credibility. Evaluations help programs identify and maximize core strengths. They can also help validate activities and achievements to stakeholders at all levels.  Employing mass media in an effective way not only raises awareness about reform activities, it can also be a source of legitimate pressure on stakeholders for reform. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 9 I. INTRODUCTION Provincial Competitiveness Index USAID launched the Provincial Competitiveness Index (PCI) in 2005 under the Agency’s Vietnam Competitiveness Initiative (VNCI). From 2005 until 2013, prime contractor Development Alternatives International (DAI), principal subcontractor The Asia Foundation and local partner Vietnam Chamber of Commerce and Industry (VCCI) implemented the PCI activity. Starting in 2013, USAID awarded a direct grant to VCCI to carry out the PCI activity until 2016. PCI aims to promote private sector growth in Vietnam by supporting provincial governments to improve business-enabling environments for sustained growth through improved economic governance. To realize this goal, PCI has organized activities around three main objectives: 1) Promote public-private policy dialogue on economic governance through the PCI survey; (2) Increase understanding of economic governance based on the results of the PCI survey; and (3) Improve economic governance capacity in targeted provinces. As part of the overall program, the PCI team conducts annual activities that include these core pieces: 1. A survey of domestic direct-invested (DDI) enterprises; 2. A survey of foreign-invested enterprises (FIEs); 3. A launch event to disseminate survey results; and 4. Diagnostic workshops conducted by the PCI team to assist provincial officials in understanding the factors behind provincial survey scores, with the goal of improving their scores. Diagnostic workshops take place at both the regional and sub-regional levels. In addition to the core PCI activities, the PCI team conducts complementary activities that include: 1. Maintaining the PCI website to provide the public with a consistent source of information; 2. Providing PCI training courses for business associations to facilitate their understanding of PCI data for public-private dialogue; 3. Providing PCI training courses for National Assembly members to facilitate their understanding of PCI data for policymaking; 4. Conducting baseline studies of provincial policymaking to issue recommendations so local governments can enhance the transparency and quality of those processes; 5. Supporting local governments to prepare or revise provincial socio-economic development strategies/plans; 6. Conducting independent studies of local governments’ websites to encourage them to provide sufficient and timely information to citizens; 7. Hosting regional forums to facilitate sharing of experiences related to improving local economic governance; 8. Creating a handbook on public-private dialogue as a tool for business associations and local governments to address difficulties and obstacles related to business environments; and 9. Providing PCI training courses to VCCI regional staff to ensure sustainability of the program. Proposed Key PCI Objectives 1. Promote public-private policy dialogue on economic governance through the PCI survey. 2. Increase understanding of economic governance based on the PCI survey results. 3. Improve economic governance capacity in targeted provinces. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 10 II. EVALUATION PURPOSE AND METHODOLOGY Evaluation Purpose USAID/Vietnam requested the Vietnam Evaluation, Monitoring and Survey Services (VEMSS) Project to assess key PCI program results and contributions to reform and improvement of local economic governance in selected provinces. The evaluation will serve as a base of understanding for ongoing and future technical assistance to the Government of Vietnam (GVN) to improve the country’s business-enabling environment. Evaluation Questions This evaluation focused on the following key research questions and sub-questions: 1. What progress has been made in achieving the PCI program’s objectives? a. To what extent has the PCI program aligned with the priorities of the central and local governments’ efforts to promote an enabling environment for private sector businesses? b. What are some examples of economic governance reform practices (success stories) within the provinces that are related to the PCI program? c. What are key drivers and constraints (subjective and objective) that contribute to the realization (or lack thereof) of the three objectives? d. To what extent have data collection, storage, processing and analysis activities complied with survey protocol and procedures? e. What efforts have been made to ensure the PCI program’s sustainability, both technically and financially? 2. How has the PCI program contributed to business environments, particularly in relation to administration reform? a. What is the extent to which private sector enterprises perceive the themes represented in the 10 sub-indexes to be crucial to their business operations and objectives? b. What role, if any, do contextual factors (e.g., proximity to commercial centers, access to natural resources, geographic location) play in influencing local economic governance reform? Methodology The evaluation team included three national experts, one international expert and two national research assistants. The team performed the following tasks from mid-November 2015 until the end of January 2016: Desk Study At the outset, the evaluation team collaborated with USAID and VCCI to collect materials containing PCI￾related quantitative and qualitative data. Provincial stakeholders provided additional documentation during fieldwork. The team conducted an in-depth review of the documentation, which provided them with background data/information regarding the progress of PCI implementation; domestic macroeconomic growth; provincial business regulatory environments; and foreign and domestic investment in Vietnam. Reviewed materials included: 1) PCI program design documents (program description); 2) PCI program implementation reports (quarterly reports, technical reports and presentations); 3) contextual literature (related academic publications and mass media articles); and 4) government documents (provincial PCI analysis reports, departmental action plans and presentations). Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 11 Field Visit and Stakeholder Interviews As the primary source of data, the evaluation team conducted interviews with numerous stakeholders during field visits to the provinces of Hanoi, Thai Nguyen, Tuyen Quang, Da Nang, Ninh Thuan, Dong Thap and Ca Mau. The team selected these provinces in coordination with VCCI and USAID based on criteria that would ensure geographic diversity and an adequate representation of high, middle and low performers based on PCI scores. During visits to each province, the evaluation team interviewed provincial government authorities, representatives of provincial and district businesses associations and private enterprises. While in Hanoi, in addition to interviewing provincial government authorities, the evaluation team conducted interviews with representatives from the central government and VCCI, as well as independent researchers and government advisors. In total, the evaluation team interviewed 143 government stakeholders at both the central and local levels; seven academic and independent researchers; 34 provincial and district businesses association stakeholders; and 30 private enterprises. Annex 1 contains a complete stakeholder list; interview guides are in Annex 2 and Annex 3. Private Enterprise Survey The evaluation team surveyed 387 private enterprises in the seven provinces it visited during fieldwork. The survey provided insight into how businesses perceive their local operating and regulatory environments. To conduct the survey, the team randomly selected private enterprises from registration lists that provincial business associations (PBAs) in each of the visited provinces provided. PBA staff then distributed surveys to the selected private enterprises, after which PBA staff sent completed surveys to the evaluation team. The complete enterprise survey data collection plan and questionnaire are in Annex 5. Provincial Government Stakeholder Survey Provincial government authorities completed a brief questionnaire during interviews. The questionnaire provided the evaluation team with additional information on how PCI has affected local economic governance and contributed to reform. The team collected 123 questionnaires completed by government stakeholders. The complete text of the questionnaire is in Annex 4. Data Analysis and Report Writing Evaluation team members at VEMSS in Hanoi performed data synthesis and analysis and developed this report. The team utilized the motivations, opportunities and analysis (MOA) framework to guide the analysis and identify PCI’s contribution to local economic governance reform and, ultimately, an improved enabling environment in the provinces the team visited. The team presented its preliminary findings to provincial officials during PPC debriefing meetings conducted at the end of field visits. The team also conducted meetings with USAID and VCCI in January 2016 to present and discuss preliminary findings. Limitations and Challenges Given the breadth of the PCI program and its implementation timeframe, this performance evaluation can provide only a snapshot of the program. Additionally, given the tools and the methods employed for data collection and analysis (including the absence of a rigorously defined counterfactual), the performance evaluation may not necessarily establish direct causal links between the PCI program and improved enabling Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 12 environments or causal relationships between local economic governance reforms and resulting levels of investment at the provincial level. Additionally, given the structure of the PCI program and the timeframe associated with the performance evaluation, the team focused primarily on the program’s effects on administration reform and economic governance at the provincial level and was unable to comprehensively analyze the PCI program’s direct impact on domestic private sector growth. The team also acknowledges two biases associated with conducting individual and group interviews. One is the difficulty of recall among key informants. To reduce any bias associated with recall, the team framed interview guides using questions to aid in accurate recall. A second bias is the subjectivity of self-reported data. To mitigate this, the team attempted to corroborate information between respondents and across data collection methods. III. COUNTRY CONTEXT Country Performance Vietnam, with a population of 89.7 million people and a per capita gross domestic product (GDP) of USD $1,901.70, 1 is classified as a middle-income country with its growth trending upward in the last two years. During 2012 and 2013, the country experienced an economic slowdown; however, in 2014 and 2015, Vietnam rebounded with 6 percent and 6.5 percent growth rates respectively.2 According to the World Bank, strengthening consumer confidence and low inflation have driven an expansion in private consumption and exports of foreign-invested manufacturing increased. Additionally, exports of foreign-invested manufacturing and overall investment have been rising. Investment levels in 2015 — fueled by foreign direct investment, increased government expenditures and credit growth — have increased 8.5 percent over 2014 levels. Table 1: Changes in Vietnam’s Economy from 2013 to 2015 Economic Measurement 2013 2014 2015 GDP growth (%) 5.4 6.0 6.5 CPI (annual average %) 6.6 4.1 1.5 Current account balance (% of GDP) 5.6 4.8 0.1 Fiscal balance (% of GDP) – GFS definition -7.4 -6.2 -6.9 Public debt (% of GDP) – Ministry of Finance definition 54.5 59.6 61.3 Source: World Bank Despite Vietnam’s recent economic growth, the country faces challenges. In regard to the World Economic Forum’s Global Competitiveness Index, Vietnam ranks behind its regional neighbors Malaysia, Philippines, Thailand and Indonesia in overall competitiveness. And according to the World Bank’s Doing Business Survey, Vietnam has shown only marginal improvements in infrastructure and institutional framework while further lagging in protecting minority investors, access to electricity and addressing business insolvency.3 Government and Administration Reform In Vietnam, the rationale and practice of administrative reform has been widely known since the 1990s and the central government has implemented a number of policies that support continued administration reform, 1 World Economic Forum, “Global Competitiveness Index.” 2 World Bank, “Taking Stock: An Update on Vietnam’s Recent Economic Developments.” Hanoi, December 2015. 3 World Bank, “Doing Business 2016: Measuring Regulatory Quality and Efficiency,” Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 13 especially as they relate to improving the domestic business environment. Table 2 notes major legislation passed by the central government in the area of administrative reform over the last several years. Table 2: Major GVN Legislation on Administrative Reform Since 1995 Resolution 30c and Resolution 19 are particularly relevant to the PCI context. Resolution 30c is the foundation of the central government’s recent efforts at administration reform. A significant portion of the resolution specifically aims to create a more favorable operating environment for enterprises by streamlining administrative procedures. Resolution 19 sought to build on the movement to improve the domestic business environment and represents an important document for the government to manage reform, aiming to ultimately narrow the gap in the business environment between Vietnam and the Association of Southeast Asian Nations (ASEAN) 6 and ASEAN 4 countries. The resolution aims to reduce the tax payment period, develop and disclose value-added tax (VAT) refund databases and shorten processing time for exports, imports and bankruptcy procedures. In addition, the resolution calls for more active participation of business associations and professional bodies in policy review, as well as closer coordination with government bodies to support business development and resolve business issues. VCCI, the implementing partner of PCI, plays a key role on the Resolution 19 monitoring committee. Both resolutions illustrate the central government’s proactive approach to enhance the domestic business environment. Economic Context of Provinces Visited Table 3 illustrates basic macro-economic data for each province the evaluation team visited. Legislation Year Description Public Administration Reform (PAR) Program 1995 Aimed to improve the efficiency and effectiveness of public service delivery. Public Administration Reform Master Program (PAR-MP) 2001 – 2010 Extended the reform agenda to almost all public service sectors. Resolution 30c 2011 Extended the implementation of PAR-MP until 2020 with the intention of “creating an equal, open, favorable and transparent business environment to reduce time and costs for enterprises of all economic sectors in the compliance with administrative procedures.” Resolution 36a 2015 Implemented information technology modernization within line ministries and subordinate departments. Resolution 19 2014 Aimed to improve the domestic business environment by further decreasing administrative procedures, improving intellectual property rights, reducing insolvency procedures, increasing electricity access for businesses and investment projects and ensuring transparency and equality in the market. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 14 Table 3: Basic Macro-Economic Data by Province Thai Nguyen Tuyen Quang Hanoi Da Nang Ninh Thuan Dong Thap Ca Mau Economic Zone North East North West Red River Delta South Central Coast South Central Coast Mekong River Delta Mekong River Delta Population 1.17 million 753,800 7.06 million 1.01 million 590,400 1.69 million 1.2 million GDP per capita* (VND) 37.3 million 25.3 million 72.9 million 116.5 million 26.4 million 30.8 million 33.2 million Number of enterprises (2013) 2,090 843 87,343 9,521 1,114 2,114 2,305 Key Industries Manufacturing, agriculture Agriculture, forestry and fishing, manufacturing, construction Manufacturing, construction, wholesale and retail trade Manufacturing, wholesale and retail trade, construction Agriculture, forestry and fishery Agriculture, forestry and fishing, manufacturing Fisheries, manufacturing Budget Revenue (VND) 10,384.3 billion 7,269.5 billion 130,100 billion 20,281.2 billion 2,881.9 billion 7,586.5 billion 3,320 billion Investment (VND) 36,382 billion 5,891 billion 313,214 billion 32,782 billion 7,125.1 billion 10,974 billion 10,334 billion Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 15 Figure 1 shows the PCI composite scores of each province visited by the evaluation team during the last five years. As indicated, Da Nang and Dong Thap have consistently been top performers in this time, while Ca Mau and Tuyen Quang have performed at the bottom of the group during the last three years. The scores for Hanoi and Thai Nguyen have improved during the last three years. Figure 1: PCI Composite Scores by Province (2010–2014) Figure 2 shows that since 2007, the overall economic governance quality of the group of seven provinces visited as well as all 63 provinces has trended upward over the last eight years. This is one indicator of the improvement in overall effectiveness of reform at the provincial level. 45 50 55 60 65 70 2010 2011 2012 2013 2014 PCI Composite Scores by Provinces (2010-2014) Thai Nguyen Dong Thap Da Nang Hanoi Tuyen Quang Ninh Thuan Ca Mau Figure 2: The Median PCI Value Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 16 IV. CORE PCI OBJECTIVES AND ACHIEVEMENTS 1. Survey Implementation This section provides a summary PCI survey implementation and launch event. The section also provides an analysis of the survey design and implementation process, the PCI program’s outreach strategies, stakeholders’ overall impression of the program and provincial officials’ use of the PCI data. Activity Description Domestic Private Sector Survey The PCI survey of domestic direct-invested firms is the program’s flagship activity. The annual survey includes three major implementation phases. 1. Phase 1: Includes the development and dissemination of the survey questionnaire. The questionnaire aims to gather information about the survey’s 10 sub-indexes, which are: entry costs; land access and security of tenure; transparency and access to information; time costs and regulatory compliance; informal charges; policy bias; proactivity of provincial leadership; business support services; labor and training; and legal institutions. Questionnaires are sent to a random sample of domestic firms (stratified by firm age, legal type and sector) that reflect provincial populations. Once questionnaires are mailed out to selected enterprises, the PCI team conducts follow-up calls with a portion of the sampled firms to confirm that they received the questionnaire and to answer any questions that the firms have. 2. Phase 2: Once the team collects data from the sampled firms, they analyze it and calculate the 10 sub￾indexes based on a 10-point scale. Scores are based on the total number of responding firms. In 2014, 9,859 domestic private enterprises participated in the survey; 8,091 completed the entire questionnaire. 3. Phase 3: When it calculates the 10 sub-indexes for each province, the PCI team calibrates the composite PCI as the weighted mean of 10 sub-indexes, with a maximum score of 100 points. Foreign Investment Survey The separate PCI survey of foreign-invested enterprises presents the state of foreign direct investment in Vietnam. The survey’s implementation is similar to that of the domestic private sector survey; however, the FIE survey focuses on international firms in Vietnam. Firms are randomly selected from the 14 provinces with the highest concentration of FIEs. Using lists from the General Tax Department (GTD), the PCI team creates a stratified random sample at the provincial level to ensure that the sampled firms mirror the population of Sub-Question 1a: To what extent has the PCI program aligned with the priorities of the central and local governments’ efforts to promote an enabling environment for private sector businesses? Sub-Question 1b: What are some examples of economic governance reform practices (success stories) within the provinces that are related to PCI program? Sub-Question 1c: What are the drivers and constraints (subjective and objective) that contribute to the realization (or lack thereof) of the three objectives? Sub-Question 1d: To what extent have data collection, storage, processing and analysis activities complied with survey protocol? 11 Situation room met!its!annual!target 2.3.!Implementation!of!the!2014!PCI!survey! Design!of!PCI!survey!package! ! Activities!of!mailBout!survey!and!situation!room! Students conduct follow-up calls to enterprises for a PCI survey. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 17 foreign firms in the province. Questionnaires for the Foreign Investment Survey gather information and explore trends in country of origin, sector and size of FIEs, as well overall performance. The survey examines how FIEs view the business environment in Vietnam in relation to regional neighbors and identifies specific weaknesses in the country’s operating and regulatory environment. In 2014, the Foreign Investment Survey included 1,491 businesses from 43 countries, which represents 8.6 percent of the population of foreign-invested projects (17,434) that have registered in Vietnam since 1988, according to the General Statistics Office, and nearly equals the number of FIEs that entered Vietnam in 2013.4 Achievement As Table 4 shows, PCI achieved the target number of firms participating in the survey during the 2013–2014 and 2014–2015 periods. The evaluation team notes that the overall response rate for both surveys was 24.7 percent for 2013–2014 and 23.7 percent for 2014–2015, which represents a reasonable response rate for a mailed survey. However, the overall response rate is greatly affected by the quality of the sampling frame, which in this case is firm data that VCCI received from the GTD. This data may not be updated with the most current firm information; VCCI noted that many firms contacted for survey participation may have relocated or stopped operating altogether. In fact, research by the PCI team indicates that listings for around 50 percent of the firms in the GTD database may not have updated information.5 Nevertheless, the PCI team must continue to use the registration lists that the GTD provides, as they are the most comprehensive lists available. Upon completion of the two PCI surveys, the team compiles the results into a comprehensive report. To generate awareness of the survey research findings, the PCI team organizes a launch event to disseminate the findings among relevant stakeholders, including provincial officials, private sector representatives and business associations. The team invites members of the media to raise awareness of the findings among the general public. As indicated in the PCI table that follows, launch events for the 2013–2014 and 2014–2015 periods exceeded targets related to the release of the PCI report, including participation of provinces, provincial leaders, businesses and business associations in the launch event and generating awareness of PCI as measured through media coverage. A deeper look at the indicators reveals that the number of provincial leaders participating in the launch significantly exceeded targets for both periods and increased from 2013–2014 to 2014–2015. This represents a positive sign that high-level policymakers who have power to set local agendas are increasingly interested in PCI and how the survey results inform their own policy planning. While the program met its targets for participation, the number of participants from business associations did not significantly exceed targets and increased by only one participant from 2013–2014 to 2014–2015. This may be a result of the PCI team’s selective outreach to business associations. It may also result from a lack of interest or resources on the part of business associations to send representatives to the launch event in Hanoi. 4 2015 PCI Report 5 2015 PCI Report Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 18 Table 4: PCI Launch Targets and Achievements, 2013–2015 Indicator 2013–2014 2014–2015 Target Achieved Target Achieved Enterprises participating in regulatory advocacy through PCI survey 9,500 9,702 9,600 11,350 Annual PCI reports released 1 1 1 1 Launch participants 400 470 420 601 Provincial leaders participating in the launch 15 23 25 54 Provinces participating in the launch 35 56 40 62 Business associations participating in the launch 15 20 20 21 Media stories about the PCI 100 280 110 216 Resistance to the PCI score and ranking was high across provinces early in the launch of PCI, especially from the economic hubs/cities and provinces with large economic endowments. However, the general acceptance of PCI by provincial officials had increased dramatically by the time of evaluation in all provinces, including those that initially opposed the results and ranking methodology. “In Thai Nguyen, PCI has been popular and taken into account all line departments’ action plans. It was not like this about five years ago, when many agencies did not even really understand nor show interest in the initiative.” — Thai Nguyen PBA “In the early days, both Ho Chi Minh City and Hanoi strongly opposed the PCI program. I observed that their views and reactions have changed a lot. They both took serious collective actions at the city/department levels and have improved their scores and rankings.” — Independent Expert PCI also surpassed its communication and media targets for both the 2013–2014 launch event (280 media stories related to the launch) and 2014–2015 launch (216 stories). These indicators refer specifically to media stories related to the launch event and do not represent media stories regarding the PCI survey or workshops at the provincial level. As such, the total number of media stories related to PCI would be much larger than just those related to the launch, as indicated in the table. The article that follows from the newspaper VietNamNet highlights PCI results in Ho Chi Minh City (HCMC) and represents how the mass media has used PCI to highlight issues of economic governance. HCM City Seeks to Improve Business Environment The HCMC government will focus on improving the business environment and streamline administrative procedures this year and next to make life easier for investors. In a plan issued last week, the city will reduce the time for tax payment to no more than 121.5 hours per year, goods export to 13 days and goods import to 14 days, business registration to three days, and business closure from 60 months to 30 months. These reforms will be carried from now to the year-end. Next year, the time needed for registering for rights to ownership and use of assets will be brought down from 57 days to 14 days, settling trade disputes from 400 days to 200 days, and processing a bankruptcy request from 60 months to 24 months. According to the city government, the competitiveness of the city is still lower than a lot of other provinces in the country. Of the 10 sub-indexes in the Provincial Competitive Index (PCI), namely entry costs, land access and security of tenure, transparency, time costs, informal charges, proactivity of the provincial leadership, policy bias, business support services, labor training policy and legal institutions, HCMC has seven below average. After 10 years in the PCI survey, HCMC got into the top five of the best-governed provinces and cities in Vietnam. But its ranking was still below Danang, Dong Thap and Lao Cai, though it is an attractive destination for many domestic and foreign investors. — VietNamNet, Oct. 7, 2015 Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 19 PCI was also recognized by the Central Party as one of the 62 projects that have had enormous contribution to the socio-economic development of the country during 2011–2015. A member of the PCI team noted, “This is the first USAID-funded project to receive such award from the GVN.” This award represents a major accomplishment for the PCI team and is a significant recognition of the program’s management and contribution to officials at the central and provincial levels. A number of published peer-reviewed academic journal articles have used PCI data. Examples include Hubert et al. in Development Policy Review 33.2 (2015), Tran et al. in “Institutions Matter: The Case of Vietnam” and other pieces in that series in The Journal of Socio-Economics (2006, 2008, 2009). Findings/Stakeholder Views Political commitment has contributed to the engagement and effectiveness of PCI implementation at the national and provincial levels. The PCI team has received consistent and strong support from both VCCI and U.S. counterpart leaders. The prime minister’s advisory group has also shown a strong commitment, boosting the PCI brand in events and discussions and ultimately helping the program surpass initial stakeholder resistance. Additionally, in all provinces visited, the People’s Provincial Committee (PPC) chairs have all enacted directives or provincial plans to mandate every line department’s participation in PCI improvement activities. This represents a unique case of such high levels of political commitment for a technical assistance initiative. Shifting from supply-driven public services to the demand-driven services has improved the understanding of economic governance issues. Prior to PCI’s launch, most line departments at the provincial level often implemented mandatory administrative procedures with little interaction or input from the business community. PCI has exposed provincial officials to the concept of seeking input and feedback from the business community to properly identify relevant economic governance issues and then provide them with quality public services. All stakeholders indicated that PCI has been successfully implemented and the program is relevant and credible. In a mini-poll of provincial stakeholders, 95 percent of respondents indicated that they used PCI “sometimes” or “frequently” in their daily activities. A provincial stakeholder noted, “The PCI is considered seriously and widely used in official documents in our province. Both provincial government and party leaders highly appreciate it.” All stakeholders reported that PCI was an effective evidence-based tool that has supported economic governance reform efforts at the provincial level as well as engagement with the business community. Provincial stakeholders in Thai Nguyen reported that they used PCI as the foundation of their provincial reform process that began in 2011 and included the streamlining of administration procedures and modernization of information technology across line departments. As part of the reform process, provincial officials also sought to actively engage the business community to seek input on how to address issues highlighted by PCI sub-indexes. All provincial stakeholders from the seven provinces visited by the evaluation team indicated their desire to see the “We have taken advantage of the PCI annual review to conduct an internal assessment of the division of tasks and responsibilities. During the review, we [became] better able to establish work plans and timelines and assign specific tasks and outcomes for specific divisions based on input from the business community.” — Department of Natural Resources and Environment (DoNRE) Representative Thai Nguyen The evaluation team found evidence of robust data quality management measures in place for the data collection and analysis phases of the PCI survey. The PCI team has taken steps to minimize the non-response rate through comprehensive follow-up procedures, minimize the impact of human error at all stages of data collection and entry by applying updated working protocols for staff working in the “situation room” and employing software to help identify any evidence of data tampering or cheating. 2 3 22 73 Never Occasionally Sometimes Frequently Figure 3: PCI Data Use by Government Stakeholders (Percentage) Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 20 PCI program continue in the years to come, and those from the top performer recommended that the PCI team integrate some new ideas or innovative aspects of economic governance into the sub-indexes. This suggests that the PCI program is still relevant and will continue to be in the near future, especially for low-performing provinces. Nevertheless, high-performing provinces such as Da Nang and Dong Thap have already begun to think about and address issues that go beyond PCI. Third-party stakeholders and the PCI team agree that a driving force behind the credibility of the PCI survey is VCCI’s recognition as an independent organization that has put an emphasis on maintaining the methodological rigor of the survey. One PCI team member reported that ensuring technical rigor is an ongoing process, guided by data collection and analysis guidelines. The team member pointed to the example of developing a script for the group making follow-up calls to enterprises that received questionnaires. During the first several years, team members making follow-up calls did not use a standardized script. As a result, the team began to notice effects in the data from varied and inconsistent information that the group gave to firms during follow-up calls. To mitigate this, the PCI team developed a script to use during calls, ensuring consistent treatment of respondents. A stakeholder with the National Economics University of Vietnam added that the survey adhered to international technical rigor standards and was a useful, credible tool for both policymakers and researchers. The international stakeholder also credited PCI’s mass media campaign with boosting the index’s credibility and ability to push provinces to keep economic governance reform as a policy priority. He elaborated that the significant attention paid to provincial scores has helped establish a “sense of competition” between provinces and incentivizes them to make reforms, ultimately increasing their scores. The vast majority of stakeholders in the visited provinces agree that PCI indexes and sub-indexes reflect the actual performance and development of the province. “We of course do not feel happy or pleased about our scores and rank by PCI in the last year, but we agree that the PCI score did reflect the situation in our province.” — Ca Mau PPC Briefing “The PCI sub-index has pointed out the weakness and loopholes in our system for our line departments and agencies to improve, especially regarding the ability to deepen the analysis of each line department.” — Thai Nguyen PPC Briefing “We have conducted an independent local survey to verify the results of PCI. However, we have discontinued the survey because after several rounds of the survey, we have verified that the PCI results accurately reflect the perceptions of businesses in Da Nang on economic governance issues.” — Representative, Institute for Socio-Economic Development Da Nang The PCI team acknowledges some continued challenges associated with survey implementation. First, the current response rate of the domestic private sector survey is about 30 percent and the data indicates that the engagement of firms in the FIE surveys is on a slight downward trend. While the response rate for the surveys can be considered reasonable for a mail-out survey, such an approach may be susceptible to non-response bias, which occurs when the businesses that respond to the survey differ in significant ways from those that did not respond to the survey. Additionally, cheating and coercion by local officials remains a concern. To address cheating, the team employs statistical techniques to identify and remove scores that appear to be manipulated. The team has continued using a paper questionnaire, having experienced higher rates of tampering in the online format. A couple of provincial stakeholders still contest the validity of the PCI data. Officials from Ca Mau noted that the labor and training sub-indexes did not accurately reflect the situation in that province. An official in Ninh Thuan reported that the PCI survey indicates that the median time businesses have to spend dealing with the local tax office has increased to 3.5 hours, but his department’s data reports a much lower median time. This questioning of data validity may be driven in large part by a general lack of understanding of the survey methodology. In particular, some officials were not familiar with survey sampling techniques and underlying statistical meaning. As a result, they indicated that they felt that the questionnaire might have been sent only to businesses that had a strong opinion about the provincial government. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 21 Some firms reported difficulties responding to the survey questionnaire. Several noted that the questionnaire was long and time-consuming, took several days and required multiple staff to provide the information due to the complexity of some questions. In Ninh Thuan, one firm owner indicated that she did not understand some of the items on the questionnaire, especially those that pertained to land clearance, as the topic did not necessarily relate to her business. As a result, she was unclear about how to appropriately respond and left most of the section blank. In Ca Mau, several firms that participated in the survey left some sections of the survey blank because they said they did not fully understand the questions. Improving the Provincial Competitiveness Index of Thai Nguyen During the initial phases of the PCI survey, Thai Nguyen consistently ranked in the lower half of provinces. In 2010–2011, the province was hit by additional troubles with the regional economic recession and the implementation of domestic fiscal scrutiny, which resulted in a dramatic reduction in budget revenue. As national policies on exploring and exporting minerals (Thai Nguyen’s most prominent economic sector) tightened, private sector investment in the province fell to its lowest levels. This poor economic performance forced provincial leaders to act and establish reforms in an attempt to escape the recession. Investors urged PPC leaders to address the provincial PCI ranking, which at the time was 57 out of 63. The PPC chairman called for a provincial meeting of line departments and discovered that Thai Nguyen was steps behind other provinces in terms of addressing PCI survey results. The Provincial People’s Committee hosted a series of meetings and determined that improving economic governance on factors measured by the PCI index and sub-indexes, which included infrastructure, economic governance and quality of provincial planning, was an effective way to create a foundation for long-term growth. PPC leaders assigned a deputy chairperson to lead the development and implementation of a PCI action plan and business environment improvement program. Soon after, the deputy chairs began developing engagement events with local enterprises. A PCI task force was formed to engage directly with enterprises, as well as aggregate and analyze their input. Each line department, with support from the Department of Planning and Investment (DPI), developed its own action plan that assigned specific roles for staff to address issues raised by enterprises. A separate set of meetings for provincial department directors was established in addition to the monthly PPC meetings to ensure information flow and coordination. During implementation of PCI action plans, the PPC took an active role in monitoring each department to ensure that action plan activities were implemented on schedule. Directors of departments that fell behind were penalized for poor performance. In 2013, PPC established the Provincial Business Association (PBA), which would subsequently played a critical role. Since its establishment, the PBA’s role has grown substantially by expanding its membership to a wider number of enterprises and established district business associations. Currently, the PBA consistently communicates with PCI task force members and often leads the technical discussion around PCI results. As a result, the confidence and satisfaction of the Thai Nguyen business community with the provincial public services and governance quality is at its highest level ever. The province also achieved a tremendous economic result, with budget revenues of VND 7,000 billion in 2014, compared to less than VND 1,000 billion in 2010. The foreign direct investment for the last three years was 25 times the province’s total FDI in the previous 25 years. Thai Nguyen became a rising province in terms of making reforms and improving its economic performance via both the domestic and international business communities. Conclusions  A key driver of PCI’s success is the program’s political leadership and its consistent support for program implementation and outreach activities.  Implementation of both the Domestic Private Sector Survey and the Foreign Direct Investment Survey has proved successful and stakeholders confirm that PCI results are relevant and support an evidence￾based approach for highlighting pertinent economic governance issues in provinces.  A key driver of success for PCI is the credibility of VCCI as a local partner and the continuous efforts made to ensure that robust data collection protocols are in place. Figure 4: PCI Graphic of Thai Nguyen, 2007–2014 Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 22  The PCI team has shown a holistic approach and continuous effort to comply with and improve data collection protocols, practices and total quality management.  The team has identified success stories regarding PCI-supported administration reform, including ongoing economic reform planning and implementation in Thai Nguyen that started in 2011. PCI results gave provincial officials the initial motivation and foundational framework for understanding and addressing reform issues.  Challenges related to survey implementation may constrain the program’s success. The length and complexity of the survey questionnaire has affected certain firms’ ability to respond. Also, challenges remain in the decreasing involvement of firms in the FIE survey and attempts at manipulation and tampering with provincial officials’ questionnaire responses. In addition, some provincial stakeholders’ lack of understanding about the PCI score by may result in inappropriate interpretation. 2. Promotion of Public-Private Dialogue Sub Question 1a: To what extent has the PCI program aligned with the priorities of the central and local governments’ efforts to promote an enabling environment for private sector businesses? Sub Question 1b: What are some examples of economic governance reform practices (success stories) within the provinces that are related to the PCI program? Sub Question 1c: What are the drivers and constraints (subjective and objective) that contribute to the realization (or lack thereof) of the three objectives? This section summarizes activities to increase and promote public-private dialogue (PPD). This section also includes an analysis of PPD activities that the PCI program directly sponsors, which include diagnostic and regional workshops. Additionally, the section will offer an analysis of PPD activities that are sponsored and conducted directly by provinces, such as the business café and Provincial Business Associations). Activity Description In addition to the PCI report launch, the PCI team and VCCI regional offices organize activities such as hosting and organizing diagnostic workshops at the provincial and regional levels. Provinces fund and lead diagnostic workshops, with technical support from the PCI team. Workshops aim to help stakeholders understand the factors driving PCI scores and facilitate dialogue between provincial officials and private sector representatives with the goal of helping provincial officials develop reform plans. Workshops generally include a review and analysis of PCI sub-index scores while also covering numerous specific topics. These topics vary between provinces, depending on the issues provincial officials want addressed. Workshops to date have focused on issues that include streamlining business tax payments, enhancing provincial transparency and support for developing comprehensive provincial economic growth strategies. Workshop participation generally includes both government officials and representatives of business associations, with the PCI team facilitating. In addition, provinces organize and directly host activities that include business cafés, meetings and social events. These events are developed and implemented locally and intend to facilitate direct discussion between provincial officials and private enterprises. For example, provinces including Tuyen Quang and Dong Thap have implemented the business café. PPC leaders hold open office hours at the café, usually within the PPC building, for enterprises to speak with provincial leaders. It is meant to be an open, friendly and informal exchange of ideas between enterprises and provincial leaders. Provincial leaders wanted to create an environment that would 8 ! ! ! ! PCI#workshop#in#Hoa#Binh# PCI#workshop#in#Quang#Nam# # ! Best#practices#in#promoting#an#enabling#business# environment#in#Da#Nang# # ! ! 2.2.!Regional!workshops!! One!of!the!most!remarkable!events!in!this!quarter!is!the!organization!of!a!regional!workshop!to!share!best! practices!among!the!central!and!highland!provinces!on!Aug!29!and!the!participation!in!regional!workshop!in!the! South.! 2.2.1.!Regional!workshop!to!share!best!practices!in!promoting!an!enabling!business!environment! The!regional!workshop!on!best!practices!in!promoting!an!enabling!business!environment!was!successfully! organized!in!Da!Nang!on!August!28 th !attracting!nearly!100!representatives!from!departments!and!agencies,! business!associations,!research!academies!and!media!in!15!provinces!in!Central!Coast!and!Highland:!Dak!Nong,! Kon!Tum,!Dak!Lak,!Gia!Lai,!Khanh!Hoa,!Phu!Yen,!Binh!Dinh,!Quang!Ngai,!Quang!Nam,!Quang!Binh,!Quang!Tri,!Hue,! Da!Nang,!Nghe!An,!Ha!Tinh.!! The!workshop!aimed!at!creating!a!forum!for!provinces!to!share!and!learn!their!best!practices!and!experiences!in! promoting!an!enabling!business!environment,!especially!improving!the!provincial!business!environment!from!the! institutional!changes!and!the!experiences!in!implementing!initiatives!to!improve!the!business!environment!and!to! PCI diagnostic workshops like this one in Quang Nam in 2014 help stakeholders understand what drives scores. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 23 allow firms to feel more comfortable in expressing their viewpoints and concerns. Often, communication with provincial officials is rigid and formal. With the café, all firms are welcome to engage over morning coffee. Da Nang: Unique Approaches to Public-Private Dialogue In Da Nang, provincial officials have taken significant steps earlier than the other visited provinces to promote and encourage dialogue between private enterprises and local government. As a result of PCI, Da Nang recognized the importance of conducting PPD activities to engage and gather input from the local business community. Events began as basic meetings between provincial officials and business associations. As dialogue events continued over the years, provincial officials recognized their increasing value and constantly sought new ways to engage the business community. Starting in 2015, representatives from local business associations were invited to attend monthly PPC briefings on a regular basis. Representatives of the local business associations took advantage of this invitation and some began attending the meeting regularly. During the meetings, business association members can listen to government officials propose and discuss potential reforms, as well as hear updates on reforms that have already been implemented. Representatives can also voice their own opinions and provide input when necessary directly to provincial officials. A provincial official in Da Nang noted that inviting representatives from business associations was an attempt to increase confidence in the policymaking process, as well as a way to keep enterprises informed about the implementation of the recent reforms. He went on to report that such meetings contribute significantly to enhancing the way government officials view business associations and private enterprises. Private enterprises are viewed more and more as relevant players in the policy making and review process at the provincial level. Over time, this has improved the ability of business associations to stay informed on a regular basis and to sustain dialogue and interact with provincial line departments regarding issues of economic governance reform. Achievements In regard to the delivery of diagnostic workshops, the program has met or exceeded nearly all targets. The one unrealized target was the percentage of government officials participating in the workshops in years 1 and 2. Workshops were well attended, but the PCI team was unable to conduct them in each province. About 50 percent of the lower-performing provinces (the bottom 32 provinces as indicated by 2014 PCI score) were able to access technical assistance from the PCI team through diagnostic workshops. Some provincial officials indicated that they were unfamiliar with the process for requesting technical assistance from the PCI team, while others noted that their requests were yet to be addressed by VCCI. Eight provinces have even organized their own PCI diagnostic workshops: Tra Vinh, Binh Thuan, Kien Giang, Hoa Binh, Binh Duong, Bac Ninh, Tuyen Quang and Vinh Long. Insufficient data exists on the quantity and quality of the events. The PCI website has also seen significant traffic and the program surpassed its target for total site visits during the 2014–2015 period. In the last quarter of that period, site visits reached 33,617, with 24,540 new visitors. The website is an easily accessible and useful tool for provincial stakeholders. Provincial officials in Dong Thap and Thai Nguyen reported that they often use the site to view provincial scores; they demonstrated for the evaluation team how many of their staff in the relevant line departments such as DPI are able to access the site to view scores or download full and provincial reports. The PCI mobile application launched in March 2014 to provide senior officials with a tool to quickly access PCI score information, even offline. The mobile app was available for Android and Apple operating systems. After its launch, new users of the application rose steadily. In the first full quarter of operation, the mobile app registered 1,253 new users. The following quarter, 2,642 new users registered, followed by another 2,575 in the third quarter. However, a PCI team member noted that maintenance of the application was discontinued in 2015 due to a lack of funding. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 24 Table 5: Diagnostic Workshop Targets and Achievements Indicator 2013–2014 2014–2015 Target Achieved Target Achieved Diagnostic workshops conducted 20 25 25 25 Percentage of participants from the private sector 20% 26% 22% 29% Percentage of government officials participating in the diagnostic workshops 80% 69% 78% 68% Percentage of female participants 15% 25% 15% 21% Regional workshops conducted 2 2 2 3 Website hits following site improvement n/a n/a 31,574 106,522 Findings/Stakeholder Views The PCI team re-emphasized PCI’s contribution to creating awareness of the value of public-private dialogue (PPD) in economic governance reforms. Provincial authorities reported that PCI contributed to increased awareness of the value of PPD as a critical tool for guide the economic governance reform process. Provincial authorities said PCI has helped them conceptualize and develop events to gather input from the private sector. According to a VCCI representative, “PCI created a new way of thinking: empowering private enterprises to assess governance. ... This is considered one of the greatest achievements of PCI. Private enterprises see PCI as a channel to get their voice heard by local authorities. PCI is the collective voice of local enterprises, which can influence provincial authorities.” The PCI program, including the survey, report launch and diagnostic workshops, is a great opportunity to conduct PPD. In particular, diagnostic workshops provide a venue in which provincial stakeholders and private enterprises can analyze, digest and fully understand provincial PCI results. It also allows participants to actively engage and discuss specific topic areas highlighted by sub-indexes and exchange ideas on ways to improve. A provincial official and PBA representative in Thai Nguyen both indicated that, “During the annual provincial diagnostic workshop, PCI composite and sub-indexes are collectively analyzed by both provincial officials and the provincial business association for the development and enforcement of a provincial action plan.” PCI has facilitated the efforts by provinces to organize and host their own PPD events to gather input from the private sector. In Dong Thap, a provincial official indicated that as a result of PCI, provincial officials make significant efforts to “not only update PCI results, we also make more frequent efforts to reach private businesses. Leaders of the department and technical divisions have all participated in events organized by the business community (entrepreneur rendezvous; business café). Government officers now have the attitude of service providers rather than administrative staff. They always go to greater lengths to communicate and understand the needs of the business community.” In Ca Mau, Thai Nguyen and Hanoi, provincial officials indicated that effective implementation of PCI action plans requires communication and receiving feedback directly from private enterprises regarding needs to better prioritize activities and reforms. Additionally, as Figure 5 indicates, the vast majority of private firms in Dong Thap, Thai Nguyen and Da Nang agreed or strongly agreed that their local governments make efforts to maintain PPD. “Specific business associations have been in place for a long time, but they could not facilitate effective participation across associations. From 2011 onward, since the PPC put provincial competitiveness improvement as a straightforward objective, the PPC has supported a unified provincial business association, which represents all business in the province. Our business association was formed in 2013 as a result of this process.” – PBA Official, Thai Nguyen Figure 5: Percentage of Private Firms Agreeing That Local Governments Try to Maintain PPD Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 25 The evaluation team has collected clear evidence that PCI has contributed to the establishment and development of business associations. In Thai Nguyen, provincial officials reported that direct engagement with the private sector was critical to understanding local issues and devising a reform plan that supported private sector growth. However, provincial stakeholders indicated difficulty in engaging directly with numerous local and sector-specific business associations. As a result, the PPC supported the establishment of a PBA to unite and link the local (district-level and sector-specific) associations within the province and provide a more accessible channel for businesses to engage with provincial officials. PCI has been identified as one of the drivers of existing PBA expansion to the district level. In most of the visited provinces, PBA officials and provincial authorities agreed that PBAs should have a presence at the district level to reach more businesses. As a result, PBA officials pushed for expansion. As a direct result of annual PCI scores and rankings, various additional channels of communication have strengthened the communication and understanding between government bodies, enterprises and PBAs. In all visited provinces, either annual or biannual dialogue events have been hosted. Additionally, key departments, including DPI and the provincial tax office, host thematic meetings to address specific concerns of the business community about each line department. In provinces including Thai Nguyen, Dong Thap and Tuyen Quang, provincial leaders sponsor monthly business cafés to create open and sustained dialogue. In Thai Nguyen, PBA officials reported, “We can talk and meet the PPC and department leaders with much more convenient and open access. They have been ready to listen to us and follow-up activities are improved significantly.” In all of the provinces visited, the evaluation team found a diverse array of business associations, such as the Small and Medium Enterprise Association, Young Enterprise Association, Women in Business, etc. However, despite some successes, the capacity and effectiveness varied significantly for PBAs and district￾level associations to engage and collaborate on local governance issues. In Ca Mau, local business associations are unable to collaborate and, despite the PPC’s recognition of the importance of a provincial business association, the PBA has been unable to effectively organize and engage with provincial authorities. This situation is reflected in Ninh Thuan, where PBA representatives indicated that the association includes around 100 enterprises; however, only about a third of these enterprises actively engage in the association. In all of the provinces visited during fieldwork, the evaluation team found evidence that provincial officials have difficulty hearing and addressing the concerns of SMEs. In most cases, provincial business association members were larger businesses that had the resources to actively participate in a PBA. In Ca Mau, a provincial business association member said, “On one hand, SMEs do not want to pay to be part of the association; on the other hand, the association does not have enough funds to organize any meaningful events.” Additionally, many SMEs did not necessarily recognize the value of PPD events and were reluctant to commit time and effort to participating, which has further marginalized the participation of SMEs in the provincial reform process. The quantity and quality of PPD activities directly sponsored by provincial officials varied greatly across provinces. In some, PPD activities have been implemented effectively and have generated fruitful engagement between provincial officials and the private sector. In Dong Thap, interviewed private enterprises relayed that PPC and department leaders randomly visit local enterprises to elicit feedback regarding the challenges that confront businesses. Enterprises indicated that this was an extremely effective approach and demonstrated the provincial officials’ commitment to creating an economic enabling environment. But Cau Mau provincial officials noted that they are unable to hold effective PPD events because of low interest from enterprises. Officials indicated that they hold few events and, at a recent event that the PPC tried to organize, they invited around 380 enterprises and only four agreed to participate. As a result, provincial officials were disappointed and were unable to hold a fruitful discussion during the event. An increase in PPD activities at the provincial level does not necessarily mean increased accountability for provincial officials. In both Hanoi and Da Nang, business association representatives and private enterprises indicated that they often propose policy and regulatory adjustments to provincial officials during “In Hanoi, there are about 140,000 enterprises in operation and almost 98 percent of them are SMEs. The city currently has 20 business associations with a combined membership of 3,000 enterprises who are mostly comprised of large and privatized (formerly state￾owned) enterprises. Basically, most of our membership excludes SMEs.” — Representative, Hanoi SME Association Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 26 PPD activities. However, representatives reported that in many cases they did not receive feedback or follow-up regarding their proposals. In Ca Mau, private enterprises indicated that a lack of feedback from provincial officials eroded the confidence enterprises had in the ability of dialogue events to have a meaningful impact. Conclusions  PCI has directly influenced stakeholders’ appreciation for PPD and has been used as common discussion framework for PBAs and provincial stakeholders, which has supported local economic governance reform efforts.  PCI has contributed significantly to promoting public-private dialogue through the establishment of new provincial business associations, as well as reinforcing existing ones. PBA leaders have all stated that the attitude and treatment toward PBAs has evolved significantly.  One constraining factor is that some provinces have weak PBAs that are unable to effectively engage with provincial officials. In provinces where PBAs are active, they are often dominated by large firms and have limited participation from SMEs.  Diagnostic workshops are an effective and useful step in supporting administrative reform through the delivery of technical assistance; however, not all provinces were able to access diagnostic workshops team due to limited budgetary resources. This may cause some challenges for those provinces that are unable to access technical assistance from the PCI team, especially for low-performing ones.  The quality of public-private dialogue activities can affect the level of confidence the business community has in the economic governance reform process and the overall level of engagement. 3. Capacity Building Sub-Question 1a: To what extent has the PCI program aligned with the priorities of the central and local governments’ efforts to promote an enabling environment for private sector businesses? Sub-Question 1b: What are some examples of economic governance reform practices (success stories) within the provinces that are related to PCI program? Sub-Question 1c: What are the drivers and constraints (subjective and objective) that contribute to the realization (or lack thereof) of the three objectives? This section summarizes the activities aimed at improving economic governance capacity and explores the contribution of PCI to increasing capacity in three separate areas. First, it presents the PCI program’s activities to improve economic governance understanding and capacity. It then explores the team’s increase in capacity over time to implement the PCI program’s activities. Next is a focus on the capacity of provincial stakeholders regarding the use of PCI data and the initiation of PCI-related activities. The final subsection covers the provincial stakeholders’ capacity in economic governance planning and reform. Activity Description Increasing understanding of local economic governance issues and improving local governance capacity to address those issues is a core tenet of PCI. The PCI team provided training to National Assembly members to boost understanding of economic governance issues, thereby enhancing the policymaking process. The PCI team also conducted trainings with business associations to encourage their participation in public-private dialogue. Additionally, the PCI team provided direct consulting expertise to some provinces to help them in their strategic and action planning for private sector development. The program’s training efforts primarily occurred in year 2. As indicated in Table 6, the program exceeded its targets for the number of training courses conducted for both business associations and National Assembly/PPC members in year 2. At trainings for business associations, 30 representatives of various associations participated. Following each training, the PCI team distributed participant surveys. The survey showed that 100 percent of participants indicated that the courses met or exceeded their expectations, while 93 percent reported that the training material was “good” or “very good.” However, the trainings included representatives from only 11 of the 63 provinces (17 percent), which indicates that the vast majority of Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 27 provincial businesses associations did not attend the training. Additionally, the team provided specialized expertise consulting to provinces including PhuTho, NinhThuan, QuangTri, Ca Mau, DongNai and many others. The purpose of the expertise consulting was to assist provincial authorities in their development of action plans meant to improve local business environments. Table 6: Number of Training Courses Conducted in Years 1 and 2 Indicator 2013–2014 2014–2015 Target Achieved Target Achieved Training courses for local business associations n/a n/a 2 2 Training course for National Assembly / Provincial People’s Council n/a n/a 2 5 PCI has also sought to encourage and support the drafting, development and implementation of strategic action plans by provincial authorities aimed at pushing local economic governance reform and improving local business environments. Table 7 highlights the program’s targets and achievements as they relate to the development of specific strategic and/or action plans in which PCI results played a contributing role. The figures in the table represent a count of provincial legal documents focused on improving provincial competitiveness. As Table 7 shows, PCI has achieved targets set forth each year. Table 7: Action Plans Developed or Adopted with Influence by PCI Indicator 2013–2014 2014–2015 Target Achieved Target Achieved Number of action plans developed / adopted using PCI findings / recommendations 5 23 10 25 Other Activities In addition to the diagnostic workshops and training events, the program completed complementary activities. These activities included upgrading the PCI website to a new interface; a study on provincial website transparency conducted by Vietnam Economica that delivered analysis of provincial governments’ websites and their ability to transmit relevant and accurate information to citizens; and an independent study titled “Integrate the improvement of local economic governance in the provincial policymaking and implementation of socio￾economic development plans.” The study aimed to provide informational support to provinces in the integration of economic governance into provincial socio-economic development planning. The study found that the private sector’s role in such planning has not been sufficient. As a result, provincial governments have not collected sufficient accurate data on businesses’ perception of governance quality. Findings/Stakeholder Views VCCI’s Capacity to Implement the PCI Program In interviews, stakeholders indicated that the VCCI/PCI team’s ability to implement the PCI survey and supporting activities has significantly improved through the years. One VCCI/PCI team member indicated that hosting all of these activities 10 years ago would have been difficult, if not impossible, for VCCI without support from international organizations. However, now VCCI has learned to conduct large-scale data collection and analysis, as well as organize and conduct supporting workshops and trainings without external support. The team member cited examples of how the team has adjusted and improved the survey, mentioning that one critical change was recruiting students from specific regions to make follow-up calls to firms in those regions. Previously, respondents had difficulty understanding and were often intimidated by receiving follow-up calls from students with strong Hanoi accents. The PCI team recognized this as a challenge and restructured the recruiting process to include students from various regions of the country. The team then assigned a region￾specific calling plan to ensure that students from the various regions made follow-up calls to the enterprises from their region. As a result, the PCI team reported much greater success in conducting follow-up calls with enterprises. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 28 The PCI Team: Capacity Development Over Time The story of VCCI, specifically the PCI team, is one of development and growth. When the PCI program began in 2005, VCCI had little experience conducting large-scale data collection activities or subsequent analysis. Researchers, the business community, government stakeholders and the general public did not view VCCI as a knowledge center. The PCI program represented the first time VCCI engaged in a large-scale data collection and analysis activity. During the first phase of the program, the PCI team at VCCI, with the assistance of an international partner, focused on implementing a core set of activities that centered on the PCI surveys. This focus, coupled with assistance from the international organization, allowed the PCI team to grow and develop its capacity in data collection, analysis and dissemination. Currently, VCCI is able to conduct the PCI surveys and dissemination events independently and government officials from the central and provincial levels recognize VCCI as a credible organization and a center of knowledge about the private sector in Vietnam. The central government of Vietnam even recognized the PCI program as “one of the 25 best projects between 2011 and 2015.” The central government also named VCCI the lead monitoring agency of Resolution 19. Both achievements speak to the ability of VCCI to consistently maintain an effective level of program implementation. Stakeholders’ Understanding of PCI Results and Economic Governance The evaluation team encountered varying levels between provinces regarding the ability of provincial stakeholders to digest, analyze and understand the PCI scores (both composite and sub-index) and the factors that drive them. Among the high performers, provincial officials in Thai Nguyen and Dong Thap demonstrated significant capacity to undertake in-depth analysis. In Thai Nguyen, provincial officials utilize a PCI analysis system that involves significant collaboration with the provincial business association. In this case, the provincial business association also exhibited significant motivation to work alongside provincial officials and capacity to contribute to a fruitful analysis. Once the PCI report is launched each year, members of the Thai Nguyen PCI task force work closely with PBA representatives to create an official analysis report of the provincial scores as they relate to each sub-index. The report is developed in a two￾week timeframe and submitted to the PPC for approval. The analysis report provides an in-depth examination of sub-index scores and allows provincial officials to translate sub-index scores into an understanding of local economic issues and begin working toward developing solutions. The evaluation team identified significantly less understanding of PCI results in Ca Mau, Ninh Thuan and Hanoi. This inability has subsequently impacted provincial officials’ ability to identify and prioritize economic governance issues within the province and develop appropriate policy responses. This was especially apparent in Ca Mau, where provincial officials did not fully grasp economic governance issues highlighted in sub-index scores and did not produce a regular report analyzing the PCI results or related issues. Additionally, the provincial business association proved to be fractured and lacked sufficient capacity to regularly engage with local officials regarding issues of economic governance. In Ninh Thuan, the Economic Development Office is the focal point for PCI. However, stakeholders within the office reported that they do not take additional steps or invest resources in creating an analysis plan. The evaluation team found no evidence of publicly accessible datasets or any protocol requiring the PCI team to provide data to provincial stakeholders. Beyond Da Nang and Hanoi, no provincial stakeholder indicated that they had received or had access to PCI raw data. Representatives from the PCI team confirmed that they had given raw data to provincial stakeholders in only those two provinces because of the concern about other provinces’ ability to analyze PCI raw data adequately. Representatives from Da Nang’s academic institutions reported difficulty in accessing the raw data generated by the survey. They said their institutions have been contacted by neighboring provinces to help create diagnostic analysis and action plans; however, they were unable to complete these tasks effectively because of the inability to conduct analysis using the PCI data. In Tuyen Quang, provincial officials indicated a general lack of capacity in the past in independently analyzing PCI results beyond what VCCI presented to them during diagnostic workshops. However, officials indicated that they have taken proactive steps to enhance the capacity of provincial officials. During the most recent diagnostic workshops, officials actively Provincial officials in Da Nang showed a solid understanding of PCI results and the factors driving sub-index scores. In Da Nang, analysis of PCI results is led by the Institute of Socio￾Economic Development (ISED), a provincial government operated think tank that leads PCI’s analysis and details findings in a formal report that is delivered to provincial line departments and the PPC. ISED conducts a representative survey of firms to verify PCI results. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 29 worked with VCCI developing their analysis and PCI action plan at the provincial level. Subsequent to the workshop and in the course of implementing the provincial action plan, they found a major bottleneck in the delivery of direct services to the business community. As a result, provincial officials recently attempted to improve the situation by working to develop administrative reforms to enhance service delivery and collaborating with the provincial business association to develop and implement a district competitiveness index (DCI). The Tuyen Quang DCI was first implemented in 2015 and will continue annually. The DCI survey, modeled after the PCI, provides additional data and evidence on economic governance issues at the district level. Economic Governance Capacity Findings from the evaluation team’s fieldwork indicate that PCI contributes to provincial officials’ overall economic governance capacity. All provincial stakeholders reported that the PCI survey is useful in helping provincial authorities better understand specific issues of economic governance in their provinces. This finding is bolstered by the results of the evaluation’s mini-poll, with 99 percent of provincial stakeholders indicating that the “PCI program helps to increase local capacity on economic governance,” as Figure 6 illustrates. This is further confirmed by the trend in PCI median composite scores, which have generally increased to demonstrate the improvement in economic governance capacity throughout the country. PCI has helped create greater dialogue about economic governance, which has enabled provincial leaders see the value in thinking about and addressing economic governance issues. PCI’s ranking methodology has also helped create a competitive atmosphere where provinces constantly look to raise their scores through improved economic governance. A representative of Hanoi’s People’s Committee commented that “... improving the investment environment, providing support for enterprises to do business is one of the top priorities of action for Hanoi city. Improving PCI is considered one of the key regular tasks for the Hanoi government this year and in the years to come.” Both interviews and document review conducted during fieldwork indicate that each of the provinces visited has developed a strategic/action plan or issued legal documentation aimed at improving provincial competitiveness. The evaluation team found evidence that significant variation exists between provinces in regard to the ability to develop plans/documents as well as implement them. In Dong Thap, Thai Nguyen and Tuyen Quang, the evaluation team found that action plans developed by provincial officials were quite effective when implemented. In each province, stakeholders indicated that PPC leadership prioritized provincial competitiveness and actively supported the planning process. Additionally, provincial action plans were based on comprehensive PCI analysis reports developed by provincial officials and, in the case of Thai Nguyen, with Figure 6: Responses to Whether the PCI Program Helps Increase Local Capacity on Economic Governance 17 2.5. Maintain and upgrade the existing PCI website and mobile interface After the PCI 2014 launch on April 16, a new interface has been introduced applying the Responsive technology which allows to adapt website interface to every type of screen. Besides, 3 main changes were made to the website. First of all, interface was improved in order to match with the PCI event branding and mobile interface was designed and activated to meet the needs of high-tech users. Second, the newly-updated website provides more information to users, from the latest survey findings, PCI reports, to updates on good practices of improved provincial business environment, provincial legal documents and international reports inspired by the PCI. Many functions were designed to be more user-friendly such as Download Report, Hot news and Decision. Finally, regarding the function of “comparing provinces”, the PCI website adds a new option that allows provinces to compare their performance over time. So far, there are about 3,103 downloads of PCI 2014 full report and 2,467 downloads of provincial profile report 2014. Business association representatives attend a VCCI training course on June 26, 2015, in Hue City. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 30 substantial input from and collaboration with provincial business associations. Provincial officials also indicated that, during the development of action plans, the PCI task force members sought input from line department staff to ensure that plans accurately reflected the capabilities and responsibilities of the departments. The resulting action plans were extremely thorough and provided specific, detailed and actionable items for the line departments to complete. Provincial officials also pointed to the fact that, during the implementation of action plans, PPC leadership took an active role in monitoring plan activities, often requiring formal documentation of progress and results achieved. Findings during fieldwork indicate that action plans developed by provincial authorities in Ca Mau, Ninh Thuan and Hanoi have faced challenges in effectiveness and implementation. In Ca Mau, the PCI task force is responsible for the development of the plan. However, officials in Ca Mau do not have a robust analysis of PCI findings on which to base their action planning. Additionally, during development of the plan, the task force does not take an active role in coordinating and collaborating with line departments. As a result, the action plan does not contain sufficient detail regarding specific timelines, responsibilities and activities for line departments to carry out. This is confounded by the fact that the PPC did not take an active enough role in monitoring the actions of line departments with regard to action plan implementation. In Hanoi, provincial competitiveness planning is often not prioritized. This may be a result of the fact that Hanoi is a large urban center and the national capital, where line departments are often heavily burdened with other responsibilities and provincial competitiveness planning is not closely monitored. Provinces’ Ability to Implement Reform Action Plans Per discussion with local stakeholders, key aspects of an effective action plan include a high level of specificity regarding the assignment of tasks and objectives, clearly stated and detailed responsibilities and frequent monitoring of the provincial leaders. Table 8 provides the evaluation team’s analysis of provincial action plans regarding these key factors. Da Nang does not utilize a specific PCI action plan, but requires line departments to develop their own annual action plans that incorporate and address issues highlighted in the PCI and by the local business community. Table 8: Evaluation of Provinces’ Action Plans Province Specific Task Responsibility Monitoring Practical Effectiveness Da Nang n/a n/a n/a n/a Dong Thap *** *** *** *** Thai Nguyen *** *** *** *** Ha Noi ** ** ** ** Tuyen Quang *** *** *** ** Ninh Thuan * * * * Ca Mau * * * * *** high level; ** moderate level; * weak level In Da Nang, provincial officials indicated that they do not develop or utilize a single action plan based on PCI. Instead, ISED, the local governance think tank funded directly by Da Nang’s PPC that can be considered on par with the provincial line departments, conducts its own analysis of PCI raw data with the findings synthesized into a comprehensive report. This report is submitted to the PPC and line departments. Subsequent to formal approval of the report by PPC, ISED distributes findings directly to line departments to ensure that findings and recommendations from its analysis are incorporated into each department’s annual strategic planning. This is bolstered by an active PPC, which closely monitors line department activities to ensure that provincial competitiveness is being enhanced through the implementation of departmental action plans. Officials in Da Nang noted that they do not need to create an annual PCI action plan nor a PCI task force because their PCI analysis and planning strategy represents a long-term approach to provincial competiveness improvement and an attempt to integrate and “mainstream” PCI data into regular department planning. Provincial officials also pointed to the fact that they connect with the business community through PPD events, which gives them real-time feedback on economic governance issues. “We actually do not understand. We try to reform, but the local business community is still not satisfied. However, to be honest, our action plan has changed very quickly in the first year of implementation because the division of tasks did not fit with the PPC mandate and actual performance of the line departments in dealing with enterprises.” — Provincial Official, Ca Mau Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 31 Conclusions  The ability of the VCCI/PCI team to implement the PCI survey and supporting activities has helped generate credibility for the PCI program in stakeholders’ view, which has supported the development and implementation of provincial economic reform agendas.  PCI has increased appreciation for economic governance issues and the value of creating business￾enabling environments among provincial stakeholders.  Key factors driving the success of provinces’ ability to enact economic governance reform include committed political leadership with respect to improving provincial competitiveness, open channels for dialogue between provincial officials and the business community and capable and engaged business associations.  Variation in provinces’ ability to analyze and understand PCI results and then act accordingly to address local economic governance issues can constrain the effectiveness and speed of reform.  A lack of access to the PCI raw data hinders opportunities for additional capacity building. Researchers in Da Nang and Hanoi noted that they were unable to carry out in-depth consulting regarding PCI improvement requested by neighboring provinces because of this hurdle.  In the absence of formal guidance, the vast majority of provincial stakeholders still have not accessed raw data related to their own province. 4. Sustainability Sub-Question 1e: What efforts have been made to ensure PCI program sustainability, both technically and financially? This section presents findings and conclusions regarding steps the PCI team has made to ensure the financial and technical sustainability of the PCI program. It also presents findings and conclusions related to the sustainability of activities that have resulted from the PCI program, such as PPD activities and action planning at the provincial level. Achievements During the last three years, the PCI team has considered various options for how best to sustain the PCI program moving forward. The team has been able to develop general guidelines and standard operating procedures to ensure consistent program implementation even in the event of staff turnover. As indicated by Table 9, the PCI program has achieved the target of developing a financial sustainability plan in year 2. Table 9: Targets and Achievements Regarding Financial Sustainability Plans Indicator 2013–2014 2014–2015 Target Achieved Target Achieved Financial sustainability plans for PCI developed n/a n/a 1 1 Currently, the funds for survey implementation come from USAID. Diagnostic workshops are requested by provinces that provide direct funding, with the exception of the travel costs, for the PCI team. As part of developing a financial sustainability plan, the PCI team has outlined six possible funding models moving forward. These are:  Licensing of diagnostic and consulting workshops to select firms. The PCI team would authorize a limited number of pre-selected firms to conduct direct consulting on PCI score improvement to provinces.  Providing PCI data to users for a fee. The PCI team would provide restricted access to PCI raw data to paying users. Data would be stored on an online database and accessible to subscribers via the internet. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 32  Selling space on the PCI survey. Individuals or organizations could purchase space within certain modules of the PCI survey questionnaire to insert their own questions. This would allow interested parties to “piggyback” on the PCI survey’s annual data-collection activities.  Selling data to investment firms. The PCI team could sell firm data to investment firms for a substantial fee. A number of investment firms are already interested in gaining access to direct firm data for trading and investing purposes.  International donor funding. The PCI team may consider maintaining its funding base with USAID or expand it to other donors, such as the World Bank or the International Finance Corporation (IFC).  Provincial funding: Provinces would start to provide direct funding for the implementation of the survey, as well as continue funding diagnostic workshops. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 33 Findings/Stakeholder Views Technical Sustainability The PCI team has the technical capacity to independently implement the program surveys and outreach activities. Over the last two years, the team has demonstrated the ability to conduct data collection and analysis, as well as hold outreach activities. One PCI team member remarked, “Ten years ago, it was necessary to have the support from an international organization; however, over the years, the PCI team has significantly built its capacity and can now implement the survey without any support.” A stakeholder at the National Economics University confirmed this assertion, noting that data collection and analysis practices are performed with rigor and the PCI team is more than capable of continuing to implement the survey. He went on to say that he believes that VCCI is also well positioned to conduct the diagnostic workshops, given the organization’s unique position at the nexus of government and the private sector — which allows VCCI to interface and engage easily with stakeholders from both sides. A small number of individuals directly manage current PCI activities. The PCI team in Hanoi, which consists of a small group of PCI staff, directly manages implementation of the survey. As a result, the majority of the knowledge regarding data collection and analysis centers on only a few individuals. Going forward, despite an operating procedure being in place, staff turnover may affect program implementation. Financial Sustainability The PCI team has considered possible funding models moving forward, but has not systematically assessed or documented the viability of each model to come to a conclusion regarding the feasibility of the various models. In discussions with the PCI team, members indicated that they have considered each possible funding model and have drawn some initial conclusions about each (listed in the box) that were outlined in a preliminary financial sustainability plan submitted to USAID. While the PCI team has indicated the development of a financial sustainability plan, the evaluation team has not found evidence that this plan represents a formal exploration of the various funding possibilities or a concrete strategy moving forward. In fact, a representative of the PCI team noted that finalization of the plan was held back due to concerns and unresolved issues. In a subsequent interview, a PCI team member noted that the continuation of USAID funding is the preferred funding mechanism, as it will allow VCCI to maintain its independence during the survey process. Possible Funding Models  Selling space on the survey: The team noted that this is a common practice for surveys. However, selling space would affect the length of the questionnaire. Additionally, this method may not serve as a consistent-enough revenue stream. One team member related having tried this once with the International Labor Organization, and said it did not generate enough revenue to be effective.  Providing paid access to the data: Team members indicated that demand for the data is insufficient.  Selling data to investment firms: One team member noted that investment firms are interested in firm-specific information to help them make investment decisions. While this option may provide significantly more revenue than some others, investment firms may need specific information, including the names of enterprises that participate in the survey. However, if the PCI team divulges names, it cannot provide anonymity to survey respondents, which may affect how respondents answer the survey in the future.  Provincial funding: The PCI team noted significant interest by provinces to pay for survey implementation. One team member noted that he received multiple requests and that funds from provinces would be sufficient to sustain survey implementation. However, unless all provinces agree to fund an equal amount each year, accepting various levels of funding may lead to the perception that the survey favors certain provinces over others, which would negatively impact the credibility of the PCI program.  Licensing of diagnostic and consulting workshops to select firms: One team member indicated that this is a viable option because a high demand already exists for such services. He pointed to the fact that a number of provinces have contracted with international consulting firms to provide provincial officials with guidance on how to improve their scores. To make this option realistic, the PCI team would have to develop criteria for selecting consulting firms that were interested in obtaining a license, as well as general protocols and regulations guiding the provision of services. To this point, no formal concept note or plan for this has been developed.  International donor funding: The PCI team indicated that this is the preferred method of funding, given the present situation. The partnership with USAID has helped establish PCI as a reputable brand and a continued partnership will ensure credibility in the future. This point was stressed even in regard to other donors. The team noted that other donors would be willing to fund PCI, but the partnership with USAID would be most beneficial to the program. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 34 Sustainability of Activities Facilitated by PCI All provincial stakeholders indicated that PCI is a valuable tool in helping sustain province-led PPD activities and economic governance reform planning. Provincial officials in Ca Mau and in Thai Nguyen reported that the annual release of PCI results helps reignite the conversation about provincial competitiveness each year within their provinces. The renewed discussions contribute to the motivation and interest of provincial leadership to stay engaged with the private sector by continually hosting PPD activities and reform planning. Provinces have varying levels of ability to maintain PCI-related activities. While PCI is a significant tool to keep the conversation about provincial competitiveness and economic governance reform alive among provincial officials, the level of ability to continue province-led PPD activities and economic governance reform planning varies. Provincial officials in Da Nang indicated that the local government is committed to a long-term strategy of provincial competitiveness improvement that includes the continuation of province-led PPD activities and strategic reform planning. In Da Nang, the ISED conducts PCI analysis and submits its findings to the PPC and relevant line departments. Each line department is then responsible for creating its own annual action plan. The PPC then monitors the development and subsequent implementation of each department’s action plan with consistent input from district and provincial business associations. Provincial leaders have acknowledged that this process has become routine among government stakeholders. They also acknowledge the importance of continuing to host PPD activities and relevant action planning as key aspects of the provincial system to improve competitiveness. But in Ca Mau, PCI-related action planning and province-led PPD activities are led by DPI’s business promotion center. During interviews, provincial officials indicated that it was difficult for the center to develop a thorough and detailed action plan and then ensure coordination between departments during implementation. Additionally, the province has a PBA that is often unengaged with provincial officials. While desire exists to continue to conduct activities that contribute to improving provincial competitiveness, no administrative planning and infrastructure exists to support consistent development and implementation of activities. Conclusions  The PCI program is technically sustainable given the current capacity of the PCI team and level of commitment from VCCI leadership to supporting the program.  Maintaining independence is critical to the survey’s credibility among provincial stakeholders and its future success.  Expectations regarding the definition of sustainability and the length of USAID/Vietnam funding for the PCI program vary between VCCI and USAID.  The ability of provinces to maintain and continue to implement PCI-related activities, such as PPD events or department action planning, varies between provinces and depends heavily on provincial officials’ commitment to provincial competitiveness improvement and the ability of the business community to engage with provincial officials. 5. Crosscutting Issues Sub-Question 1e: What efforts have been made to ensure PCI program sustainability, both technically and financially? Program Management This section explores the management structure and communication strategy employed by the PCI team, VCCI and USAID. The section also provides an analysis of how the PCI team is able to capture knowledge and employ new strategies on an annual basis. Achievements USAID and VCCI’s partnership has resulted in the successful implementation of the PCI program over the last two years. The PCI team located at VCCI’s central offices in Hanoi has been the lead in conducting the surveys Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 35 with no significant support from the VCCI regional branches. VCCI regional branches do participate in some of the diagnostic workshops. USAID has provided support during this period by helping VCCI address challenges that arose during the implementation process. Findings/Stakeholder Views Government, private sector and third-party stakeholders have all indicated that the PCI program has been effective in establishing a brand that is recognized and respected. This stems in large part from the consistent and effective implementation of both the surveys and the supporting activities over the last 10 years in collaboration with USAID, as well as the significant media coverage of the survey results. The evaluation team finds that the PCI team is able to maintain knowledge regarding operations (survey implementation, diagnostic workshops, media plan development, communication with USAID, etc.) from year to year. Knowledge is maintained through a small management team, as well as open communication between USAID and stakeholders. This has allowed the PCI team to effectively tap into institutional knowledge of program implementation while simultaneously continuing to improve the way activities are conducted. PCI has adopted a simple management structure that has been complemented by targeted support from USAID. The PCI team has adopted a horizontal management structure; the chief of party works with the PCI team on a daily basis and the team’s lead methodologist provides support when necessary. PCI team members indicated that USAID takes a supportive role and does not take an overbearing management role. This simplified structure has allowed the PCI team to be flexible and adapt while still focusing on implementing core program activities. The partnership between VCCI and USAID has benefitted from leaders who are committed to the PCI program. PCI team members indicated that they have had full support from the chairman of VCCI, which has allowed them to operate freely and seek support from senior leadership at VCCI when necessary. They noted that the VCCI chairman is committed to PCI and has been a great spokesperson. Additionally, he has been able to work with members of USAID’s leadership who have also supported the PCI program. This has resulted in an effective working partnership between the two organizations. The PCI team includes only a few permanent members who are responsible for overseeing the full implementation of the survey without significant involvement of VCCI’s regional offices. Interviews with the PCI team and VCCI regional offices revealed that the current structure has allowed for the concentration of knowledge within the central PCI team with no significant involvement by the VCCI regional offices in the implementation of the survey (although VCCI branch staff received training on PCI from the central team). As a result, the regional offices have not experienced the same growth in capacity as the central team. PCI team members did indicate that they did not want to involve the VCCI regional offices in the survey; it would compromise their ability to operate on a daily basis due to the pressure they would receive from local officials regarding the survey. Conclusions  Program management of PCI has been largely effective. The PCI team has been able to transfer knowledge from year to year and learn from previous experiences. Additionally, communication between the PCI team and USAID, as well as between local stakeholders, has been effective.  VCCI has employed from a simplified organizational structure and benefitted from targeted support from USAID.  Given the current structure, limited opportunities exist for staff within VCCI regional offices to engage with the PCI survey and learn from the operations of the central team. “Our most publicly recognized contributions have been to trade-related governance and provincial-level competitiveness. The Provincial Competitiveness Index has demonstrated the value of transparency and incentives to improve.” – USAID Senior Leadership Representative Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 36 6. Monitoring Evaluation and Learning This section provides a summary as well as analysis of the monitoring system used by the PCI program. The section provides an analysis of the effectiveness of the current system in tracking program progress toward realizing the objectives. Monitoring System The PCI program employs a monitoring system that tracks both USAID standard indicators and custom indicators. As part of the system, the PCI team tracks a total of 22 indicators and sub-indicators that allow measurement of the program’s progress in achieving its objectives. For objective 1, the team tracks seven unique indicators and sub-indicators that follow the release of the PCI report and the level of participation in the report launch event. For objective 2, eight indicators and sub-indicators measure the number of diagnostic workshops and trainings, as well as the level of participation for both types of events. Objective 3’s three indicators track PCI score improvements, action plans developed and provincial websites improved by provinces. For crosscutting issues, three indicators track VCCI regional branch participation in training and financial sustainability reports developed. Figure 7 represents an example of the indicator tracking that the PCI team conducts, listing targets and achievements by quarter and year. Data for the indicators is collected directly from the PCI survey, workshops and training documents. The PCI team provides information regarding the progress of the indicators in quarterly reports that are submitted to USAID. In addition to indicator data, quarterly reports provide detailed descriptions of activities undertaken during the quarter. Findings/Stakeholder Views Currently, the indicators used to track progress toward the program’s objectives can record only program outputs. A review of the quarterly reporting documents and interviews with the PCI team reveal that the present design does not include any indicators to track the quality of activities. Conclusions While it has successfully complied with the project M&E plan, the PCI program has been unable to systematically assess the quality of outreach activities sponsored by the PCI program (diagnostic workshops or training), which impacts the program’s ability to understand progress made toward achieving objectives 2 and 3. Additionally, under its current design, the program does not collect data on PPD events/activities that were initiated and sponsored by provinces (business cafés, entrepreneur rendezvous, etc.). Understanding the quantity and effectiveness of such PPD events would be beneficial for provincial stakeholders in their efforts to maintain an effective and continuous PPD. Example of quarterly indicator tracking. Targets and achievements for objective 3 Figure 7: PCI Economic Governance Improvement Indicators Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 37 IV. PCI PROGRAM CONTRIBUTIONS TOTHE BUSINESS ENVIRONMENT Administration and Regulatory Reform Sub-Question 2a: How has the PCI program influenced provincial business environments, particularly in relation to administration reform? Sub-Question 2b: What role, if any, do contextual factors (e.g., proximity to commercial centers, access to natural resources, geographic location, etc.) play in influencing local economic governance reform? This section explores the progress that provinces have made toward administrative reform. The section also explores the PCI program’s contribution to the process of reform. Findings/Stakeholder Views In all provinces visited by the evaluation team, the PPC and line departments have carried out administrative reforms as required by Resolution 30c, enacted in 2011. The reforms have been planned and monitored annually by provincial PPCs and several in each province address the improvement of public services related to business community. PCI has supported implementation of the central government’s agenda regarding administration reform at the provincial level. In all of the provinces visited during fieldwork, the evaluation team observed that all PPCs had developed action plans regarding provincial administration reform (as required by Resolution 30c). However, the plans tend to be quite general and do not necessarily point out specific tasks for departments. PCI has strengthened this reform and supported departments to create action plans that are specific and focused on key issues by providing evidence regarding the sub-indexes from the local business community. In 2011, as a result of PCI, the PPC of Hanoi developed a five-year PCI action plan titled Plan 141 (The Hanoi PPC, 2011). The plan outlined specific tasks, expected outputs and timelines for each line department that deals with enterprises. The plan also required each line department to develop its own action plan that was in line with the larger PPC plan and encouraged coordination between line departments. As a result, some line departments developed action plans with specific activities meant to improve economic governance. The Hanoi Department of Investment and Industry implemented a policy of “never say no to a request from an enterprise” to change the attitude and improve the effectiveness of public services being delivered to thousands of Hanoi enterprises. The 2015 Administrative Reform Plan enacted by the Hanoi PPC directs all line departments and district people’s committees (DPCs) to implement the general administrative reform objectives and tasks as follows:  To effectuate a one-stop shop for citizens and a one￾stop shop for interagency coordination to shorten procedures and reduce time cost for individuals, organizations and enterprises. Specific tasks for the Department of Planning and Investment (DPI) are also included:  (DPI) is responsible for reviewing and developing a working protocol to shorten the time for enterprise registration and establishment/adjustment by 30 percent, which means reducing registration procedures from 14 days, as outlined in the 2014 enterprise law, to 10 days. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 38 PCI has provided provincial leadership with a performance evaluation tool. Provincial officials, domestic researchers and international experts all report that PCI has provided a tremendous boost to the economic governance reform process at the provincial level by providing the PPC with a tool for monitoring progress. In Thai Nguyen, the PCI task force collaborated with line departments and the PBA to develop a detailed action plan following the release of the PCI results each year. Following the development of the plan, the PPC uses the PCI results to ensure that the plan addresses key issues and that all departments are making progress with implementation. The PPC chairman issues official performance review letters every two or three months to line departments, which stakeholders say has significantly helped push the reform process in Thai Nguyen. “In the last three years in Thai Nguyen, the line departments have become very responsive and responsible to their action plan. In response to our request, the PPC chairman has been continuously informed about their performance. DPI staff and members of our association also can reach out the chairman and PPC members to provide input regarding the performance of departments. Though a certain number of outstanding issues still remain, we see that the attitude and effectiveness of the public agencies toward enterprise has changed completely for the better.” — Thai Nguyen PBA The prioritization of resources to support economic governance reforms varies across provinces. To implement the administrative reform, provincial line departments have to utilize their own budgets and carry out expenditures related to reform activities. The launch of PCI and subsequent development of action plans has boosted economic governance reform and several line departments have made concrete allocations of funding and human resources to support initiatives. In Tuyen Quang, PPC and PBA invests resources to carry department diagnostics and a district competitiveness index (DCI). Da Nang has invested in the provincial think tank (ISED) to conduct its own survey on enterprises, parallel to PCI surveys, to further inform their planning and verify the validity of PCI results for the local stakeholders. However, in Ninh Thuan, provincial officials indicated that they conduct planning as well as assign tasks for individual departments, but are unable to fund most of the activities. PCI provides a platform for stakeholder discussion around administration reform issues. In all visited provinces, the PPC and line department used the PCI annual score and ranking for internal review purposes. In Da Nang, Hanoi, Thai Nguyen, Tuyen Quang and Dong Thap, the People’s Council, mass media and business associations employ sub-indexes as a framework and foundation for discussion of reform issues. Increased public awareness from mass media coverage has positively affected reform. The mass media often reports on annual scores and rankings at both the national and provincial levels, which has created legitimate pressure on provincial governments to reform. In Thai Nguyen and Ca Mau, provincial officials noted the role of the media in putting pressure on local officials to maintain the reform process. Additionally, in all provinces visited, provincial newspapers and television broadcasts have aided provincial officials in providing the public with information on different aspects of economic governance reforms. In each province, the tax office has worked closely with television stations and newspapers to maintain the communication and dialogue with enterprises through advertisements and public announcements of initiatives and reforms. To respond to the province’s effort regarding PCI improvement, the tax office in Ca Mau introduced the initiative of “listening to the taxpayer.” Such efforts resulted in a top ranking for the Ca Mau Tax Office for key reform pillars under the national tax modernization plan hosted by the General Department of Taxation. (Photo by Nhat Nguyen Duc) Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 39 PCI supports coordination between provincial departments for sectoral reform. In all visited provinces, line department representatives interviewed confirmed that sectoral reform was possible only with the collective action from related agencies. For example, the process of cutting down the number of working days required for enterprise establishment would not be possible without the joint cooperation of DPI, the Tax Office and the Police Department, as they are the three agencies in charge of providing enterprises with business permits, tax registration and an enterprise seal. In each case, provincial officials pointed to the sub-index of time cost for enterprise establishment as the motivation and foundation for integrated reform. In Ca Mau, tax office representatives indicated that the “tax office has signed an agreement with relevant departments including DPI, [the Department of Natural Resources] DoNRE [and] police to jointly cut down the time for relevant administrative procedures. We have even employed a higher standard than required in the new 2014 enterprise law.” “Our tax office has been always ranked first among line departments and our results are better than the central government’s requirements in terms of implementing our reform procedures and time-/cost-cutting before. PCI has supported us even in ranking the administrative reform of line departments.” — Tax Office, Da Nang The awareness and acceptance level of PCI results has increased over time. As highlighted by Madam Pham Chi Lan, former member of the prime minister’s advisory group, in the early years of PCI, the index and the program received strong opposition from many cities and provinces, including Hanoi and Ho Chi Minh City. In the past, provincial officials in Hanoi requested their own capital index and were not interested in provincial competition. Starting in 2009, the PPC of Hanoi began requesting yearly reviews of PCI performance. From 2011 onward, the PPC of Hanoi began to take PCI much more seriously and enacted a five-year action plan, which called for the participation of all line departments. And according to Hanoi DPI, they upgraded their commitment in December 2015 with a party resolution putting business environment improvement among the four key pillars of the next five-year plan (2016–2020). “Before 2011, PCI was not among the top agenda or policy concerns of our local government and line departments. Only when our PPC chairman was challenged by potential investors, our PPC has called for a review of the system. The leader of DPI in those years has received serious criticism for ignoring the economic governance improvement competition and failure to report to the PPC. As a result, a new leader of DPI was introduced, PCI task force and provincial business association were in place and we have been implementing our joint action plan in all aspects of economic governance reform since then.” — Thai Nguyen PBA Central government regulations may hinder provinces’ ability to develop and implement reforms. Provincial stakeholders in Dong Thap and Ca Mau also reported that sometimes central government regulations constrain their planning. Stakeholders often pointed to central government regulations related to infrastructure or land access that often contradict or severely limit provincial authorities’ ability to design and implement policies tailored to the local environment. Conclusions  PCI has contributed to provincial economic governance reform in all provinces; however, the degree to which reform has been enacted varies between provinces.  PCI has contributed to creating a platform for stakeholders to discuss and provide input on possible ways to enact economic governance reform.  Central government regulations and planning may hinder or restrict provincial reform planning. As a number of provincial authorities indicated, they feel they cannot properly address a number of large issues, given the legal framework provided by the central government.  PCI has contributed to changing awareness and attitudes at the provincial level, which has supported administration reform efforts. “The procedure and time for setting up a new firm in Hanoi is very convenient and less timely by now. Some of our members reported that they only need about three days or less.” — The Hanoi SMEs Association Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 40 Relevance of Sub-Indexes to Firm Operation This section presents an analysis of issues covered by the PCI survey’s 10 sub-indexes and their relevance to local enterprises. Findings/Stakeholder Views Sub-indexes reflect key governance issues that concern private enterprises. Discussions with stakeholders revealed that the sub-indexes that make up the provincial composite score reflect key issues that concern businesses. A representative of a district business association in Da Nang indicated that the sub-indexes reflect some key issues that local businesses are always thinking about and concerned with. The sub-indexes cover some particularly important topics, such as access to land, labor and infrastructure that private enterprises must consider on a daily basis. An international stakeholder furthered this point by noting it is impossible for a firm to operate without appropriately planning for and having access to infrastructure, land and a competent labor force. Some private enterprises indicated that some issues of growing concern to them are not necessarily reflected in the PCI sub-indexes. These issues were highlighted by the mini-poll of private enterprises, which noted the issues of dispute settlement, the enforcement of counterfeit food and services, the unfair trade practices and various aspects of credit access and tax preferences. In Ca Mau, a private enterprise owner expressed his concern over the ineffectiveness of dispute settlement between him and the tax office regarding interpretation of tax preference legislation. He said it took six years to settle the dispute in his favor. Figure 8: Key Concerns of Business Conclusions  PCI sub-indexes highlight common issues affecting private enterprises.  Private enterprises report additional issues not addressed properly in the PCI sub-indexes, such as unfair competition and enforcement of anti-counterfeit goods policies. Identification of Key Drivers of Reform This section identifies and classifies the provinces visited by the evaluation team into categories based on the progress made with regards to administration reform. The section also presents an analysis of common factors that have been observed in provinces that have successfully undertaken the administration reform process as well as common factors in provinces that have been less successful. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 41 Understanding Implementation of Reform and Key Drivers of Success In each of the provinces visited, the evaluation team found evidence of economic governance reform over the last several years. However, the level of reform and its effectiveness has varied across the provinces. During fieldwork, the evaluation team did not encounter any instance where provincial leadership or officials within provincial line departments rejected PCI results or the idea of administrative reform. To better understand the variation in the level of administrative reform implemented between provinces, the evaluation team divided the provinces into three major groupings. The three groups include 1) those that implemented limited or ineffective reform; 2) those that conducted rapid assessment of PCI annual scores, which resulted in a short-term response; and 3) those that have taken long-term and strategic action. The first grouping consists of the provinces that experienced limited or ineffective reform. This grouping includes Ca Mau and Ninh Thuan. Both provinces are low performers in terms of PCI score (relative to the other five provinces visited by the team) and were unable to develop or implement comprehensive reform over the last several years. Neither province’s leadership exhibited sustained commitment to economic governance reform, which contributed to ineffective reform action planning and limited PPD activities. Additionally, neither province had an active or engaged business community. The second grouping includes provinces that have used the PCI to conduct assessments of their local business environments and have subsequently enacted policies and programs aimed at economic governance reform. This group is represented by Hanoi and Tuyen Quang. Both provinces are mid-tier performers in terms of PCI score and have demonstrated an attempt at understanding the local business environment through PCI analysis and engagement with the local business community and have, subsequently, taken action to implement economic governance reform. In this case, Tuyen Quang has traditionally been a low performer in terms of PCI rank; however, provincial officials have recently taken action to improve local economic governance through increased analysis (i.e., DCI) and PPD activities. The final group are provinces that have taken long-term strategic action aimed at improving local economic governance. This group includes Thai Nguyen, Dong Thap and Da Nang. These provinces represent the highest performers in terms of PCI score and provincial leadership in each has taken the view that economic governance planning is a long-term process and that short-term or ad-hoc plans that respond to the annual PCI scores are not sufficient to ensure comprehensive economic governance reform. Provincial officials have also prioritized PPD events to ensure consistent input and collaboration from the business community. In Da Nang, provincial officials no longer make annual PCI action plans, but rather the PPC actively monitors each line department to ensure that their departmental plans adequately respond to issues of economic governance and that the departments are meeting their objectives for implementation. Also, business association representatives are invited to attend and participate in monthly PPC briefings to help keep the business community engaged. • No evidence found at Reject PCI Results province level • Ca Mau • Ninh Thuan Limited/ineffective action • Hanoi Rapid assessment and response • Tuyen Quang • Thai Nguyen • Da Nang • Dong Thap Long-term and strategic action Figure 9: Administrative Reform by provinces Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 42 High Performers The top-performing provinces (listed by PCI rank) include Da Nang (1), Dong Thap (2) and Thai Nguyen (8). During fieldwork in these provinces, the evaluation team observed traits that it identified as integral to the successful reform process in each of the provinces. Prioritization of Provincial Competitiveness Improvement: In each case, provincial leaders made provincial competitiveness a policy priority and ensured adequate resources behind efforts to improve competiveness. In Da Nang, provincial authorities supported the role of the Institute of Economic and Social Development (IESD) in conducting a robust analysis of the PCI results, as well as the institute’s efforts to replicate the PCI survey within the province to utilize an evidence￾based approach to improving economic governance. Policy/Program Development That Complements Central Government Planning: Provincial leaders in each high-performing province developed and implemented reform policies while still complying with central government regulations. As noted by a number of provincial stakeholders, central government planning may impose limitations on provincial competitiveness planning. However, in high-performing provinces, provincial officials found innovative ways to develop and implement reform policies and programs while still complying with central government planning. Active Monitoring by Provincial Leaders: High-performing provinces also demonstrated effective coordination and collaboration between line departments to ensure comprehensive integration and implementation of competitiveness planning. This efficiency was driven in large part by active monitoring on the part of provincial leaders to ensure that line departments were meeting implementation targets and goals. In Thai Nguyen, the PCI task force sought input from line departments during the development of the province’s PCI action plan. As a result, action plans provided specific guidance to line departments and provided a framework for efficient coordination between them. During implementation of the action plan, provincial leaders took an active role in ensuring the realization of targets by each department. Active Engagement by Business Community: In high-performing provinces, the business community proved to be extremely capable and engaged with the process of economic governance reform and provincial competitiveness improvement. In Thai Nguyen, the PBA played an integral role in creating an analysis report with the PCI task force. Mass Media Engagement: In Thai Nguyen, provincial officials integrated a mass media campaign into the provincial PCI action plan. Stakeholders are constantly informed of the progress of any program or reform activity through the provincial newspaper and television. Low Performers The evaluation team notes that the opportunity for governance reform at the provincial level can be considered equal among the provinces given the national reform agenda and legislative framework adopted by the central government. Given the national government’s agenda, the variation in reform at the provincial level can be attributed to a number of key factors related to motivation and ability (MOA analysis framework). In Da Nang, the chair of the DPC in Hai Chau district became chair of the district business association, with party approval, so he could better facilitate dialogue between district officials and the district business community. He and his deputy in the DPC successfully connected enterprises from HCMC to entrepreneurs in Hai Chau district. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 43 Figure 10: MOA Analysis Framework I The evaluation team found that low performers generally lack local resources/capacity to reform, such as financial or human resources (e.g., Ninh Thuan). In other instances, a weak connection with the business community combined with an inability to effectively understand that local economic governance issues contribute to low performance, as in Ca Mau Province. Finally, in all low-performing provinces, the evaluation team found evidence of weak motivation and commitment to reform by provincial officials, which resulted in infrequent monitoring of reform implementation and a lack of accountability by line departments to provincial leaders. In provinces such as Ca Mau, Hanoi, Tuyen Quang and Ninh Thuan, the evaluation team found low prioritization of provincial competitiveness improvement by provincial officials. Stakeholders in each case indicated that improving provincial competiveness was not a top priority for provincial leadership. This resulted in a situation in which provincial officials were unable to develop detailed action plans and the PPC was unable to subsequently monitor line departments as they implemented provincial competitiveness action plans. Additionally, in each province, business associations were weak and often not integrated into the economic governance planning. In Ca Mau, the provincial business association was fractured and unable to operate effectively. As a result, it was unable to put forth a coherent agenda for reform. In Hanoi, the PPC noted that, given the 2011 City 141 Action Plan on improving its competitiveness and the business environment, all line departments and especially the DPI had delayed the implementation. The deployment meeting took place almost a year and half later, in April 2012. The city also reviewed and criticized all line departments for delayed development and delivery of each and coordinated action plans, which resulted in a lower PCI score and ranking in the previous two years. Conclusions  Effective implementation of reform policy is reliant on political leadership and horizontal coordination between line departments.  Pressure generated through public communications has informed all stakeholders and has helped push reform forward. V. OVERALL CONCLUSIONS Overall, the PCI program has supported provincial government to understand and to engage the business community to improve the business environment over the lifespan of the program. It has supported provincial governments to improve local business environments in line with the national/provincial administrative reform agendas, promoted public-private dialogue and increased understanding of economic governance from all stakeholders. Survey Implementation  The PCI team has employed high standards of quality in the design and implementation of the PCI survey. The team has been able to achieve this through the effective engagement with international and Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 44 domestic experts, national and provincial stakeholders, and other donors during the design and implementation phases of the PCI survey, as well as during the subsequent data dissemination and use activities.  Some areas of survey implementation need to be addressed on a continual basis, including reducing the length and complexity of the questionnaire, developing procedures to prevent cheating and coercion and increasing stakeholders’ knowledge about survey methodology. Promoting Public-Private Dialogue  PCI has achieved its objectives of promoting increased public-private dialogue at the provincial level by bringing provincial officials and businesses together on various aspects of economic governance. In some provinces, PPD has facilitated direct coordination between the private sector and government agencies of action plan development and implementation.  The quality of PPD events varies greatly across provinces due to provincial-level capacity and resources.  SMEs’ ability to participate in PPD was often limited in the provinces visited by the evaluation team.  Provincial diagnostic workshops are a useful tool in strengthening PPD and governance capacity; however, not all provinces were able to access technical assistance from the PCI team due to limited budgetary resources. This may result in challenges, especially for low-performing provinces. Improving Economic Governance Capacity  Understanding of economic governance issues and the capacity to implement reforms has improved at the provincial level. However, implementation and effectiveness of reform vary across provinces.  VCCI’s capacity and credibility have grown significantly as a result of PCI implementation and achievements. VCCI has proved to be a capable partner in the implementation of both PCI surveys and supporting activities. This has resulted from the long-term implementation of the program (10 years) with collaboration and support from USAID/Vietnam.  Due in large part to limited resources, PCI outreach activities have mainly focused on government officials at the provincial and national levels. Direct training to business associations has been limited in participation and geographical coverage.  Provinces’ capacity to independently perform in-depth analysis of PCI raw data is still low. While the VCCI/PCI team is willing to share, provision of raw data to provincial stakeholders has been limited because of the concern about local capacity. Sustainability  VCCI/PCI technical capacity is sustainable, although the number of VCCI (central and regional) staff who have technical capacity is still limited.  Beyond receiving direct funding from USAID/Vietnam, no established plan exists to sustain the PCI program financially. Contributing to Improving the Business Environment  PCI has contributed to creating momentum for stakeholders to coordinate around reform planning and implementation.  Key factors driving reform at the provincial level include provincial leaders’ prioritization of competitiveness improvement and commitment to actively monitoring the implementation of economic governance reform policies and programs by line departments. Additionally, an active and engaged business community that can help provincial officials identify key policy bottlenecks and provide input regarding a solution can contribute greatly to pushing through economic reforms.  Economic governance is one of many factors that influence the development of the domestic private sector. The PCI program’s overarching objective is to promote private sector growth in Vietnam. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 45 However, while PCI has made a contribution to improving economic governance in Vietnam, the total impact of the PCI program on private sector growth is still unclear. Crosscutting Activities  Management of the PCI program has been effective.  While it has successfully complied with the project M&E plan, the PCI program has been unable to systematically assess the quality of outreach activities sponsored by the PCI program (diagnostic workshops or training), which impacts the program’s ability to understand progress toward achieving objectives 2 and 3. Additionally, as the program is currently designed, it does not collect data on PPD events/activities (business cafés, entrepreneur rendezvous, etc.) that provinces initiated and sponsored. Understanding the quantity and effectiveness of such PPD events would be beneficial for provincial stakeholders in their efforts to maintain an effective and continuous PPD. VI. RECOMMENDATIONS Future Implementation of the PCI Survey  USAID should ensure that the PCI program continues to focus significantly on its core activities, which include implementation of the Domestic Private Sector Survey, the Foreign Investment Survey and diagnostic workshops. These aspects remain the most critical to the PCI program and should continue to be implemented with the same technical rigor and efficiency. These core activities are still and will continue to be relevant for provincial stakeholders, especially low-performing provinces, over the next three to five years or until evidence of PCI composite score conversion (among all provinces) shows that the ranking is no longer relevant in relation to absolute score.  Local economic environments can change quickly; therefore, it is important for the PCI team to review the survey questionnaire to annually assess the relevance of the sub-indexes and the specific questions posed. Such a review should include a pilot test of the questionnaire using a diverse sample of the country’s business community. The team can also consider collecting public opinion regarding survey content through its website. In addition, the PCI team may consider providing more in-depth information on the survey methodology to provincial stakeholders, to avoid misconceptions and misunderstandings regarding PCI data. Promoting Public-Private Dialogue  To further strengthen the effectiveness of PPD, the PCI team should consider providing technical guidelines for provinces on developing and implementing effective PPD activities. How to involve SME in PPD events should be included in the guidelines.  The PCI team should consider developing a criteria for the selection of provinces that will receive technical assistance and support through diagnostic workshops. Low-performing provinces that demonstrate a willingness to reform should be prioritized. Regional PCI workshops should also be maintained or expanded, since they provide a greater number of provinces with the opportunity to access technical assistance. Improving Economic Governance Capacity  USAID/Vietnam and the PCI team should consider increasing the number of trainings. Trainings should take place in all seven economic zones, with the primary beneficiaries being the local business community and newly appointed provincial leaders. By centering trainings in each of the economic zones, local private sector representatives can more easily access the trainings organized in their regions. Trainings should include a presentation of lessons learned and best practices from the region in regard to business association engagement and its impact on provincial reform. Trainings should also provide beneficiaries with a comprehensive analysis of PCI sub-indexes and its interpretation to the local reform Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 46 agenda. Finally, trainings should provide guidance on how business associations can best develop and maintain linkages between each other, including often-marginalized SMEs.  The PCI team should review and consolidate common stakeholder questions into a detailed frequently asked questions (FAQ) section on the website. This FAQ section would provide an easily accessible and official source of information to the public. Sustainability of PCI Program  The PCI team should develop a human resource plan for the PCI program. The plan should include a system for developing and utilizing additional well-trained permanent staff at both VCCI central and regional offices.  The PCI team and USAID should establish clear expectations regarding funding and sustainability of the PCI program. Next, representatives from the PCI team and USAID should work together to develop a clear and feasible financial sustainability plan that considers two or three of the most viable funding options, with forecasting and revenue models based on research and pilot testing. Better Monitoring Quality and Impacts of PCI Program Activities  The PCI team should employ in-depth monitoring of PPD and capacity-building activities. By doing this, the team can more systematically assess the quality and impact of such activities. This can be done by adding a section in the PCI survey regarding the quality of PPD or by the PCI team conducting a separate, smaller survey. To better monitor PPD and capacity-building activities that are directly part of the PCI program, such as diagnostic workshops and trainings, the PCI team can conduct pre- and post￾tests during events to gauge knowledge gained by participants and assess the helpfulness of specific topics covered during the events. VII. LESSONS LEARNED  Reform is an evolutionary process that needs consistent support over time without compromising the rigor of reform activities. Programs should be designed and implemented with a clearly defined and singular objective. This is especially true for programs supporting governance reform: Local stakeholders (both government and non-government partners) are engaged over a prolonged period. By following this strategy, programs can achieve their core objectives and produce positive externalities resulting from local reform.  Having the right partner is key to success. In an operating context of a transitional economy, it is necessary to choose a partner that is capable of learning and on delivering on both technical outputs and outreach activities. In the case of Vietnam, VCCI’s legal department is a good example of an organization that had a flexible structure that allowed it to continuously adapt during a long learning process and deliver on technical and outreach activities.  Capacity building is a key factor that enables and motivates local partners. By learning, adapting and ultimately being capable of operating independently, local partners can solidify ownership over activities and programs. By having ownership, partners can also mobilize resources not directly related to the program to support program activities. Capacity building is a process, not an ad hoc activity. It took the PCI program several years with direct and indirect (through USAID/VNCI) assistance from USAID/Vietnam before the PCI team became technically independent.  Knowledge-sharing is key in supporting reform efforts. Information can help guide stakeholders on program rationale, activities and general outcomes. Informed stakeholders can help push reform agendas forward.  Independent evaluations are critical to ensure program alignment and credibility. Evaluations help programs identify and maximize core strengths. They can also help validate activities and achievements to stakeholders at all levels. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 47  Employing mass media in an effective way not only raises awareness about reform activities, it can also be a source of legitimate pressure on stakeholders for reform. Independent evaluations are critical to ensure program alignment and credibility. Evaluations help programs identify and maximize core strengths. They can also help validate activities and achievements to stakeholders at all levels. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 48 REFERENCES The Government of Vietnam, Resolution 36a. (2015). http://vanban.chinhphu.vn/portal/page/portal/chinhphu/hethongvanban?class_id=509&mode=detail&docu ment_id=181767 The Government of Vietnam, Resolution 19. (2014). http://36mfjx1a0yt01ki78v3bb46n15gp.wpengine.netdna￾cdn.com/wp-content/uploads/2014/03/Resolution-19-NQ-CP-18-03-2014-of-the-Government-Informal￾English-Translation-24MAR.pdf The Government of Vietnam, Resolution 30c. (2011). http://www.chinhphu.vn/portal/page/portal/chinhphu/hethongvanban?class_id=509&mode=detail&docume nt_id=152523 Tu Phuong Nguyen (2013). Business associations and new policy dynamics in supporting private sector development in Vietnam. Australian Political Studies Association. http://www.auspsa.org.au/sites/default/files/business_associations_and_new_policy_dynamics_tu_phuong_ng uyen.pdf The People’s Committee of Hanoi – The Hanoi PPC (2015) Plan on Administrative Reform for 2016. https://www.thudo.gov.vn/newsdetail.aspx?NewsID=d99dee7e-bf5a-1b48-9d11- 93ab0193acf0&CateID=28ccfbee-dd0e-844c-a1cf-f4418b4d5697 The People’s Committee of Hanoi –The Hanoi PPC (2013) Review of Action plan 141 implementation. http://hanoi.gov.vn/ubndthanhpho/-/hn/AlBuH0bMcRAB/2807/106457/13/kiem-iem-trach-nhiem-cac-on￾vi-trong-thuc-hien-ke-hoach-141kh-ubnd.html;jsessionid=pal04eiRFVPorSijrnLRsFjg.app2 Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 49 ANNEX 1: LIST OF STAKEHOLDERS U.S. GOVERNMENT (USG) AGENCIES • Joakim Parker, Mission Director, United States Agency for International Development in Vietnam (USAID) • Randolph Flay, Assistant Director, United States Agency for International Development in Vietnam (USAID) • Michael Trueblood, Director of Economic Growth & Governance Office, United States Agency for International Development in Vietnam (USAID) • Emily Rupp, Deputy Director of Program Development Office, United States Agency for International Development in Vietnam (USAID) • Nguyen Thi Ha, M&E Specialist/Gender Advisor, United States Agency for International Development in Vietnam (USAID) • Le Thi Thanh Binh, Development Assistance Specialist of Economic Growth & Governance Office, United States Agency for International Development in Vietnam (USAID) IMPLEMETING PARTNERS Vietnam Chamber of Commerce and Industry (VCCI) • Vu Tien Loc, Chairman • Edmun Malesky, Program Lead Researcher • Dau Anh Tuan, General Director of Legal Department • Pham Ngoc Thach, Deputy Director of Legal Department • Nguyen Ngoc Lan, Program Technical Coordinator • Nguyen Thi Thu Hang, Program Assistant Vietnam Chamber of Commerce and Industry – Da Nang Branch • Nguyen Tien Quang, Deputy Director • Nguyen Toan Thang, Executive Vietnam Chamber of Commerce and Industry – Can Tho Branch • Nguyen Phuong Lam, Deputy Director GOVERNMENT OF VIETNAM (GVN) • Nguyen Xuan Tien, Deputy Head of Foreign Economic Relations Department, Ministry of Planning and Investment • Le Truong Son, Foreign Economic Relations Department, Ministry of Planning and Investment INTERNATIONAL ORGANIZATIONS • Kyle Kelhofer, Acting Country Manager, International Finance Corporation (IFC) • Soren Davidsen, Senior Governance Specialist of Governance GP, World Bank (WB) ACADEMIA AND INDEPENDENT RESEARCHERS • Nguyen Van Thang, Director, Asia-Pacific Institute of Management • Pham Chi Lan, Senior Expert • Dao Huu Hoa, Vice Rector, Da Nang University of Economics • Nguyen Truong Son, Dean of Department of Business Administration, Da Nang University of Economics • Le The Gioi, Senior Lecturer of Department of Business Administration, Da Nang University of Economics Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 50 • Le Van Huy, Lecturer of Department of Business Administration, Da Nang University of Economics • Ngo Xuan Hoa, Head of Economics Division, Can Tho Office of People’s Committee PROVINCIAL AND DISTRICT LEVEL Ha Noi City City level • Tran Ngoc Nam, Deputy Director, Hanoi Department of Planning and Investment • Nguyen Huu Luong, Deputy Chief Officer of Department Office, Hanoi Department of Planning and Investment • Nguyen Huu Loi, Head of Administrative Operations Division, Hanoi Department of Planning and Investment • Nguyen Thi Minh Ha, Head of FDI Management Division, Hanoi Department of Planning and Investment • Tu Danh Trung, Head of Business Registration Center, Hanoi Department of Planning and Investment • Tran Quang Hung, Deputy Head of Industry Management Division, Hanoi Department of Planning and Investment • Pham Thi Diem Ngoc, Representative, Small & Medium Enterprise Supporting Center, Hanoi Department of Planning and Investment • Trinh Quang Anh, Officer of Administrative Operations Division, Hanoi Department of Planning and Investment • Hoang Le Viet Anh, Officer of Administrative Operations Division, Hanoi Department of Planning and Investment • Tran Thi Phuong Lan, Deputy Director, Hanoi Department of Industry and Trade • Tran Dinh Nghia, Deputy Chief Officer of Department Office, Hanoi Department of Industry and Trade • Nguyen Trong Toi, Head of Planning and Finance Division, Hanoi Department of Industry and Trade • Nguyen Thuy Quynh Van, Head of Industry Management Division, Hanoi Department of Industry and Trade • Nguyen Dinh Thang, Head of Electricity Management Division, Hanoi Department of Industry and Trade • Nguyen Van Minh, Head of Environmental Protection Management Division, Hanoi Department of Industry and Trade • Nguyen Linh, Deputy Head of Planning and Finance Division, Hanoi Department of Industry and Trade • Luong Minh Duc, Deputy Head of Industry Management Division, Hanoi Department of Industry and Trade • Nguyen Ngoc Long, Deputy Head of Trade Management Division, Hanoi Department of Industry and Trade • Le Thi Minh Trang, Officer, Hanoi Department of Industry and Trade • Phan Thanh Quang, Deputy Head of Land Planning Division, Hanoi Department of Natural Resources and Environment • Tran Duc Minh, Officer of Land Planning Division, Hanoi Department of Natural Resources and Environment • Vien Viet Hung, Deputy Director, Hanoi Department of Taxation • Nguyen Tien Truong, Head of Administration Division, Hanoi Department of Taxation Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 51 • Le Huu Dong, Head of Legal Division, Hanoi Department of Taxation • Le Tuan Anh, Deputy Head of Communications Division, Hanoi Department of Taxation • Dinh Xuan Tan, Deputy Head of Administrative Operations Division, Hanoi Department of Taxation Business Association Stakeholders • Nguyen Ngoc Tuan, Deputy Chairman, Hanoi Business Association • Dinh Tien Duc, Deputy Secretary General, Hanoi Small and Medium Enterprise Association • Trinh Thi Ngan, Head of Advisory Board, Hanoi Small and Medium Enterprise Association • Luu Minh Duc, Member of Executive Committee, Hanoi Small and Medium Enterprise Association Private Enterprises • Chu Duc Luong, Chairman, Phu My Group JSC. Da Nang City City level • Nguyen Van Hung, Deputy Director, Da Nang Institute for Socio – Economic Development • Le Canh Duong, Director, Da Nang Investment Promotion Center • Nguyen Thi Anh Thu, Deputy Head of Information Division, Da Nang Investment Promotion Center • Pham Thi Bich Hien, Officer of Investment Cooperation Division, Da Nang Investment Promotion Center • Nguyen Thi Thuy Mai, Deputy Director, Da Nang Department of Industry and Trade • Nguyen Quang Tam, Deputy Chief Officer of Department Office, Da Nang Department of Industry and Trade • Dang Cong Chu An, Head of Administrations Division, Da Nang Department of Industry and Trade • Vo Thi Ha Phuong, Head of Trade Management Division, Da Nang Department of Industry and Trade • Huynh Thi Ngoc Yen, Head of Industry Management Division, Da Nang Department of Industry and Trade • Nguyen Thuy Nhung, Officer of Planning and Finance Division, Da Nang Department of Industry and Trade • Nguyen Tri Tho, Deputy Head of Land Economics Division, Da Nang Department of Natural Resources and Environment • Tran Thanh Nam, Officer of Land Economics Division, Da Nang Department of Natural Resources and Environment • Nguyen Dinh Anh, Deputy Director, Da Nang Department of Taxation • Ho Thi Thu Trang, Deputy Head of Tax Payer Support Division, Da Nang Department of Taxation • Vy Van Tu, Deputy Head of Information Technology Division, Da Nang Department of Taxation • Phan Thi Mai Phuong, Deputy Head of Tax Declaration Division, Da Nang Department of Taxation • Nguyen Thi Thanh Van, Officer, Da Nang Department of Taxation • Nguyen Thi Thanh Huong, Deputy Director, Da Nang Department of Planning and Investment Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 52 • Le Thi My Ngoc, Officer, Da Nang Department of Planning and Investment Business Association Stakeholders • Pham Bac Binh, Chairman, Small and Medium Enterprise Association • Nguyen Van Ly, Vice Chairman – Secretary General, Small and Medium Enterprise Association • Le Anh, Chairman, Hai Chau District Business Association • Tran Quoc Bao, Deputy Chairman, Hai Chau District Business Association • Nguyen Quang Minh, Deputy Chairman, Hai Chau District Business Association • Nguyen Hong Son, Deputy Chairman, Hai Chau District Business Association • Bui Thi Kim Hoa, Head of Economics Division, Hai Chau District People’s Committee • Phan Ha Phuong, Officer of Economics Division, Hai Chau District People’s Committee Private Enterprises • Doan Minh Chau, Senior Operations Manager, MagRabbit Vietnam • Trinh Bang Co, Director, Phuong Dong Viet Company • Ly Dinh Quan, General Director, SQ Vietnam Company • Tran Minh Dong, General Director, Viettronimex Company Thai Nguyen Province Province Level • Nhu Van Tam, Deputy Chairman, Thai Nguyen People’s Committee • Nguyen Thanh Binh, Deputy Director, Thai Nguyen Department of Investment and Planning • Tran Tuan Viet, Officer, Thai Nguyen Department of Investment and Planning • Nguyen Ngo Quyet, Director, Thai Nguyen Department of Industry and Trade • Nguyen Quoc Huy, Deputy Director, Thai Nguyen Department of Industry and Trade • Don Van Thuy, Head of Planning and Finance Division, Thai Nguyen Department of Industry and Trade • Phung Tuan Anh, Officer of Administrative Operations Division, Thai Nguyen Department of Industry and Trade • Hoang Van Binh, Officer of Administrative Operations Division, Thai Nguyen Department of Industry and Trade • Dam Anh, Officer of Land Management Division, Thai Nguyen Department of Natural Resources and Environment • Nguyen Thanh Ha, Officer of Land Management Division, Thai Nguyen Department of Natural Resources and Environment • Pham Anh Tuan, Officer of Land Management Division, Thai Nguyen Department of Natural Resources and Environment • Nguyen Van Ha, Deputy Director, Thai Nguyen Department of Taxation • Nguyen Quang Cuong, Head of Personal Income Tax Division, Thai Nguyen Department of Taxation • Nguyen Thi To Oanh, Head of Tax Accounting Division, Thai Nguyen Department of Taxation • Pham Thi Hoai Lan, Head of Communications Division, Thai Nguyen Department of Taxation • Nghiem Quang Khuong, Head of Tax Supervision Division, Thai Nguyen Department of Taxation • Vi Van Tu, Deputy Head of Tax Supervision Division, Thai Nguyen Department of Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 53 Taxation Business Association Stakeholders • Nguyen Van Thang, Deputy Director, Provincial Business Association • Pham Van Quang, Deputy Director – Secretary General, Provincial Business Association • Hoang Thi Thuy, Deputy Director, Provincial Business Association • Nguyen Thi Tuyet, Deputy Director, Provincial Business Association • Hoang Huu Son, Member of Executive Committee, Provincial Business Association • Chu Phuong Dong, Member of Executive Committee, Provincial Business Association • Pham Thi Quy, Deputy Chairwoman, Association of Female Enterpreneurs • Nguyen The Dai, Chairman, Dai Tu District Business Association • Nguyen Xuan Thu, Deputy Chairman, Song Cong District Business Association • Nguyen Hung Cuong, Chairman, Pho Yen District Business Association • Phan Quang Huy, Chairman, Phu Luong District Association Private Enterprises • Do Trong Hiep, General Director, Da Huong Tourist and Hotel Company • Dinh Ngoc Cu, Director, Phat Loc Company, Ltd. • Nguyen Quang Hai, Vice Director, Thai Nguyen Truong Hai Co., Ltd. Tuyen Quang Province Province Level • Tran Van Luong, Deputy Director, Tuyen Quang Department of Planning and Investment • Nguyen Thi Bich Ngoc, Head of International Economics Division, Tuyen Quang Department of Planning and Investment • Mr. Hung, Officer of International Economics Division, Tuyen Quang Department of Planning and Investment • Ngo Tien Ha, Deputy Director, Tuyen Quang Department of Industry and Trade • Pham Quang Thieu, Head of Industry Management Division, Tuyen Quang Department of Industry and Trade • Nguyen Thu Trang, Head of Administrative Operations Division, Tuyen Quang Department of Industry and Trade • Pham Van Luong, Deputy Director, Tuyen Quang Department of Natural Resources and Environment • Nguyen Manh Tien, Chief Officer of Department Office, Tuyen Quang Department of Natural Resources and Environment • Pham Dinh Tu, Head of Land Registration Division, Tuyen Quang Department of Natural Resources and Environment • Nguyen Thi Hong Thai Thai, Head of Information Technology Center, Tuyen Quang Department of Natural Resources and Environment • Pham Dinh Trung, Officer of Land Registration Division, Tuyen Quang Department of Natural Resources and Environment • Le Manh Hai, Deputy Director, Tuyen Quang Department of Taxation • Hoang Thanh Phong, Head of Communications Division, Tuyen Quang Department of Taxation • Nguyen Quoc Tuan, Head of Administrative Operations Division, Tuyen Quang Department of Taxation • Nguyen Ngoc Xuan, Head of Tax Accounting Division, Tuyen Quang Department of Taxation Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 54 • Truong Xuan Quy, Director, Tuyen Quang Investment Promotion Center • Hoang Hai Truong, Deputy Director, Tuyen Quang Investment Promotion Center • Trinh Thi Lan, Head of Investment Division, Tuyen Quang Investment Promotion Center • Nguyen Thi Hai, Head of Planning and Administrations Division, Tuyen Quang Investment Promotion Center District level • Ma Van Long, Vice Chairman, Chiem Hoa District People’s Committee • Ma Thi Nhung, Deputy Chief Officer of People’ Committee Officer, Chiem Hoa District People’s Committee • Ha Phuc Lam, Head of Infrastructure – Economics Division, Chiem Hoa District People’s Committee • Dang Thi Mai, Deputy Head of Infrastructure – Economics Division, Chiem Hoa District People’s Committee • Dong Van Ha, Deputy Head of Natural Resources and Environment Division, Chiem Hoa District People’s Committee • Truong Quang Chien, Deputy Head of Planning and Finance Division, Chiem Hoa District People’s Committee • Tran Van Phu, Deputy Head of Agriculture and Rural Development Division, Chiem Hoa District People’s Committee • Tran Van Tuan, Deputy Head of Internal Affairs Division, Chiem Hoa District People’s Committee • Nguyen Minh Phu, Deputy Head of Labor, Invalids and Social Affairs Division, Chiem Hoa District People’s Committee Business Association Stakeholders • Lai Van Que, Chairman, Tuyen Quang Small and Medium Enterprise Association • Hoang Viet Chien, Executive, Tuyen Quang Small and Medium Enterprise Association • Nguyen Van Huy, Chairman, Tuyen Quang Association of Young Enterpreneurs Private Enterprises • Vi The Manh, Director, Linh Luc Co., Ltd. • Nguyen Vu Linh, Director, Chinh Hoa Steel Company, Ltd. • Hua Van Thu, Director, Yen Thu Private Enterprise Ninh Thuan Province Province Level • Luu Xuan Vinh, Chairman, Ninh Thuan People’s Committee • Truong Xuan Vy, Deputy Director, Ninh Thuan Department of Planning and Investment • Le Thi Thanh Thuy, Director of Economic Development Office (EDO), Ninh Thuan Department of Planning and Investment • Le Thi Kim Phuong, Head of Development Support Division, Economic Development Office (EDO), Ninh Thuan Department of Planning and Investment • Pham Dang Thanh, Deputy Director, Ninh Thuan Department of Industry and Trade • Ms. Nguyet, Officer of Industry Management Division, Ninh Thuan Department of Industry and Trade • Mr. Anh, Officer of Commerce Management Division, Ninh Thuan Department of Industry and Trade • Le Ngoc Thach, Deputy Director, Ninh Thuan Department of Natural Resources and Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 55 Environment • Do Xuan The, Chief Officer of Department Office, Ninh Thuan Department of Natural Resources and Environment • Nguyen Van Thanh, Deputy Head of Land Management Division, Ninh Thuan Department of Natural Resources and Environment • Doan Hanh Phuc, Deputy Director, Ninh Thuan Department of Taxation • Tran Chi Lieu, Head of Communications Division, Ninh Thuan Department of Taxation • Bao Huan, Head of Administrative Operations Division, Ninh Thuan Department of Taxation • Bui Ngoc Chien, Deputy Head of Tax Accounting Division, Ninh Thuan Department of Taxation • Luu Tai, Head of Tax Supervision Division, Ninh Thuan Department of Taxation • Ho Cong Hoai Ai, Head of Technology Division, Ninh Thuan Department of Taxation Business Association Stakeholders • Trinh The Cuong, Chairman, Association of Young Enterpreneurs Private Enterprises • Cao Minh Hoang, Director, Trung Nam Development & Investment Co., JSC. • Dinh Van Du, Deputy Director, Trung Nam Development & Investment Co., JSC. • Dao Duc Kiem, Director, Hoang Long Company, Ltd. • Pham Van Sinh, Director, Truc Nguyen Service & Commerce Company, Ltd. • Nguyen Thanh Hong, Director, Kim Song Ma Co., Ltd. • Phan Luu Phuong, Director, Dai Thanh Co., Ltd. • Le Thi Thuy, Director, Tam Tu Steel Company, Ltd. Dong Thap Province Province Level • Chau Hong Phuc, Vice Chairman, Dong Thap People’s Committee • Truong Hoa Chau, Deputy Director, Dong Thap Department of Planning and Investment • Nguyen Dinh Phuong Quyen, Head of Business Registration Division, Dong Thap Department of Planning and Investment • Ngo Thanh Thanh, Deputy Head of Business Registration Division, Dong Thap Department of Planning and Investment • Nhi Van Khai, Director, Dong Thap Department of Industry and Trade • Nguyen Huu Dung, Deputy Director, Dong Thap Department of Industry and Trade • Huynh Van Hoang, Head of Planning Division, Dong Thap Department of Industry and Trade • Nguyen Trung Ngay, Director, Dong Thap Department of Natural Resources and Environment • Nguyen Phu Duc, Head of Land Management Division, Dong Thap Department of Natural Resources and Environment • Nguyen Nhat Thanh, Officer of Land Management Division, Dong Thap Department of Natural Resources and Environment • Le Thanh Tong, Director, Dong Thap Department of Taxation • Vo Viet Thang, Head of Communications Division, Dong Thap Department of Taxation • Vo Quoc Hai, Deputy Head of Communications Division, Dong Thap Department of Taxation Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 56 District Level • Le Hoang Bao, Vice Chairman, Cao Lanh People’s Committee • Viet Long, Head of Economics and Infrastructure Division, Cao Lanh People’s Committee Business Association Stakeholders • Do Quoc Hung, Chairman, Provincial Business Association / Director of Investment Promotion Center • Nguyen Xuan Vi, Chief Officer of the Provincial Business Association Office, Provincial Business Association • La Tuan Cuong, Head of Administrative Operations Division, Investment Promotion Center Private Enterprises • Pham Thanh Binh, Director, Bich Chi Company, JSC. • Nguyen Van Ben, Director, Ben Linh Company, Ltd. • Luong Nguyen Duy Thong, Director, K&Y Company, Ltd. • Truong Thanh Son, Director, Viet-France Cosmetics Successful Company, Ltd. • Dinh Thi Kim Tuyen, Director, Bao Huy Construction and Investment Company, Ltd. • Tran Huy Hien, Director, Le Nhu Brothers Co., Ltd. • Ngo Chi Cong, Director, Khoi Minh Thanh Cong Company, Ltd. Ca Mau Province Province Level • Lam Van Bi, Deputy Chairman, Ca Mau People’s Committee • Mai Huu Chinh, Director, Ca Mau Department of Planning and Investment • Nguyen Chi Thien, Deputy Director, Ca Mau Department of Planning and Investment • Vo Minh A, Head of Investment and Cooperation Division, Ca Mau Department of Planning and Investment • Truong Dang Khoa, Deputy Head of Business Registration Division, Ca Mau Department of Planning and Investment • Vo Chau Kim Hoang, Officer, Ca Mau Department of Planning and Investment • Au Truong An, Deputy Director, Ca Mau Center for Enterprise Development Promotion • Phan Van Minh, Deputy Director, Ca Mau Department of Natural Resources and Environment • Truong Thanh Toan, Chief Officer of Department Office, Ca Mau Department of Natural Resources and Environment • Huynh Van Minh, Deputy Director, Ca Mau Department of Industry and Trade • Vuong Ngoc Hao, Chief Officer of Department Office, Ca Mau Department of Industry and Trade • Nguyen Viet Trung, Head of Legal Division, Ca Mau Department of Industry and Trade • Do Gia Toan, Head of Science and Technology Division, Ca Mau Department of Industry and Trade • Nguyen Van Hien, Deputy Head of Electricity Management Division • Nguyen Thanh Trung, Deputy Head of Trade Management Division • Duong Trung Khoi, Officer, Ca Mau Department of Industry and Trade • Chau Thanh Su, Deputy Director, Ca Mau Department of Taxation • Nguyen Van Tuan, Head of Tax Inspection Division, Ca Mau Department of Taxation • Nguyen Van Duong, Head of Tax Declaration Division, Ca Mau Department of Taxation • Tu Hoang Anh, Deputy Director, Ca Mau Department of Labors, Invalids and Social Affairs Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 57 District Level • Tran Doan Hung, Deputy Chairman, Nam Can People’s Committee Business Association Stakeholders • Ngo Tri Dung, Chairman, Provincial Business Association • Nguyen Le Thai, Secretary General, Provincial Business Association • Lam Hieu, Deputy Chairman, Provincial Business Association • Huynh Ngoc Nhan, Deputy Chairwoman, Provincial Business Association Private Enterprises • Nguyen Duc Trung, Director, Red Tiger Prawn Trading Company • Khong Duy Toan, Director, Khanh Toan Construction One Member Ltd., Company • Le Ngoc Hoa, Director, Ca Mau Investment and Construction Company • Duong Tuan Anh, Owner, Household Business Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 58 ANNEX 2: INTERVIEW GUIDE: PROVINCIAL STAKEHOLDERS Introduction The United States Agency for International Development (USAID) currently provides funding for the annual PCI program. Presently, USAID is in the process of trying to understand how the PCI program has contributed to improving business enabling environment in the provinces. USAID has contracted our firm, Management Systems International, to send expert teams to different provinces in the Vietnam to speak with various stakeholders, including government officials and private enterprises in order to ascertain their opinions and perceptions of local economic governance issues. VCCI as the partner of USAID who conduct the annual surveys have recommend the representatives from provincial/cities agencies to share your opinions and observation of the business environment in general and the of the PCI program impact in particular. We assure you that our conversation today is purely for our program understanding and learning efforts in order to help construct a more effective PCI program in the coming years. Your responses will not be shared publicly. The discussion should last approximately 1.5 hours. After that, I will recap all the point discussed and we are willing to hear your comments. In the end, please fill in our mini survey which only takes about 5 minutes. We also would like you to share with us relevant materials which provides more insights for the evaluation. Let me introduce our team today…….And please let us know about your information in brief (position, how many years you were with PCI, etc.) Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 59 INTERVIEW GUIDE Introduction: Background Information 1. Name 2. Title 3. Role and responsibilities 4. Organization 5. Organization operations and activities regarding PCI improvement program of the PPC Suggested questions A. PROVINCIAL STAKEHOLDERS A. 1. PCI Task force (Briefing meeting) 1. The adoption of PCI improvement action plan and task assignment for provincial government agencies a) Which members/agencies in PCI task force are in charge of implementing tasks under the action plan? b) What is the organizational structure of the PCI task force? How does the task force delegate tasks to its members? c) Does the PCI task force have a supporting mechanism that encourages the performance of its members? d) Which reporting mechanism does PPC/task force apply to monitor the performance of related departments and authorities at lower levels? 2. Diagnostic of key issues related to provincial governance via PCI results a) Do the provincial authorities analyze PCI results? Which departments/officials are responsible for data analysis? b) What methodology is used for data analysis? How long does it take? 3. Provincial action plan a) Please list the action plans developed in your province. Which department(s) are in charge of action plan development? Which department(s) are in charge of implementation? Please specify the implementation period for the action plan(s). Ask the stakeholders to share provincial action plan, department action plans, and performance reports. b) How is PCI information used in action plan development? How are sub-indices used for analysis? How do other programs/priorities program support the action plans? 4. Complete the legal and economic regulatory framework a) Which policies/measures have been developed and which constraints in local economic governance have they addressed, especially in the last 5 years? Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 60 b) How are the policies/measures enacted? 5. PCI task force’s assessments on Public-Private dialogue 5.1. Frequency and effectiveness of the PPD dialogue events: a) Who often takes the lead in organizing and directing dialogues/meetings (PBA, provincial authorities, or both)? b) In what kind of forum do dialogues events take place? c) What sort of issues have been discussed in these interactions? d) How have enterprises in general, and SMEs in particular, raise their voice during events? e) What issues have been addressed due to such dialogues? Please provide specific examples. 5.2 PCI impact on public-private dialogue: a) How has the nature of public-private dialogue changed since PCI publication? b) Was the PCI report publication a direct cause of meetings/dialogues? c) During the last 10 years of PCI, what have been the major develops regarding PPD in the province? d) Are there any constraints regarding public-private dialogue? What actions should provincial authorities take to address such constraints in the future? 6. Local economic governance capacity How does PCI task force assess PCI impacts on the provincial governance capacity from both public agencies and private enterprises? a) Has the PCI program helped to create a better business enabling environment? b) How have the costs of business operations and transactions been affected? Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 61 A. 2. Provincial Stakeholders (following meetings) Provincial stakeholders include: 1. Department of Planning and Investment; 2. Department of Natural Resource and Environment; 3. Department of Industry and Trade; 4. Department of Taxation; 5. Center for Investment Promotion; 6. Leaders of the Provincial People’s Committee Note: For DPI, ask about their role in coordinating PCI-related activities, developing provincial and department action plan, and following the monitoring and reporting mechanism. 1. Are stakeholders aware of PCI and to what extent did they use PCI information for PCI improvement action plan development? Ask them further question to see their understanding of PCI: What does PCI measure? What are its sub-indices and their respective weights? Ask stakeholders to give specific examples and access to their action plan/performance report. 2. Action plan implementation. How are PCI improvement action plans received and implemented by local authorities at provincial, district and commune level? • Time frame? • What are issued discussed? • How did the organization implement the tasks? • Results? • What are the drivers and constraints in the implementation process? 3. Complete legal and economic regulatory framework a) What policies/measures have your organization advised PPC to issue? What constraints have the policies/measures in economic governance addressed? b) How are the policies/measures enacted? 4. The department’s assessments on Public-Private dialogue 4.1. T he a ctiveness a n d effectiveness of the d ia logue: a) Have you taken the lead in organizing and directing dialogues/meetings? b) Are all parties treated equally in the dialogues/meetings? 4.2 P CI imp a cts on p u b lic - priv ate dial ogue a) How have public-private dialogues changed since PCI publication? b) Was PCI publication a direct cause of meetings/dialogues? c) Are there any constraints in public-private dialogue? What actions should your organization take to remove such constraints in the future? Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 62 5. Local economic governance capacity How do you assess PCI impacts on improving governance capacity in your department? How have staff performance and responsibilities changed since PCI publication? How has the image of administrative agencies changed since PCI publication? How has work results been affected? Please give specific example. 6. Impacts on provincial investment and business operation What are the substantial investment and business operations in the province in the last 5 years? How do PCI and activities to improve business environment contribute to the investment inducement of the province? 7. Department’s action plan in the future Does your department have any action plan related to PCI? Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 63 ANNEX 3: INTERVIEW GUIDE: PRIVATE ENTERPRISES Introduction The United States Agency for International Development (USAID) currently provides funding for the annual PCI program. Presently, USAID is in the course of trying to understand how the PCI program has contributed to improving business enabling environment in the provinces. USAID has contracted our firm, Management Systems International, to send expert teams to different provinces in Vietnam to speak with various stakeholders including government officials and private enterprises in order to ascertain their opinions and perceptions of local economic governance issues. Your company has been chosen because you have participated in the PCI survey and we hope to gather information on your opinions of the PCI program and how the PCI program has affected the business environment in your province and your business in particular. We assure you that our conversation today is purely for our program evaluation efforts and will not be shared publicly. The discussion should last approximately 1.5 hours. After that, I will recap all the point discussed and we are willing to hear your comments. In the end, please fill in our mini survey which only takes about 5 minutes. Let me introduce our team today…….And please let us know about your business in brief (key line of business, establishment year, how many years you were with PCI, etc.) Section 1: Background Information 1. Name 2. Title 3. Role and responsibilities 4. Company name 5. Field of Business Operation Section 2: Interview Guide 1. Cou ld y ou sha r e w ith u s y ou r b u siness si t u a t i o n s r ecen t ly ? What are the key advantages and challenges did you have in the business establishment stage, and in the maturity stage? 2. W hat a r e the d r ivers for y ou r d ecision to in vest a n d r egister a s a b u siness in y ou r p r ovin ce? Ask about the time they decided to expand their business; market opportunities; planning; and land access? How was the business environment at that time? What challenges did you face? Which support did you get from the local authorities? Compared with the time you decided on expansion, is business environment now more favorable or challenging, and why? Do you wish to change your investment decision if you had a chance? How would you change it? Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 64 3. W hich d ep a r tmen ts ha s y ou r fir m ma in ly w or ked w ith? How do you assess the performance of staff in the department then and now? Which department are you most impressed by, and by which services? Which services are you unsatisfied with? Which services experience slow progress? 4. W ha t p olicy a n d p r ogr a mmin g su p p or ts ha s the p r ovin ce offe re d y our firm s ince the est a b lishmen t ? Please give specific example. When did you receive such supports? How did the support improve your business operations and investment? 5. W hich d ia logues b etween p r ovin cia l a u thorities a n d fir ms a r e y ou a w a r e o f or have participate d in? How often are such events organized and what results have they brought about? E.g.: Is the dialogue helpful? Do your firm and other firms have the chance to raise your voice? Are your issues heard and addressed? How did they response to your concern? 6. A r e y ou a memb er of a n y loca l b u siness a ssocia tion s? Which association? If you are, how do you assess the association? How does the association advocate the voice of your firm? If you have not become a member, please specify the reason why. 7. Ha ve y ou hea r d a b ou t P CI? Where is your province in PCI ranking? Is the rank higher or lower compared with the previous year? What are the impacts of PCI on business environment in the province and in your firm in particular? 8. Ha ve y ou ever take n th e PCI surve y ? a) If you have, please share your opinion about the questionnaire? Did you fill out the questionnaire yourself or assign your employee to do that? How long did it take to complete the questionnaire? b) If you have not, are you willing to take the PCI survey? Why (not)? c) Do you have any recommendations to improve the questionnaire and survey process in the coming years? 10. P lease sha r e y ou r con cern a b ou t shor tcomin gs in the econ omic gover n a n ce in y ou r p r ovin ce? Ask about the transparency, the fairness of the authorities, the predictability of regulations in general, and of policies on capital access, land access, labor, taxation, and social insurance, etc. 11. For b u siness household s: What are the drivers/constraints that affect your household decision not to register as a business? RECAP CONDUCT MINI SURVEY THANK YOU SO MUCH! Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 65 ANNEX 4: QUESTIONNAIRE: GOVERNMENT STAKEHOLDERS 1. To what extent are you and your agency involved in local economic governance? 1 2 3 Not very involved Involved to a certain degree Strongly involved 2. To what extent do you use data from the PCI in your work? 1 2 3 4 Never Occasionally Sometimes Frequently Please tell us if you agree or disagree with following statements: 3. The PCI program is very effective in motivating local economic governance reform as it relates to creating an enabling environment for private enterprises. 1 2 3 4 5 Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree 4. Data and results from the PCI program provide you with a good understanding about economic governance and competitiveness issues in your province. 1 2 3 4 5 Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree 5. PCI program helps increase local capacity to deal with economic governance issues. 1 2 3 4 5 Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree 6. PCI program helps local government/governance agencies to take actions to improve business environment for private sector's growth. 1 2 3 4 5 Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree 7. PCI program helps to improve the transparency of the local policymaking process. 1 2 3 4 5 Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree 8. PCI program provides local private enterprises with opportunities to engage in dialogue with local management authorities/policy makers. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 66 1 2 3 4 5 Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 67 ANNEX 5: QUESTIONNAIRE: PRIVATE ENTERPRISES INTRODUCTION The United States Agency for International Development (USAID) provides funding for the annual PCI program. Presently, USAID is in the process of trying to understand how the PCI program has contributed to improving business enabling environment in the provinces. The donor wants to conduct a mini survey on the Provincial Competitiveness Index (PCI) in the private business community in Hanoi, Tuyen Quang, Thai Nguyen, Danang, Ninh Thuan, Dong Thap, and Ca Mau. In each province, we randomly selected a number of private firms to conduct the survey. We would appreciate your support in completing this short questionnaire which takes about 7 minutes. You DO NOT need to provide your personal and firm information (including company name, address, and contact information) in this questionnaire. The information you provide will be of great contribution for us to formulate recommendations that improve the business environment in all provinces. Thank you so much for your kind support. Evaluation team. Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 68 1. Firm Size: Your company is categorized as: (Please put an x in the corresponding box) Small or medium enterprise (<300 employees or <100 if operating in service sector) Large firm (>300 employees or >100 if operating in service sector) 2. Please note your level of agreement with the following statements (for each statement, please choose 1 option only) The local business environment has improved over the last 5 years. 1 2 3 4 5 Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree You feel satisfied with the business environment in your province. 1 2 3 4 5 Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree In which aspect(s) of the business environment do enterprises encounter the most difficulties? (You can choose more than 1 option for this question) Land access 1 Market entry 2 Credits/Loans 3 Taxation 4 High quality labor 5 Others (please specify) 6 Your provincial government has been making efforts to maintain substantive dialogue with the local business community. 1 2 3 4 5 Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree Your provincial government has carried out administrative reforms to improve business environment. 1 2 3 4 5 Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 69 3. Are you aware of the Provincial Competitiveness Index (PCI)? (Please put an x in the corresponding box) Never If you have never heard of PCI, please skip the rest of the questionnaire I am aware of PCI, but not understand the index profoundly I am fully aware of the PCI 4. Please note your level of agreement with the following statements (for each statement, please choose 1 option only) I am fully aware of PCI sub-indexes. 1 2 3 4 5 Disagree Partially disagree Neither agree nor disagree Agree Strongly agree Your province’s performance on PCI is in line with your assessment of the business environment. 1 2 3 4 5 Disagree Partially disagree Neither agree nor disagree Agree Strongly agree Vietnam Evaluation, Monitoring and Survey Services | Provincial Competitiveness Index Performance Evaluation 70 5. Have you ever taken part in PCI survey? Yes, I have No, I haven’t I don’t remember 6. Have you ever been consulted about PCI results? (Please put an x in the corresponding box) Yes, I have No, I haven’t I don’t remember 7. Would you like to take part in PCI survey in the coming years? (Please put an x in the corresponding box) Yes No I am not sure