BANGLADESH TRADE FACILITATION PROJECT Baseline Evaluation July 15, 2021 – Revised August 12, 2021 BASELINE REPORT | 2 Bangladesh Trade Facilitation Project: Baseline Evaluation Program: Bangladesh Trade Facilitation (BTF) Project Agreement Number: FCC-388-2020/003-00 Funding Year: Fiscal Year 2020 Project Duration: 10/1/2020 – 9/30/2025 Implemented by: Land O’Lakes Venture37 Evaluation Authored by: NORC at the University of Chicago Dr. Clifford Zinnes Dr. Santadarshan Sadhu Anna Solovyeva Ridhi Sahai Mayumi Rezwan Letitia Onyango Gerardo E. Ayzanoa Cover photo source: Dhaka Tribune (https://www.dhakatribune.com/business/2021/07/07/exporters￾seek-government-help-in-resolving-the-container-crisis) Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPOR | i Table of Contents ABBREVIATIONS ........................................................................................................................................... iv EXECUTIVE SUMMARY .................................................................................................................................. v Project Background and Purpose ........................................................................................................... v Evaluation Questions, Design, Methods and Limitations....................................................................... v Findings and Conclusions ....................................................................................................................... v Recommendations................................................................................................................................. vi 1. INTRODUCTION & BACKGROUND............................................................................................................. 1 1.1 Project and Country Context............................................................................................................ 1 1.2 Project Description ........................................................................................................................... 1 1.3 Purpose of the Evaluation ................................................................................................................ 2 2. EVALUATION DESIGN AND METHODOLOGY............................................................................................. 4 2.1 Evaluation Questions........................................................................................................................ 4 2.2 Theory of Change ............................................................................................................................. 4 2.3 Overview of Evaluation Approach.................................................................................................... 6 3. FINDINGS: SYSTEM LEVEL ....................................................................................................................... 14 3.1 Project Strategy Alignment with USDA, GOB, Private Sector Priorities (EQ1) ............................... 14 3.2 Project Strategy Alignment with WTO TFO Standards (EQ2)......................................................... 14 3.3 Project Impact on Exports and Imports (EQ3.1)............................................................................. 15 3.4 Project Impact on Clearance Time, Inspection Rate, & Rejection Rates (EQ6).............................. 18 3.5 Likelihood of achieving Project targets and outcomes (EQ8) ........................................................ 19 4. FINDINGS: INSTITUTION LEVEL ............................................................................................................... 20 4.1 Government Organizational Capacity Changes (EQ5.1)................................................................. 20 4.2 Laboratories’ Organizational Capacity Changes (EQ5.2)................................................................ 21 5. FINDINGS: PROGRAM PARTICIPANT LEVEL............................................................................................. 27 5.1 GOB Implementation and Enforcement of Testing Procedures (SQ3)........................................... 27 5.2 Project Impact on Enterprise Business Investment (EQ3.2)........................................................... 29 6. RECOMMENDATIONS.............................................................................................................................. 31 6.1 Initial Recommendations on the Evaluation of BTF Trade Impacts ............................................... 31 6.2 Initial Recommendations on BTF Institutional Activities................................................................ 31 6.3 Initial Recommendations on BTF Effects on Beneficiaries............................................................. 31 ANNEXES ..................................................................................................................................................... 33 Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | ii Annex A. BTF Results Framework......................................................................................................... 33 Annex B. Table of key program indicators with baseline values.......................................................... 34 Annex C. Systems-Level (Chapter 3) Methodological and Statistical Details....................................... 37 C.1 List of Treated and Untreated HS Categories.......................................................................... 37 C.2 Historical Trade Trends............................................................................................................ 40 C.3 Overview of the Methodological Approach............................................................................. 42 C.4 Data Sources for the Outcome Indicators ............................................................................... 43 C.5 Overview of the Methodological Approach for ITS Potential Outcomes Approach................ 44 C.6 Statistical Matching Process.................................................................................................... 46 Annex D. Evaluation Scope of Work..................................................................................................... 49 Annex E. Evaluation Inception Report.................................................................................................. 61 Annex F. WTO February 2021 Notification Detail ................................................................................ 62 Annex G. WTO Notifications & Indicative/Definitive FTA Articles Date for Implementation .............. 64 Annex H. Government Scorecards Data Summary............................................................................... 65 Annex I. Laboratory Scorecards Data Summary................................................................................... 66 Annex J. BTF Indicators by Categories.................................................................................................. 67 List of Figures Figure 1. Causal channels of BTF impacts..................................................................................................... 6 Figure 2: Bangladesh’s value and volume of exports of priority agricultural and food products, 2016-2019 ................................................................................................................................................. 40 Figure 3: Bangladesh’s value and volume of imports of priority agricultural and food products, 2016- 2019......................................................................................................................................... 40 Figure 4: Total value (mln. USD) of Bangladesh’s agricultural and food exports, top 10 destinations, 2016- 2019......................................................................................................................................... 41 Figure 5: Bangladesh’s agricultural exports to its top trading partners, 2016-2019.................................. 41 Figure 6: Total value (mln. USD) of Bangladesh’s agricultural and food imports, top 10 origins, 2016-2019 ................................................................................................................................................. 42 Figure 7: Bangladesh’s agricultural and food imports from its top trading partners, 2016-2019.............. 42 Figure 8: Illustration of how ITSPOF depicts the effect of an intervention ................................................ 45 Figure 9: HS-level matching of exports, 2016-2020 (mean-adjusted, by treatment HS)............................ 48 Figure 10: HS-level matching of imports, 2016-2020 (mean-adjusted, by treatment HS) ......................... 48 Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | iii List of Tables Table 1. Summary of BTF Activities............................................................................................................... 2 Table 2. Key Evaluation Questions................................................................................................................ 4 Table 3. Evaluation Design Matrix, System Level.......................................................................................... 9 Table 4. Evaluation Design Matrix, Institution Level .................................................................................. 11 Table 5. Evaluation Design Matrix, Program Participant Level................................................................... 12 Table 6. Total values and volumes of Bangladesh’s trade in priority agricultural and food products, Million USD.............................................................................................................................. 16 Table 7. Time Taken to Complete Testing................................................................................................... 26 Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | iv ABBREVIATIONS ASYCUDA Automated System for Customs Data BAEC Bangladesh Atomic Entergy Commission BFSA Bangladesh Food Safety Authority BSTI Bangladesh Standards and Testing Institution BTF Bangladesh Trade Facilitation project CTA Comprehensive Trade Assessment CTV Continuous treatment-variable (model) DID Difference-in-differences (estimator) DLS Department of Livestock Services DoF Department of Fisheries FFPr Food for Progress (USDA program) FGD Focus Group Discussion GoB Government of Bangladesh ISO International Standards Organization IT Information Technology ITC International Trade Commission ITSPOF Interrupted Time-Series Potential Outcomes Framework KII Key Informant Interview LDC Less Developed Country MEL Monitoring, Evaluation and Learning NORC NORC @ the University of Chicago NEP National Enquiry Point NTFC National Trade Facilitation Committee PQW Plant Quarantine Wing RM Risk Management SPS Sanitary and Phytosanitary RCT Randomized Control Trial (experiment) SOP Standard Operating Procedure SO2 USDA FFPr’s Strategic Objective 2 TBT Technical Barriers to Trade TCC Trade Classification Code TFA Trade Facilitation Agreement USDA United States Department of Agriculture Venture37 Land O'Lakes Venture37 WTO World Trade Organization Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | v EXECUTIVE SUMMARY Project Background and Purpose Land O'Lakes Venture37 (Venture37) is implementing the Bangladesh Trade Facilitation project (BTF), funded by the United States Department of Agriculture (USDA) which will help the Government of Bangladesh (GoB) implement World Trade Organization’s (WTO) Trade Facilitation Agreement (TFA) measures through five key activities: 1. Increasing harmonization of processes, formalities, and pre-clearance processing; 2. Increasing transparency of trade information and WTO notification compliance; 3. Strengthening risk management systems; 4. Improving laboratories and testing procedures; and 5. Enhancing cold storage infrastructure for perishable goods. As the Monitoring & Evaluation partner of the BTF project, NORC at the University of Chicago (NORC) designed the evaluations of BTF in order to assess the project’s impact on improving Bangladesh’s implementation of the WTO TFA, with the purpose of the baseline evaluation to provide current context and set the pre-intervention values for key indicators of interest. These investigations will be repeated midway through the project period (the “midline”) and again at project end (the “endline”). Comparing midline and endline indicator values to those at baseline will allow NORC to test whether the institutional behavioral changes and outcomes expected along the theory of change have in fact occurred as well as to validate whether key assumptions underlying the latter are sound. Evaluation Questions, Design, Methods and Limitations NORC developed a mixed method evaluation which will utilize quantitative and qualitative methods to answer twelve key questions covering five relevant criteria: relevance, impact, effectiveness, efficiency and sustainability. The evaluation will assess the BTF project’s impact at the system level as the ultimate goal, which quantitatively entails developing a trade model to statistically detect changes in priority agri￾cultural imports and exports attributable to the BTF project. Additionally, NORC will analyze the BTF interventions’ effects on a number of intermediate sector- and institution-level outcomes by measuring performance changes in government agencies and laboratories as well as in private-sector experience and its investment. Due to the nature of the interventions implemented by the BTF project, it was not feasible to design the evaluation around a randomized control trial, and as result, NORC will utilize quasi-experimental designs where quantitative evaluation is required. Findings and Conclusions Given that the interventions have not been implemented at the baseline stage, the main objectives of the baseline stage were to assess the current status of the trade facilitation framework, establish the baseline values of key indicators of interest, document views of the stakeholders, and assess the current capacities of relevant institutions. As part of benchmarking the baseline, NORC utilized both quantitative Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | vi data gathered from secondary sources and qualitative information from Organizational Assessment Scorecards and Key Informant Interviews (KIIs) with various stakeholders. In relation to the evaluation criteria of relevance, baseline findings indicate that there is a consensus among all stakeholders that BTF objectives are clearly defined and that the activities are designed to secure the overall improvement of the country’s international trade. Additionally, the project’s overall design, activities, and approach appear well aligned with standards outlined in the WTO TFA, and our assessment indicates that all the proposed targets will likely be achieved within the planning schedule’s timeframe. In regard to measuring program impact at the system level, quantitative analysis of the trade model at baseline confirm that the Interrupted Time-Series Analysis Potential Outcomes Framework (ITSPOF) approach should allow NORC to determine the BTF project’s impact on Bangladesh’s trade with a satisfactory level of statistical confidence. At the institutional level, scorecard data indicate that strengthening of IT systems and management systems/accreditation processes are key areas of development for a significant number of organizations. On the program participant level, qualitative findings suggest that priority areas to be addressed related to GoB staff’s capacity of testing include staff training, development of testing manuals and SOPs, and development of automated and integrated systems. Finally, with regard to program effectiveness, there is a high likelihood of achieving performance indicator targets related to value and volume of trade, as well as quality improvements of traded goods, realistic target values and reasonable progression of BTF activities as per the BTF project work plan. Recommendations Given this is a baseline report, there are as yet few significant “recommendations” to make. Some preliminary recommendations are noted below: • For successful implementation of automation and a Risk Management framework, it is important to entrust NBR Customs as the lead organization to spur collaboration within several government agencies. • For fostering automation process and replacing outdated manual procedures within respective agencies, National Single Window (NSW) should include initiatives to develop “transaction modules” (e.g., electronic payment of any fee, application process, queries on application progress, communication of partial application outcomes, request/submittal of missing/incomplete information, allowing document flow along all agencies involved). • In order to speed up the certification process, innovative pilot programs can be tested to “privatize” any or all functions (e.g., inspection, quarantine, testing) related to the issuance of certificates. Similarly, to improve service delivery, private participation in other functions (such as process re-engineering and system updates for targeted agencies) need to be incentivized. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPOR | 1 1. INTRODUCTION & BACKGROUND 1.1 Project and Country Context The World Trade Organization’s (WTO) Trade Facilitation Agreement (TFA) creates standard rules that customs and other appropriate authorities must follow to be more transparent and streamline import, export, and transit processes for all WTO members. The TFA also contains special provisions for technical assistance and capacity building for developing country and less developed country (LDC) members to meet the measures for effective cooperation between customs and other appropriate authorities on customs compliance issues. Implementation of the TFA is expected to increase trade by reducing costs, rapidly releasing goods, and increasing predictability. Under Food for Progress (FFPr), the United States Department of Agriculture (USDA) assists selected developing and least-developed country (LDC) members to implement measures under the TFA for which they have requested technical assistance and capacity building. Over the last several years, with donor assistance, the Government of Bangladesh (GoB) has made sig￾nificant investments into improving the flow of goods across its borders. Policy commitments include the National Board of Revenue’s Customs Modernization Strategic Action Plan for 2019-2022, implementing the TFA, adoption of the One Stop Service Act, and improving the business enabling environment for more inclusive growth and to improve food safety and security measures. The GoB has adopted an enhanced customs website and the National Enquiry Point, implemented Advanced Ruling, and launched an Authorized Economic Operators program. The GoB will soon begin implementation on its Risk Management Decree and continue preparation for a Single Window. While there has been significant donor investment in TFA capacity building in Bangladesh, minimal work has focused on articles impacting the agriculture sector. Therefore, to contribute to the highest-level strategic objective of expanded trade of agricultural products, USDA plans to leverage current investments to ensure that agricultural trade can capitalize on the benefits they produce. The goal is to simplify, modernize, and harmonize processes for the export, import, and transit of agricultural goods through support to Bangladesh to implement the WTO TFA. 1.2 Project Description Land O'Lakes Venture37 (Venture37) is implementing the Bangladesh Trade Facilitation project (BTF), which will expand regional and international trade in agricultural goods in accordance with USDA’s Strategic Objective 2. BTF is a five-year, $27.4 million ($17.9 million operational budget) project funded by the USDA’s Food for Progress (FFPr) Program. It is slated to run from October 1, 2020 to September 30, 2025. BTF will contribute to the expanded trade of agricultural products (FFPr Strategic Objective 2) by addre￾ssing systemic constraints to agricultural trade at Bangladeshi ports, simplifying and automating import and export processes, improving the capacity of government agencies, laboratories, and warehouses, and by fostering private and public investment, especially in cold storage. These activities will help GoB fully implement WTO TFA Schedule C measures including test procedures (5.3), risk management (7.4), perishable goods (7.9) and formalities (10.1). Activities focused on automating and streamlining the Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 2 processes for import and export, instituting a risk management system, and improving handling of perishable goods at the border, including increasing capacity of cold storage will help BTF reach its goal of expanding trade in agricultural products. BTF will implement five interrelated key activities and associated sub-activities.1 These are summarized in Table 1. Table 1. Summary of BTF Activities Key Activity Sub-Activities Activity 1: Increase Harmonization of Processes, Formalities, and Pre-Clearance Processing. 1.1: Enhance legal framework to reduce requirements 1.2: Streamline processes through updated standard operating procedures (SOPs) 1.3: Support Plant Quarantine Wing (PQW) to become Plant Quarantine Authority 1.4: Expand the electronic certificate system 1.5: Engage the private sector on an e-certificate system 1.6: Improve pre-clearance processing Activity 2: Increase Transparency of Trade Information and WTO Notification Compliance. 2.1: Develop and support a coordinating body 2.2: Connect the National Enquiry Point (NEP), Sanitary and Phytosanitary (SPS) and Technical Barriers to Trade (TBT) Enquiry Points 2.3: Engage private sector on trade portals and NEP Activity 3: Strengthen Risk Management Systems 3.1: Advance legal framework for Risk Management (RM) 3.2: Create or strengthen RM units 3.3: Create Information Technology (IT) infrastructure for RM 3.4: Build capacity of GoB agencies in RM processes 3.5: Engage the private sector in RM compliance Activity 4: Improve Laboratories and Testing Procedures. 4.1: Automate laboratory reports and certificates 4.2: Improve sampling at point of entry 4.3: Build capacity of laboratories, testing facilities and certification bodies for implementing and accrediting against ISO17025 standards 4.4: Increase trust in private lab Activity 5: Enhance Cold Storage Infrastructure for Perishable Goods. 5.1: Advance legal framework for warehouse operations 5.2: Streamline warehouse processes through updated SOPs 5:3: Engage the private sector on cold storage infrastructure 5.4: Develop materials to incentivize and attract investors and operators 5.5: Identify and support potential investors and operators 5.6: Develop an operational auditing system 5.7: Build capacity of certification bodies to certify against food safety management systems Source: BTF Evaluation TOR & BTF Evaluation Plan, Venture37 documents. 1.3 Purpose of the Evaluation Venture37 contracted NORC at the University of Chicago to conduct the baseline evaluation of the BTF project. The BTF evaluation will answer twelve key questions covering five relevant criteria – relevance, 1 The descriptions that follow come from the BTF Evaluation TOR & BTF Evaluation Plan, Venture37 documents. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 3 impact, effectiveness, efficiency and sustainability. To evaluate the BTF project, NORC is employing a mixed-methods approach with quasi-experimental and non-experimental designs. The objectives of the baseline study are to: • Measure baseline values for outcome indicators in the approved performance monitoring plan (PMP); • Validate indicators and targets and provide recommendations on changes in the PMP and evaluation plan; • Review the project-level results framework and assumptions and recommend changes; and • Provide recommendations on implementation approaches to most efficiently achieve results. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 4 2. EVALUATION DESIGN AND METHODOLOGY NORC designed a mixed-methods evaluation that will draw on a combination of primary and secondary data, both quantitative and qualitative. After presenting the evaluation questions and theory of change that underpins the choices of Venture37’s intervention activities and, therefore, the evaluation, this chapter summarizes the various evaluation methodologies NORC is using to assess BTF’s impact at the system level, institutional level, and program participant level, which are first defined. The chapter ends with a detailed “evaluation matrix” mapping of the methods and data to be applied to each research question requested by USDA. 2.1 Evaluation Questions The BTF evaluation will answer twelve key questions, listed in Table 2 below. Table 2. Key Evaluation Questions CRITERIA EVALUATION QUESTIONS Relevance 1. How well aligned is the project strategy and activities with the development goals, objectives and strategies of USDA, GoB and private sector priorities? 2. How well is it aligned with the achievement of the standards outlined in the WTO TFA? Impact 3. How has the project affected imports and exports of priority agricultural products, and business investment? 4. How has support to the trade facilitation legal framework affected the progress of updating trade legislation, rules and SOPs at PQW, DLS, DoF, BSTI, BAEC and BFSA? 5. How have capacity development efforts affected the organizational capacity of PQW, DLS, DoF, BSTI, BAEC and laboratories? 6. How has the project affected the time to trade agricultural products, number of inspections required and number of rejections of Bangladeshi goods by the import country? How has the project reduced the cost to trade, number of documents required, and steps in the clearance process? Effectiveness 7. What internal and external factors have influenced the ability of the project to meet expected results and targets? 8. To what extent are the project targets and outcomes likely to be achieved by project end? 9. What key successes should be replicated, or key improvements made to the implementation to maximize the results? Efficiency 10.Were project activities carried out in a timely manner and with effective use of resources? 11.How well has the project been managed and MEL data used to make programmatic decisions? Sustainability 12.Which project activities and benefits are likely to be sustained or not past the project lifespan and why? Source: BTF Evaluation Terms of Reference (TOR) & BTF Evaluation Plan, Venture37 document 2.2 Theory of Change Following an understanding of the project activities and scope, an important initial step in the eval￾uation design process is to develop an explicit heuristic theory of change that illustrates the pathways by which the BTF program leads to changes in capacity and behavior, which in turn lead to positive out￾comes for program participants. The theory of change is intended to reflect a broad understanding of the overall program logic, which is then used to develop research hypotheses that are the subject of the evaluation. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 5 Venture37 developed a theory of change for the BTF project based on interviews with key stakeholders in agricultural trade in Bangladesh and secondary research. The results framework in Annex A provides a high-level depiction of the BTF project’s theory of change in line with the USDA FFPr’s Results Framework. Project-relevant indicators are defined to measure each result using FFPr standard and custom indicators. BTF is expected to have an impact at the system level by making trade more efficient and safe by introducing streamlined requirements and automated processes at the borders. In addition, the project is also expected to have an impact at the institutional level, increasing the capacity of laboratories and government agencies by providing capacity building and infrastructure support. It is envisioned that positive system- and institution-level impacts will lead to progress toward the FFPr’s Strategic Objective 2 (SO2) of expanded domestic, regional and international trade of agricultural products. At the foundational level, through the BTF project, government institutions, laboratories and other key organizations in the trade sector are offered the opportunity to increase and improve organizational capacity, provide agro-traders greater access to market information, leverage private sector resources, and improve the policy and regulatory framework. Five inter-related key project activities will contribute toward increasing the adoption of established industry standards, improved quality of post-production agricultural products, and improved market and trade infrastructure. This will lead to increased value being added to post-production agricultural products, increased access to markets to sell agricultural products, and improved transaction efficiency. Positive changes in these areas will likely expand regional and international trade in agricultural goods in Bangladesh. Finally, BTF will implement activities to familiarize and engage the private sector in the use of e￾certification, the trade portal, national enquiry points (NEPs), and risk management (RM). These interventions are programmed under the assumption that they will increase private-sector compliance, thereby improving export quality and reputation (a country-level externality) while at the same time lowering transaction costs for traders. This BTF results framework can be used to construct an evaluation-appropriate theory of change (caus￾ality) as shown in Figure 1. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 6 Figure 1. Causal channels of BTF impacts 2.3 Overview of Evaluation Approach NORC has selected a mixed-methods approach for this evaluation. The first stage was to conduct baseline data collection, drawing on a combination of primary and secondary data, both quantitative and qualitative, to assess the initial conditions as they pertain to later BTF evaluation stages. Its results are presented in the sections that follow. A similar approach will be used to carry out the midline and endline data collection. Then, at the midline and endline stages, NORC will evaluate BTF’s impact at the system level, institutional level, and program participant level. Improvements at the system level are the ultimate goals of the USDA and BTF. Improvements at the institutional level and program participant level are intermediate-level outcomes along the theory of change. Moreover, as it has done at baseline, at the midline and endline evaluation stages NORC will also test the soundness of the key underlying assumptions. In this way, where impacts are not detected or appear smaller than expected, NORC can provide empirical evidence to understand the reason. Likewise, as the evaluation progresses, NORC may propose to test for behavioral changes of institutional components besides laboratories or of stakeholders besides trading firms. System-Level Impacts These refer to the highest, most aggregate and macro level where the consequences of all the interven￾tion’s activities (and their negative and positive externalities) come together to manifest economy-wide effects. In terms of impacts, one expects to observe project-generated increases in exports and imports. In terms of intermediate outcomes, these should result in the achievement of the standards outlined in the WTO TFA, reduced customs clearance time, and a lower rate of rejections of exports by trading partners. BTF TA to laboratories Increased ISO 17025 testing Increase in exporter demand for lab Increase in agricul￾tural trade Increase in e￾certification & trade portals Additional ISO 17025 testing capacity Information dissem￾ination on benefits to testing Reduced compliance cost and processing time BTF TA on public dissemination of SPS benefits BTF TA to regulations and inspections More efficient and quicker customs BTF infrastruc￾ture funding Increased cold storage BTF TA to e￾certification & trade portals Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 7 The evaluation matrix in Table 3 presents a summary of the evaluation design for the system-level ana￾lyses based on our current understanding of the project. Quantitative approaches. As discussed in detail in Section 3, NORC will assess BTF’s impact at the system level in two ways. First NORC has selected a quasi-experimental design to track changes in trade flows of agricultural products. The design entails developing a trade model to statistically detect changes in targeted agricultural imports and exports attributable to BTF, while accounting for other macro, supply-side, and demand-side factors that might also affect trade movements. The impact of BTF on key system-level indicators will be analyzed through interrupted time-series analysis within a potential out￾comes framework. This entails using secondary data for the periods before and after project implementation to compare the changes in agricultural, horticultural, livestock, and fish trade flows observed at midline and endline to changes in other sectors with similar characteristics from a logistical standpoint, but whose trade flows are not expected to be impacted by the BTF Project; these other sectors serve as “synthetic” counterfactuals. Second, NORC will analyze whether, according to the theory of change, the BTF intervention leads to changes in trade through its effects on a number of intermediate sector- and institution-level outcomes. Toward this end, for each of the latter effects NORC has selected a two-step model-based approach to test both links in this causal chain. First, NORC will test the hypothesis that the BTF intervention leads to the specific intermediate changes at the sector and institution level, as described under the lower-level impacts, below. Next, NORC will use a continuous treatment-variable model to test the hypothesis (and an assumption of the theory of change) that such an intermediate change (from whatever outside cause) at the sector or institution level leads to an increase in trade flows. Qualitative approaches. NORC is also taking advantage of more qualitative information being collected from the Organizational Assessment Scorecards, GoB staff and laboratory surveys, and information gathered from focus group discussions (FGDs) and from key informant interviews (KIIs) to examine BTF trade impacts using subjective perceptions of key stakeholders from the GoB, laboratories, and the private sector. While these rich information sources will also be utilized to assess other types of impact, in the case of trade they will serve several purposes. First, in elucidating the underlying regulatory processes, they will help specify the two multivariate statistical models, above, by identifying key explanatory variables to include in the models. Second, these sources of qualitative information will help to interpret the statistical results – whether they support or reject the hypotheses – and provide guidance on quantitative results that are not statistically significant. This second use of qualitative information is especially important for generating vignettes or examples, whether for or against the statistical conclusions, since those provide the intuition necessary to interpret the conclusions, thereby increasing the latter’s credibility. Institution-Level Impacts This level refers to BTF’s effects on Bangladesh’s progress on updating its trade legislation, rules, and standard operating procedures at the various competent ministries and agencies as a result of project support to the trade facilitation legal framework. NORC will measure performance changes in government agencies and laboratories to ascertain some of BTF’s intermediate-level impacts on institutions. Examples of anticipated laboratory impacts include an increase in the diversity of tests offered, the speed at which requested tests are fulfilled, and the Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 8 increase in the number of tests conducted. In the case of the government agencies, examples of anticipated impacts include reductions in the number of documents and in the number of approval processes required to trade. The evaluation matrix in Table 4 presents a summary of preliminary eval￾uation designs for the institutional level analyses based on our current understanding of the project. A major constraint on developing a powerful evaluation design for the BTF intervention is that the evaluator will not be permitted to influence where (who) or when assistance is assigned. NORC proposes to confront this challenge statistically. In the case of laboratories, during the supplemental analysis immediately following this baseline report we will determine which of the following evaluation-design approaches to attribute performance changes are most feasible: (a) a pretest-posttest quasi-exper￾imental design with statistical matching; (b) a pretest-posttest quasi-experiment, this time using a sharp or fuzzy regression discontinuity design; and (c) a continuous treatment-variable model-based approach. On the qualitative side, NORC is drawing on the organizational assessment scorecards developed by BTF for the six main government agencies – Plant Quarantine Wing (PQW), Department of Livestock Services (DLS), Department of Fisheries (DoF), Bangladesh Standards and Testing Institution (BSTI), Bangladesh Atomic Energy Commission (BAEC) and Bangladesh Food Safety Authority (BFSA) – to track key capacities necessary to achieve project results. These were fielded by the BTF team during the initial assessment and will be fielded by NORC at midline and endline. For this baseline report NORC also reviewed qualitative data collected through KIIs and FGDs with target government agencies, NTFC members, laboratories, private sector associations, freight forwarders, traders, BTF staff and other stakeholders.2 Analogous KIIs and FGDs will be conducted at midline and endline to embellish the quantitative analyses, either illustrating or contradict the statistical findings. Program Participant-Level Impacts To assess how the project impacts are transmitted through various causal chains that are hypothesized to affect outcomes of interest, it is important to explore how the project affects the different groups of program participants. Based on a close review of the CTA inception report draft, NORC identified potential groups of program participants and users of the trade facilitation system to be reformed and upgraded and whose engagement and compliance are critical for the BTF project to achieve its trade￾growth objectives. These anticipated effects are considered intermediate outcomes, and Table 5 sum￾marizes the evaluation design for this level of impact based on our current understanding of the project. Among potential intermediate outcomes, by taking the present baseline, NORC will explore at midline and endline how the project has affected (i) the knowledge, practices, and perception of the staff of six key GoB agencies (named above), (ii) access and usage of up-to-date agricultural information available on the trade and customs portals by private sector agricultural traders, (iii) stakeholder perceptions regarding the new system in facilitating trade practices, and (iv) time and cost to trade. Therefore, in addition to the evaluation questions listed in Table 2, we will also address specific questions regarding these intermediate outcomes on program participants as described in Table 5. Based on availability of data and importance of addressing such questions, NORC will then determine the feasibility of conduct￾ing pre and-post-test analyses of the responses of such groups to project activities that target them. 2 The qualitative data were collected by the BTF team at the baseline and will be collected by NORC at the midline and final evaluation stages. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 9 Table 3. Evaluation Design Matrix, System Level Evaluation Question and Sub-Questions3 What to Analyze Data Phase Addressed* EQ1: How well aligned is the project’s stra￾tegy and activities with the development goals, objectives and strategies of USDA, GoB, and the private sector priorities? • Subjective perception of priority align￾ment • WTO TFA Category C Measures identify the GoB’s priorities related to this project. Analyze the Bangladesh TFA Category C Measures List. CTA report, Document Review B, M EQ2: How well aligned is the project’s strategy and activities with the achieve￾ment of the standards outlined in the WTO TFA? Determine the degree of compliance with WTO TFA by analyzing the World Bank assessment of Bangladesh’s TFA achievement and alignment status World Bank TFA tracking tools report B, M EQ3.1: How has the project affected imports and exports of priority agricultural products? Impact trade: Changes in exports and im￾port flows (value, volume, unit values)4 • UN Comtrade data • Secondary sector- and macro-level data • ASYCUDA World Data • EPB Data B, M, E EQ6: How has the project affected clearance time, number of inspections required, and number of rejections by the US and EU for agricultural products? Impact: • Changes in consignment clearance time • Changes in rejection rate of Bangladeshi agricultural goods by the US and EU TRS data, USDA, European Commission, inspection numbers, export rejection rates, clearing & forwarding (C&F) agents and freight forwarders, and ASYCUDA World Data S, M, E EQ7: What internal and external factors have influenced the ability of the project to meet expected results and targets? Subjective perception of enabling factors influencing ability of the project, including geopolitical factors and environmental factors (e.g., a major storm severely damages a port/labs/etc.) KIIs and perception surveys with GoB, pri￾vate sector, laboratories, trade organi￾zations E 3 NORC has developed sub-questions to detect how the BTF project affects various intermediate outcomes. 4 The impact on quantity and unit value is contingent on the underlying trade data being sufficiently reported. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 10 Evaluation Question and Sub-Questions3 What to Analyze Data Phase Addressed* EQ8: To what extent are the project targets and outcomes likely to be achieved by project end? At both baseline and midline, examine the targets and qualitatively assess the likelihood of the project in meeting targets and achieving expected outcomes by project end. KIIs with GoB, private sector, laboratories, trade organizations; Venture37 project descriptions B, M EQ9: What key successes should be replicated or key improvements made to the implementation to maximize the results? Subjective perception of success of the project NORC own expert opinion from its assessment of the intervention and interpretation of stakeholder perceptions. KIIs and perception surveys with GoB, pri￾vate sector, laboratories, trade organizations E EQ10: Were project activities carried out in a timely manner and with effective use of resources? Subjective perception of project’s imple￾mentation NORC trade expert assessment; KIIs and perception surveys with GoB, private sector, laboratories, trade organizations E EQ11: How well has the project been managed and MEL data used to make programmatic decisions? Subjective perception of project manage￾ment and use of data-driven decision making NORC expert assessment; KIIs and perception surveys with GoB, private sector, laboratories, trade organi￾zations E EQ12: Which project activities and benefits are likely to be sustained or not past the project lifespan and why? Subjective perception of sustainability of the project NORC’s expert assessment; KIIs and perception surveys with GoB, private sector, laboratories, trade organi￾zations E Sub-Question 1 (SQ1): Has the project resulted in simplified processes and reduced documentation for import and export of agricultural goods? • Time to complete approval of documentation to import and export • Number of document and approval process curtailed Survey of importers and exporters; GoB and Clearing & Forwarding (C&F) Agents and freight-forwarder administrative data; FGD of inspectors S * The column abbreviations are: Baseline (B), Supplemental Analysis (S), Midline (M), Endline (E). Note: This level refers to intermediate outcomes: GoB Staff, Private Sector. The GoB staff to interview will be selected with Venture37 recommendations. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 11 Table 4. Evaluation Design Matrix, Institution Level Evaluation Question and Sub￾Questions5 What to Analyze Data Phase Addressed* EQ4: How has support to the trade facilitation legal framework affected the progress of updating trade legislation, rules and SOPs at PQW, DLS, DoF, BSTI, BAEC and BFSA? Subjective perception of trade facilitation legal framework KIIs with PQW, DLS, DoF, BSTI, BAEC and BFSA M, E EQ5.1: How have capacity development efforts affected the organizational capacity of PQW, DLS, DoF, BSTI, BAEC and BFSA? • Changes in organizational capacity • Number of organizations with performance improvement • Subjective perception of capacity development efforts Scorecards of GOB agencies; perception surveys with GoB staff; KIIs with PQW, DLS, DoF, BSTI, BAEC and BFSA B, M, E EQ5.2: How have capacity development efforts affected the organizational capacity of laboratories? • Changes in laboratory capacity • Number of laboratories with increased performance improvement • Subjective perception of capacity development efforts • Diversity of tests offered • Time to complete requested tests Scorecards of laboratories; Surveys of laboratories; KIIs with labor￾atories B, M, E * The column abbreviations are: Baseline (B), Supplemental Analysis (S), Midline (M), Endline (E). Note: This level refers to intermediate outcomes: GoB Staff, Private Sector. The GoB staff to interview will be selected with Venture37 recommendations. 5 NORC has developed these sub-questions to detect how the BTF project affects various intermediate outcomes. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 12 Table 5. Evaluation Design Matrix, Program Participant Level Evaluation Question and Sub-Questions6 What to Analyze Data Phase Addressed* EQ3.2: How has the project affected business investment for firms engaged in international trade of priority agricultural products? Intermediate impacts: • Increase in agricultural production investment • Firm survey B, M, E • Increase in cold storage capacity • Bangladesh Cold Storage Association (BCSA) • Bangladesh Agricultural Development Corporation (BADC) SQ2: Has the project improved knowledge of sample-based testing and lab procedures by the GoB staff (inspectors)? Knowledge and perception of GoB staff (inspectors) of sample-based testing procedures. Knowledge and perception survey of GoB staff; KII of GoB inspectors M, E SQ3: Has the project led to implementation and enforcement of sample-based testing procedures by the GoB staff (inspectors)? • Knowledge and perception of GoB staff (inspectors) of sample-based testing procedures. • Analysis of inspection records Knowledge and perception survey of GoB staff; KII of GoB inspectors; admin data B, M, E SQ4: Has the project improved risk-ranking and risk￾profiling practices for inspections of agricultural products? • Knowledge and perception of GoB staff (inspectors) of sample-based testing procedures. • Analysis of inspection records Knowledge and perception survey of GoB staff; KII of GoB inspectors M, E SQ5: Has the project resulted in improved know￾ledge of GoB staff who collect and disseminate trade information within the GoB, update informa￾tion, and consistently respond to enquiries on the • Knowledge and perception of GoB staff (inspectors) about trade information, trade portals and enquiry points. • Analysis of trade portal responses • Knowledge and perception sur￾vey of GoB staff; • Trade Portal and Customs Portal data; • KIIs of GoB inspectors; M, E 6 NORC has developed these sub-questions to detect how the BTF project affects various intermediate outcomes. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 13 Evaluation Question and Sub-Questions6 What to Analyze Data Phase Addressed* country’s trade portal/ customs portal and enquiry points? • Analysis of the number of agricultural trade-related queries and responses through Customs Enquiry Points and NEPs for Trade • Customs Enquiry Point and NEP for Trade records SQ6: Has the project resulted in improved know￾ledge and access to the improved systems Knowledge and perception of importers and exporters (private sector trade and business associations) about the pre￾clearance processes. Survey of importer/exporters; KIIs of private sector trade and business associations M, E SQ7: Has the project increased usage of the cold storage for agricultural products? • Knowledge and perception of busi￾nesses and trade associations. • Analysis of cold-storage usage and throughput Survey of importer/exporters; KII of businesses and trade asso￾ciations; cold-storage records M, E * The column abbreviations are: Baseline (B), Supplemental Analysis (S), Midline (M), Endline (E). Note: This level refers to intermediate outcomes: GoB Staff, Private Sector. The GoB staff to interview will be selected with Venture37 recommendations. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 14 3. FINDINGS: SYSTEM LEVEL This section presents key findings as of the baseline stage for the EQs of Table 3 relevant to system-level impacts. 7 A mix of qualitative and quantitative data sources informed the analysis and findings in this section. UN Comtrade data for Bangladesh was the primary quantitative data source. Qualitative sources included the CTA report, World Bank TFA tracking tools report and other relevant project documents. 3.1 Project Strategy Alignment with USDA, GOB, Private Sector Priorities (EQ1) This research question asks how well aligned are the project’s strategy and activities with the develop￾ment goals, objectives and strategies of USDA, GoB, and the private sector priorities. There is a consensus among all the stakeholders (USDA, GoB, private sector) that the objectives and priorities of the project are clearly defined, and the outlined activities have been designed to secure the overall improvement of the country’s trade. Furthermore, this improvement is expected to be measurable in a reasonable time, with the year 2030 set as an indicative date for implementing the relevant WTO provisions. The project’s first diagnosis reports (Trade Ecosystem Evaluation Report and BTF Project Final Report) document and summarize the strategies and activities developed in the first phases of the project. A comprehensive set of information/data gathering and analysis tools (KIIs, Organizational Scorecards, and the Prioritization Matrices) have been used to ensure that the project’s implementation will result in gradual progress in key components: process automation, development of SOPs, designing of a risk management framework, and investment plans for cold storage and other facilities and complementary services. Other activities will support legal and procedural reform, enhance transparency, improve the GoB WTO Notification system, and training of public technical staff. A sole activity pending that is top priority is the TRS that has been rescheduled for the next months. This study will provide valuable quantitative information and document current practices in order to support setting targets/indicators and guide activities for balance of the project. 3.2 Project Strategy Alignment with WTO TFO Standards (EQ2) This research question asks how well aligned are the project’s strategy and activities with the achieve￾ment of the standards outlined in the WTO TFA. The project has designed its activities with the specific purpose of assisting in the achievement of the WTO TFA outlined provisions. With the guidance of the WTO periodic notifications that document the country’s degree of compliance, the project’s activities have been designed to contribute to obtaining timely implementation of the agreement. Complementarily, the project has identified areas of improvement for the GoB WTO notification procedures within the current obligations (see Section 3.5 of 7 System level impacts for EQ6, EQ8 and EQ12 will be provided in Supplementary Analysis Report. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 15 the Trade Ecosystem Evaluation Report); these actions will result in closer and more efficient work between the government agencies and the WTO (Enquiry Points) and facilitate periodic assessments in a timely manner. The last two WTO notifications (January 2020 and February 2021) provide a comprehensive assessment of the level of compliance for the relevant articles, as summarized in Annex F. The last compliance notice lists all the articles that under Category C, “Provisions that the Member will implement on a date after a transitional period following the entry into force of the Agreement and requiring the acquisition of assistance and support for capacity building.” All of the provisions that are part of BTF project fall under Category C, “Provisions that the Member will implement by the time the Agreement enters into force (or in the case of a least-developed country Member within one year after entry into force).” At this time, the indicative date for implementation of Articles 5.3, 7.4, 7.9 and 10.1.1(a) by the GoB is June 30, 2030. This indicative implementation date has been defined in the last year as a reasonable target due, in part, to the BTF planned initiatives. The project’s overall design, activities and approach is well aligned with the spirit and the letter of the standards outlined in the WTO TFA provisions and we estimate that the proposed targets will likely be achieved in the planned indicative schedule. 3.3 Project Impact on Exports and Imports (EQ3.1) One of the key sets of indicators of BTF impact is the degree to which the project has affected imports and exports of priority agricultural products.8 NORC will empirically examine this using an econometric approach to measure changes in imports and exports to determine if the approach would be sensitive enough to detect and attribute such a change to the USDA BTF intervention. The analysis requires a counterfactual, that is, an estimate of what the trade flows would have been without the intervention, which is unobservable, since by definition the intervention did occur. By comparing an estimate of the counterfactual at midline and again at endline to the trade flows that actually take place at those points, we can statistically test whether the intervention had an impact. In this baseline report we consider a potential outcomes approach that involves statistically matching the trade categories subject to the BTF intervention to a group of comparison trade categories not regulated by authorities engaged with BTF. The method utilizes the pre-intervention values of the latter group to construct a “synthetic” counterfactual. At endline, we will use the actual endline values of the comparison group of trade categories to compute the endline values of the synthetic counterfactual. We will then compare the endline value of this (synthetic) counterfactual to the actual observed values of the trade categories subject to the BTF intervention to statistically test whether there are changes in international agricultural trade. Before describing the methodological approach in more detail, we first discuss levels and trends in Bangladesh’s agricultural trade9 . 8 See Annex C for the methodological and statistical details, as well as the graphs referenced in this chapter. 9 By “agricultural and food trade”, we mean trade in “treated” product categories, namely the two-digit HS categories that are subject to the BTF intervention, as indicated in “The list of goods under the regulatory authority of agro trade-related Goods…”. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 16 Bangladesh’s Trade Products Subject to BTF Intervention Trends in total values and volumes. We used reversed trade data, i.e., data reported by Bangladesh’s trade partners, to examine Bangladesh’s trade in agricultural and food products over the past four years, since no recent trade data are available for Bangladesh through international databases. Table 1 details the value and volume of agricultural and food trade in 2016-2019 in constant 2010 million US dollars. Table 6. Total values and volumes of Bangladesh’s trade in priority agricultural and food products, Million USD Performance Indicator 2016 2017 2018 2019 202010 Value of agricultural imports 12,279 14,470 14,927 14,592 - Value of agricultural exports 1,629 1,624 1,462 1,573 - Volume of agricultural imports 41,028 49,111 54,555 61,603 - Volume of agricultural exports 1,618 1,637 1,584 1,721 - Source: UN Comtrade Database (https://comtrade.un.org/) Figure 1 in Annex C presents the trajectories of Bangladesh’s exports of agricultural and food products in value (constant 2010 million US dollars) and in volume (net weight, million kilograms) terms over 2016- 2019, the last four available years. As can be seen from the figure, the value of Bangladesh’s agricultural and food exports had a slight decline in the value terms from about US$1.6 billion in 2016-2017 to US$1.46 billion in 2018 and recovered to US$1.57 billion in 2019. Figure 2 in Annex C shows an increase in Bangladesh’s imports of agricultural and food products from US$12.3 billion in 2016 to US$14.5 billion in 2017 and US$14.9 billion in 2018. As with exports, the value of imports demonstrated a decline in 2019 to US$14.6 billion. 11 Top trading partners. Figure 3 in Annex C shows the total trade values of Bangladesh’s exports in the treated agricultural and food categories over 2016-2019 to the top ten destination countries. India is the top destination for the treated agricultural and food exports, accounting for about 14 percent of the total agricultural and food exports over the considered period. Turkey and China are the next two top destinations for Bangladesh’s agricultural and food exports accounting for about 10 and 9 percent, respectively. Next, we examine trends by the top trading partners in the priority agricultural and food categories over the past four years. As shown in Figure 4 in Annex C, exports to India have increased, while exports to Turkey have declined since 2017. Trade with the second-largest export partner, China, has been rather stable over the considered period. In terms of imports (Figure 5 in Annex C), China is Bangladesh’s largest trading partner, accounting for 20 percent of its total agricultural and food imports over 2016-2019. India is the second largest origin of 10 The 2020 trade statistics appears to be severely underreported by Bangladesh’s partners, possibly because of the following reasons: (i) delayed reporting due to COVID-19, and (ii) reduced economic activity by Bangladesh and its trading partners (again, due to COVID-19). At present it would not be possible to disentangle and correct for these two effects, so including 2020 values in the current version of the Baseline Report could create confusion and be distracting for the reader. We will download the revised and updated Comtrade data for 2020 at midline, which we would then use for comparison purposes as foreseen in the inception report. 11 This is likely to be due to limited data availability of trade data for 2019. Out of 137 Bangladesh’s import partners, 99 (72%) provide the 2019 data. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 17 Bangladesh’s agricultural and food imports, accounting for 18 percent of the import value. The share of the third-largest import partner, Indonesia, is much smaller, about 5% of the total import value over the considered period. India has been an increasingly important source of agricultural and food imports for Bangladesh, but its share in total imports declined in 2019 (Figure 6 in Annex C). Imports from China were rather stable in since 2017. Overview of the Methodological Approach We set up an econometric model for estimating the project’s impact on Bangladesh’s imports and exports of agricultural and food products. Towards this end, we utilize interrupted time-series analysis within a potential outcomes framework (ITSPOF), a type of quasi-experimental approach. The analysis compares changes in trade indicators of agricultural products targeted by the BTF (treatment group) against those of non-targeted products (comparison group). The ITSPOF approach identifies the treatment effect by detecting a temporal break (either a one-time shift and/or trend increase), eventually manifesting in the period following the introduction of the intervention, in the relationship between treated and comparison commodities on outcomes of interest, such as export and import values. The potential outcomes approach consists of two steps: (i) use statistical matching to create a compa￾rison group of non-targeted commodities that are statistically similar, as a group, to the group of commodity categories subject to the BTF intervention; and (ii) estimate an attribution equation to detect whether there is a statistical difference between the counterfactual and intervention groups in the period after the start-up of BTF. For this baseline report, our task is to determine the feasibility of just the first step. Implementation of the second step of estimating the attribution model must await the receipt of trade data at endline. We start the analysis process by identifying a set of agricultural and food commodities that are subject to the BTF intervention based on the information provided to us by the BTF team. Next, we construct a pool of products that are not directly affected by the BTF intervention. Then we use statistical matching to create a “synthetic” comparison group. This group, once constructed, should be statistically similar (“balanced”) on characteristics of interest to the group of treated commodities. The idea is that once BTF begins its implementation, the synthetic comparison group will continue to mirror what the treated trade categories would have experienced had BTF not existed. That way, in the attribution step we can compare that counterfactual experience to that which the intervention group actually exhibited. Any statistical differences would be “impacts” that the analysis would attribute to the existence of BTF. Figure 7 in Annex C conceptually illustrates the calculation of impact. We conduct synthetic matching over only the pre-treatment period to avoid any possibility that the later intervention could somehow affect the matching and compromise the results. With the comparison group observations thereby constructed, we plan to estimate a difference-in-differences (DID) model specification over the evaluation period. Its econometric specification and motivation are presented in Annex C. The Statistical Matching Process Step 1: Select the HS comparison categories to include. We start with a list of all the two-digit HS trade categories covering goods subject to BTF regulation. We then use all the two-digit trade categories that Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 18 are not directly affected by BTF to build a comparison group. The list of treated and comparison categories is provided in Annex C. Step 2: Run entropy matching. Using the sample of trade transactions (“observations”) for the treated and comparison HS categories from the pre-treatment period (2016 to 2020), we utilize the entropy matching approach of Hainmueller (2009) to compute a set of synthetic weights which, when applied in a linear combination with the comparison-group observations, result in a sample of “synthetic controls” which serve as counterfactuals for each transaction in the treatment (intervention) HS categories. Step 3: Test in-sample prediction error. Once the matching is done, we measure the in-sample prediction error by comparing the actual values of trade flows with those predicted by the synthetically matched comparison group. Conceptually, if the synthetically matched comparison group predicts the actual trade values observed in the sample period (2016-2020) with small errors, then we can be confident that they will also predict the no-intervention trade flows in the post intervention period that we can then use to compare the actual trade values with to estimate impacts. We computed the mean in-sample absolute prediction errors by trade category over time. These statistics allow us to determine whether certain trade categories are responsible for the significant prediction error, should we observe any. The results of these tests are uniformly excellent. The quality of the forecast errors is graphically illustrated for exports in Figure 8 in Annex C and for imports in Figure 9 in Annex C. Let us describe how to interpret these graphics as we examine their findings. These figures present the within-sample forecast errors for exports and imports, respectively, for the treatment HS trade categories.12 The graphs indicate the degree to which matching reproduces the actual historical trade flows of our baseline period. Each value displayed represents the average value of trade for the category over the years 2016-2020. We see that the five-year-average prediction error of the synthetic comparison group for most of the available HS treatment categories is approaching zero (Figure 8 in Annex C). Similarly, Figure 9 in Annex C demonstrates that most of the available HS treatment categories, the five-year￾average prediction error is below 0.001 percent. Thus, the results indicate this approach will generate a reasonable estimate of the counterfactual, and thereby will be able to estimate the impact of BTF on trade volume with a high level of confidence.13 3.4 Project Impact on Clearance Time, Inspection Rate, & Rejection Rates (EQ6) Reducing the time for international-trade consignments to pass through the “port” of entry or exit in Bangladesh is one of the key intermediate outcomes targeted by the BTF project.14 A reduction in the rate of regulatory shipment inspections from 100 percent to a lower figure in line with international best practice is a contributory factor also targeted by BTF. A final indicator of BTF success, according to the 12 The reader may note that not all the treated HS codes are depicted on Figure 8 and Figure 9. It was not possible to generate a synthetic comparison for each treated two-digit HS code because of convergence issues caused by the small sample size. Therefore, not all the treated HS codes appear in Figure 8 and Figure 9. We will re-run our estimations using the more detailed four-digit HS codes resulting in a larger sample size in the supplementary analysis. We expect to avoid the convergence issues working with the larger sample size. 13 This result also indicates that there would be no need to use the alternative CTV approach for estimating the impact. 14 See Annex C for the methodological and statistical details, as well as the graphs referenced in this chapter. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 19 theory of change, is that the rates of rejection of Bangladesh exports due to failure to meet SPS stan￾dards in the European Union and United States declines. The baseline values are presently being acquired for each of these indicators and will be reported in the Supplemental Analysis Report. NORC will compare these baseline figures to those at midline and endline to analyze whether there is statistical evidence that these rejection rates have declined over the evaluation period. 3.5 Likelihood of achieving Project targets and outcomes (EQ8) This research question considers whether the evaluators believe that BTF will achieve its project targets and outcomes. BTF has 21 indicators, which are documented in the Performance Monitoring Plan (PMP). Of those 21 indicators, 17 have near-term (Years 1 to 5) and Life-of-the-Project targets.15 Our assessment of the likelihood of achieving these targets is based on a straightforward approach: categorizing the indicators to facilitate the analysis, defining (proposing) the necessary condition(s) for achieving it, assessing the impediment (bottleneck) for meeting the condition, and suggesting a degree of likelihood for its achievement based on expert opinion and relevant benchmarks. In a broad sense, the indicators fall in three categories: (a) related to supporting public/private insti￾tutions (e.g. regulations, procedures, partnerships, increasing certificates) and improving human resources (e.g. training), (b) related to value or volume of trade, and (c) related to the quality improve￾ment of the goods traded (e.g. release of goods, rejection ratios, investment in facilities). Please refer to Annex J for the list of indicators per category. The PMP has defined 10 indicators that fall into Category (a). The targets are relatively difficult to quan￾tify or make a comparative assessment with relevant international experiences as the targets’ achieve￾ment is highly dependent on local conditions and focused interventions as those designed in the BTF project. These interventions follow extensive diagnosis and are well planned so as to consider the achievement of the proposed targets as highly likely. For Categories (b) and (c), although the achievement of the targets depends on multiple factors and action of multiple agencies, the likelihood of succeeding is “High” due to realistic target values and reasonable progression of BTF activities as per the BTF project work plan. 15 Four of the indicators are context indicators, which do not have targets. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 20 4. FINDINGS: INSTITUTION LEVEL This section details findings by EQ relevant for institution-level impacts at the baseline stage. A multitude of qualitative data sources informed the analysis and findings in this section, including relevant project documents, GOB agency and laboratory scorecards, laboratory surveys, and KIIs with GOB agency and laboratory representatives. 4.1 Government Organizational Capacity Changes (EQ5.1) Findings suggest that though capacity building efforts have helped to improve strategic approaches to capacity development, they have had limited ability to improve overall capacity. Stakeholders highlight opportunities related to capacity building, infrastructure development, and staffing. Respondents report that while several policies have been implemented to support organizational capacity improvement, their ability to implement these policies has been limited. Annex H provides an overview of nine organizational capacity characteristics of government cooperating agencies at the baseline stage, according to government scorecard data. PQW scorecards indicate average institutional capacity, with limited cold storage and limited human resources being the primary drivers. Scorecards indicate that PQW facilities are fully operational with an overall score of 2.14. This is also reflected in interviews with PQW respondents as they highlight limited changes to processes across the 30 plant quarantine stations. In their view, there are not enough staff to properly implement inspection activities, and limited training for existing staff. Respondents also note that efforts to automate processes are hampered by their inability to operate the machinery they have been provided. As such, they request additional support for training officials, C&F agents, and implementing automation measures. Respondents from the Department of Fisheries report facing similar challenges related to limited staffing and automation or digitization procedures. Despite limited implementation of e-certification and other automation measures, respondents highlight that there is high enthusiasm for the implementation of these activities. Respondents also express optimism for improving regulatory procedures as new laboratories are being built with support from the World Bank. DoF scorecards show average institutional capacity, driven by limited human resources and lack of adequate facilities, as there were no cold storage facilities at the time of assessment. Similar to DoF respondents, BFSA respondents also report challenges related to training and implementation. They note that there are several newly appointed officials who have yet to receive training. This is also reflected in the scorecards, which reveal that while the number of individuals needed to perform needed tasks is adequate (at a score of two), the staff with the skills needed to perform their tasks remains low at a score of one. Institutional capacity is also hampered by a lack of adequate facilities, where BFSA scored one, with no cold storage capacity. BSTI scorecards indicate that BSTI capacity is limited by human resources, adequate facilities, and limited cold storage capacity. BSTI scored two in each of these categories, revealing that while some structure is in place, additional resources are needed to enable BSTI to function at full capacity. BSTI interview respondents report that over the last ten years, there have been improvements in testing procedures and expansion in the types of molecules tested. BSTI respondents also report that labs are meeting current needs, and staff have received adequate training. However, there are several Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 21 opportunities for improving future processes, including expanding training and increasing staffing. BSTI respondents also highlight opportunities for improving coordination mechanisms between agencies. Although BSTI has ongoing MOUs with Customs (NSW), there are limited automated systems, and therefore no central interface to coordinate with other agencies. BAEC respondents report ongoing efforts to automate their systems. However, respondents felt that additional capacity building activities were needed to improve their overall capacity related to testing speed, coordination, storage, and centralization. This finding is reflected in BAEC scorecards, in which BAEC scored two (average) in institutional capacity. While there are no scores for cold storage facilities, scorecards indicate that human resources and facilities have some structure and component of each, but operation remains limited. Interviews with DLS respondents were incomplete as the respondents chose not to respond. However, scorecards indicate average institutional capacity, most impacted by limited human resources and a lack of adequate facilities. These findings reveal that while changes in procedures have been limited, there is a lot of enthusiasm among government agencies to undergo the training needed to improve overall organizational capacity. 4.2 Laboratories’ Organizational Capacity Changes (EQ5.2) Activity 4 under BTF involves efforts to improve the overall ability of laboratories to gain accreditation to perform a larger scope of tests. BTF will support the establishment of Standard Operating Procedures (SOPs) to ensure that personnel perform their duties in a competent and consistent manner. To support trade facilitation of agricultural products, BTF will support the development of IT systems to automate functions and to connect with Customs to transmit important lab results and other required release documentation. Specifically, Activity 4 focuses on strengthening the test capacity of government and private food testing laboratories; improving their credibility through implementation of ISO 17025 and accreditation of labs; and automation and online submission of reports and approvals that are linked with Automated System for Customs Data (ASYCUDA), to hasten the process of approvals and release agricultural goods faster. It basically addresses 4 areas of work. These include: i. Developing systems to automate lab reports and certificates and their on-line submissions and approvals thereby reducing the time to release agricultural goods. ii. Strengthening the activity of sampling at the point of customs as well as for the purpose of exports to ensure that these are appropriate and aligned with international standards. iii. Focusing on building capacity of labs in terms of testing including development of Testing manuals and SOPs, implementing ISO 17025 and supporting the AB to strengthen its accreditation scheme. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 22 iv. Strengthening private laboratories to enable these to be used for official testing purposes. This will be in terms of building the technical capacity for identified parameters, implementation of ISO 17025 to enhance their credibility, and system for developing their obligation including sharing reports and results. The targeted program participants of BTF are PQW, DLS, DoF, BAEC, BSTI, BFSA, Bangladesh Accreditation Board and selected private and public labs. To evaluate the impact of BTF interventions on the organizational capacity of laboratories, NORC will assess the following indicators at the baseline, midline and endline stages: 1. Changes in laboratory capacity in automating testing and e-certification, upgrading tools, equipment, and processes, and complying to international standards 2. Number of laboratories with increased performance improvement 3. Subjective perception of capacity development efforts 4. Diversity of tests offered 5. Time to complete requested tests Changes in Laboratory Capacity & Performance Improvement Laboratory scorecards are our key data source for assessing changes in laboratory capacity and mea￾suring performance improvement from baseline to endline. BTF technical personnel and specialist consultants developed laboratory scorecards to assess labs against six specific characteristics and relative to each other: i. Information Technology Systems, Level of Automation and Online Reporting ii. Sampling Procedures and Capabilities iii. Testing Capabilities iv. Implementation of Management Systems and Accreditation v. Recognition Agreements/ Arrangements with Countries vi. Laboratory's Institutional Capacity (Skills of Personnel, Quality of Infrastructure, Appropri￾ateness of Equipment) Annex I provides an overview of five organizational capacity characteristics of laboratories at the baseline stage, according to laboratory scorecard data. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 23 For each characteristic, labs were given a rating from 0 to 4: 0 meaning the particular element is non￾existing; 1 implying components and structures of element are in place but not operational; 2 being components and structured of element are in place but operational at a limited scale; 3 implying that all components and structures of the element are fully operational; and finally, 4 meaning all components and structures of the element are fully operational and demonstrated signs of sustainability and evolution. The scorecards were administered to 10 public and private laboratories. Overall, the BSTI had the highest average score (3.11), followed by the Waffen Private Lab (3.05) and DLS QC Lab (3.02). On the other hand, the three PQW labs had the lowest average scores, under 1, indi￾cating a need for overall strengthening across all five elements. All organizations had components and structures related to testing capabilities and institutional capacity in place i.e., they had a rating of 1 or above for these elements. However, in terms of IT systems components and structures, six out of 10 organizations had a rating of 0 or marginally more. In addition, five organizations had no implementation of management systems and accreditation processes. This indicates that strengthening of IT systems and management systems/accreditation processes are key areas of development for a significant number of organizations. NORC will use the laboratory scorecard data collected at the midline and endline to measure changes in laboratory capacity and performance improvements across the 10 organizations. Perception of Capacity Development Efforts At the baseline stage, lab surveys captured what kind of capacity development efforts organizations required in testing and not the perception about planned BTF activities. During the midline and endline stages, NORC plans to conduct KIIs that will glean insights on what progress is being made and how program participants perceive the capacity development efforts undertaken by BTF. Diversity of Testing Offered Baseline KII and lab questionnaire data capture the diversity in testing approaches employed by the different organizations. At the midline and endline, NORC will use qualitative and quantitative data to map out and capture what changes in testing procedures and processes have occurred across the organizations. BSTI. As per the Import Policy Order, all processed foodstuff/edible substances imported into Bangladesh must undergo mandatory tests of BSTI and BCSIR. However, currently there is no risk-based testing being done. BSTI performs testing for a range of parameters using international and national testing methods for which written protocols are available. For exports, at times, testing is required for issue of phytosanitary certificates. In this case, exporters bring the samples for testing to the lab locations. DOF. Import testing is limited to formalin absence and most of testing is done for export, based on importing country requirements. DOF has facilities for both chemical and microbiological testing. For chemical, LCMSMS (gold standard technology), for heavy metal ICPMS are used. In their Khulna and Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 24 Chittagong labs, Eliza kits are used for chemical screening and in case of any anomaly, the sample is sent to the Dhaka lab for confirmatory tests. In addition to their own labs, testing is also done at BARI (pesticide residues), BCSIR (mycotoxin in BCSIR) and now at DOF’s own lab which is validated for mycotoxin but not accredited. Additionally, samples of fish and fishery products for exports are tested in QC lab, Savar. Relevant DOF SOPs and test manuals have been developed. For microbiological and chemical tests, standard methods are used and where there is no standard method, the method is validated and used. For heavy metal, in-house validated methods are being used, however, these are not accredited. BAEC. All food products (including food grains, chocolate, Salmon fish, Indian sea fish, American steak, pulses, milk and milk products, edible oil, etc) entering into Bangladesh require to be tested for radioactivity levels using the Cobalt-60 and Caesium tests. However, BAEC usually does not conduct radioactivity tests of food products imported from SAARC countries16. Similar to BSTI, there is no risk￾based selectivity for testing and all consignments need to be tested. BAEC only uses its own labs and has not recognized any other facility. For exports, tests are done as per exporters request and with very few samples. KIIs reveal that BAEC test protocols are documented, personnel are well trained, and equipment are calibrated. PQW. On arrival of the container at the entry point, the import permits and Phytosanitary certificates are checked. Moreover, samples from every consignment (all countries) are visually checked by PQW officials only for infestation of pests and diseases, based on which a clearance/quarantine certificate is given. In case of additional issues, samples are further tested microscopically. Testing of food is generally done only for pests and disease, while for seeds, grain health is also checked. Different locations of the PQW have follow different testing approaches based on their capacity. For instance, the Chittagong PQW lab does not conduct all tests in their own labs, other labs like BCSIR, Food Safety Lab Mohakhali and Entomology lab of BARI are used for residue analysis and import-based tests. The HSIA PQW Station conducts a majority of test in the PQW lab. However, for some specific tests, samples are sent to one of four public and private accredited labs (WAFFEN Research Lab, BARI, NFSL, and WAFFEN QC Lab). In case of export of plants or plant products, each and every consignment has to be examined by the plant quarantine officer and phytosanitary certificates are issued in accordance with the phytosanitary requirements of the importing countries. The samples can be tested in any private or public accredited lab. In special cases, some countries request for pesticide residue levels tests, which are done at either of the four accredited labs listed above. DLS. DLS is responsible for animal feed testing for both import and exports, and for animal disease control. For food safety of imported meat (beef, chicken, etc), the responsibility lies with BFSA which is currently formulating standards on these products. Three labs are involved in testing animal products at 16 Apart from Bangladesh, SAARC member countries include Afghanistan, Bhutan, India, Maldives, Nepal, Pakistan, and Sri Lanka. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 25 the three entry/exit points for DLS: QC Lab, Savar; BLRI, Savar; and PRTC Chittagong. For exports, when any exporter intends to export, they send the sample to the lab for the required tests. When an exporter applies to DLS, then DLS either sends a field officer to collect the sample or sends a letter to their local office in that district for sample collection, which are then sent to the QC Lab. DLS use validated testing methods and have facilities for pesticide residue analysis, detecting heavy metals (arsenic, lead, mercury), aflatoxin, microbiological testing (Salmonella and E.coli), antibiotic residues detection, species analysis (beef or pork), and nutrition tests (NIR and proximate analysis). BFSA. As per the Import Policy Order, all foodstuff/edible substances imported into Bangladesh must undergo mandatory tests of BSTI and BCSIR. Therefore, this role is being carried out by them even though the mandate as per the Food Safety Act, 2013 (FSA) lies with BFSA. However, BFSA needs to establish a system and build up expertise in the area before any changes can be made. BFSA currently does not have its own labs but has designated food laboratories by department and ministries which carry out the tests required by them. They have prepared two Development Project Proposal (DPP) for setting up an accredited laboratory under its own authority, the proposals (one funded by GOB, the other by JICA) are currently in the process of review and approval. BCSIR. The main duty of BCSIR is to carry out, promote and guide scientific, industrial and technological research that optimizes the economic, environmental and social benefit for the people of Bangladesh. However, they also conduct some tests for the purpose of trade. According to the Import Policy Order, there are 55 listed tests for trade and products include milk, feeds, pulses, cereal crop, dry food, etc. In situations where the test parameter is different from BSTI specifications or BSTI cannot conduct tests, customs or BSTI send it to BCSIR. However, BCSIR only provides lab test results and is not involved in the clearance procedures. BCSIR does not have provisions for risk-based selection for testing. They test product quality and safety based on Customs’ request and provide required test results to them. For exports, if stakeholders send the sample to BCSIR, they carry out the testing according to their request and required parameters. KIIs find that there are established protocols for each BCSIR conducted test, with the methods being developed either in-house or adapted based on international methods. Personnel appear well trained and there is a testing manual developed. Waffen Research Lab (Private). The Waffen lab has state-of-the-art equipment and has the capability to test a range of food products and use a range of tests (microbial and analytical - aflatoxin, pesticide, heavy metal, antibiotic, vitamin, minerals, etc.). Tests are conducted both for products used in domestic consumption and exports. It carries out testing for various government agencies, such as BCSIR, BSTI, BFSA, PQW etc. and also for private companies that are exporting. Time to Complete Tests Baseline lab KIIs capture data on the time taken to complete testing at the different laboratories. These values will be compared with corresponding data collected at the midline and endline to assess process improvements and efficiency changes. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 26 Table 7 provides the time estimates for completing testing at the different facilities. Table 7. Time Taken to Complete Testing No. Organization Testing Time (Per KIIs) 1 BSTI Maximum of 7 days to give test reports 2 DOF Import testing is limited to formalin, for which results are given in 1 day 3 BAEC Average time to give test results is 1 to 3 days 4 PQW Test results are given in 1 day 5 DLS Maximum of 7 days are taken for the tests, but the time depends on the test to be conducted 6 BFSA NA 7 BCSIR Maximum of 7 days to give test reports 8 Waffen Research Lab (Private) Average time for results is 2 to 5 days, maximum of 6 days. (Time depends on the testing matrix, methods, and parameters) Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 27 5. FINDINGS: PROGRAM PARTICIPANT LEVEL This section details findings by EQ relevant for program participant-level impacts at the baseline stage. Findings in this section were informed by program documents, the CTA report, and Government and Laboratory KIIs and scorecard data. 5.1 GOB Implementation and Enforcement of Testing Procedures (SQ3) This sub-question asks how the project has led to implementation and enforcement of sample-based testing procedures by the GoB staff (inspectors). Priority actions under Activity 4 may increase testing and sampling capacities of GoB staff, as interviews and Scorecards indicate that government labs may benefit from support in the areas of: i) Linking staff to be trained at BAB and supporting BAB to ensure training is offered for accreditation procedures; ii) Developing testing manuals and SOPs, trainings, and online systems development; iii) Developing systems to automate online submission of reports and approvals that are linked with ASYCUDA to reducing the time to release agricultural goods; and iv) Improving the credibility of labs through implementation of ISO 17025 and accreditation Bangladesh Atomic Energy Commission (BAEC) BAEC provides elemental analysis and laboratory testing for identification of heavy metals and radiation. BAEC receives samples from Customs, and then conducts necessary analysis to share the test reports with the relevant agencies. Currently, BAEC shares its test reports via email, and data files are currently maintained manually, although an interface with ASYCUDA is being implemented to provide test results to Customs. While sample collection, inspection, and testing SOPs are in place, a risk management system is not in place and are not risk-based assessments. Finally, BAEC lacks trained lab experts, and expert availability in conducting tests is low, while lab machinery and equipment are also inadequate. Based on this context, BTF can assist the testing capacity of BAEC staff by: • Linking BAEC staff to be trained at BAB • Preparing a plan to assess and prioritize the three labs to achieve full implementation of ISO 17025 and accreditation; • Introducing risk-based inspection and controls, and strengthen online and e-systems that link to Customs as well as other departments, and consolidate test results to make the data meaning￾ful for implementing risk-based checking • Helping automate laboratory reports and certificate transfer process to the Bangladesh Customs for goods clearance. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 28 Bangladesh Standards and Testing Institution (BSTI) All processed foodstuffs imported into Bangladesh must undergo mandatory tests by BSTI, and the agency issues clearance certificates to Customs. BSTI operates three labs, including food-bacteriological division that tests about 75 products, and the chemical division which tests about 136 products. Given BSTI’s expressed interest in receiving technical support in logistics, equipment, training and human resources to develop their testing capacities, the following steps could improve testing procedures for staff: • Link BSTI staff to training • Automate laboratory reports and certificate transfer process to the Bangladesh Customs for goods clearance • Improve the sampling process of the imported product at the point of entry, including g updating SOPs • Support BSTI to conduct an evaluation of the status of their coordination with other labs involved in testing of goods within the commodities that they are responsible for testing Department of Fisheries (DoF) The role of DoF is to enforce quality control measures and issue health certificates for exportable fish and fish products and to ensure the safety of imported fish. While DoF labs have made significant progress in testing capacities, with all 3 labs being accredited and implementing ISO 17025, further accreditation is required for some Chemical and Antibiotic residues. Additionally, its data records are currently being maintained manually. In addition to similar priority actions above in providing staff with training support as well as automating laboratory report and certification processes, the capacity of DoF staff can be strengthened through developing and revising SOPs on inspection and sample collection as well as developing SOPs for lab testing. Department of Livestock Services (DLS) The DLS is authorized to conduct tests and issue certificates related to importing and exporting animal feed, animal products, animal-based food items, etc., and has established a quality control lab to ensure the consumption and trade of quality meat and meat products. A quality manual based on ISO 17025 has been prepared, and the lab has applied for ISO 17025 accreditation for 14 tests. However, DLS has gaps in its process and capabilities in terms of skilled personnel and access to testing kits. In addition to automating report and certificate processes, linking DLS staff to training, and supporting accreditation, the inspection-sampling process and PS treatment of imported products at the point of entry can be improved to increase capacity. Plant Quarantine Wing (PQW) As PQW is responsible for preventing harmful pests and diseases from entering the country along with plants, seeds, and plant-based products that are imported, PQW officers inspect import consignments for plants and plant products and issues clearance certificates. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 29 The Plant Quarantine Act (PQA) of 2011does not provide any specific provisions on the testing procedure of samples collected by the plant quarantine officer, the standard of the laboratories, or alternative process to express dissatisfaction with results. Other constraints affecting PQW’s capacity includes lack of proper financial management, quality control and assurance, diagnostic methods and protocols, sample management, and reporting and information management. Additionally, no PQW lab has gotten accreditation or are implementing ISO 17025, although they are planning to. However, the Scorecard indicates that the PQW’s overall capability and willingness to improve sets them apart from other government labs, which will be crucial in increasing their staffs’ sampling and testing capacities by: • Providing support in accreditation processes • Implementing process improvements for the inspection and sample collection processes as well as updating priority SOPs • Assisting in identifying the legislative gaps regarding second testing in a consistent manner 5.2 Project Impact on Enterprise Business Investment (EQ3.2) In order for the BTF activities hypothesized along the theory of change to increase Bangladesh’s interna￾tional trade, firms (and their supply chains) need to invest in, among other things, productive capacity, efficiency improvements, and cold storage. Hence, an important intermediate signal of BTF success is whether its assistance has stimulated business investment in these areas. There are two types of investors that are of special interest to BTF, owners of cold-storage capacity and firms engaged in inter￾national trade. Investments in cold storage There are currently at least 429 cold-storage units in Bangladesh that are a member of the Bangladesh Cold Storage Association (BCSA); in addition, there are approximately 50 other privately owned cold storages that are not a member of BCSA and 30 cold storages operated by the Bangladesh Agricultural Development Corporation (BADC). 17 NORC plans to use administrative data from these bodies to examine the changes in cold-storage capacity over the evaluation period (i.e., between baseline and endline). NORC will also conduct KIIs with members and representatives of these bodies to understand their motivations for any changes in capacity the data reveal. Enterprise investment in production capacity As part of this evaluation, NORC will conduct a web survey of a representative sample of firms in Bangladesh engaged in international trade. Among the data we hope to collect from this process are the amount the respondent-firm or its subsidiaries have invested in increasing their production, efficiency, and composition (diversity) of products they trade. If the survey pilot demonstrates the firms’ willingness to provide the information, NORC will estimate a continuous treatment-variable model in which alternative proxies of a firm’s likelihood of exposure to BTF activities will be used to detect whether the changes in the firm’s investment in production capacity, sales per employee, and product diversification over the evaluation period (i.e., between baseline and endline) are correlated. If such information is not forthcoming, then NORC will infer the motivations for the respondent-firm’s 17 LixCap Cold Storage Data Analysis_210706_144234.pdf. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 30 investments (or lack thereof) over the evaluation period using qualitative or behavioral protocols within the web survey instrument. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 31 6. RECOMMENDATIONS Given this is a baseline report, there are as yet few significant “recommendations” to make. In addition, the upcoming Supplementary Analysis Report will include additional findings. However, a few preliminary recommendations are included in this section. 6.1 Initial Recommendations on the Evaluation of BTF Trade Impacts For estimating the BTF project’s impacts on international trade flows, NORC’s inception report (see Annex E) proposed to use the ITSPOF approach (first developed to evaluate USDA’s SPS technical assistance in Egypt) as its preferred analytic evaluation strategy. The inception report also laid out a backup analytic strategy in case the preferred approach turned out to be inadequate in some unan￾ticipated way. Examining the baseline trade data prior to the intervention, NORC has been able to confirm that the ITSPOF approach in the case of Bangladesh is able to closely track trade flows. Accordingly, NORC will use the ITSPOF approach and thus the alternative backup strategy will not be necessary. 6.2 Initial Recommendations on BTF Institutional Activities Concerning opportunities to improve design and implementation features of BTF activities, NORC makes the following observations: A. Involvement of NBR Customs is considered key to lead automation of other agencies. The Score￾cards document some initiatives (BAEC Chattogram branch using an ASYCUDA World terminal, BSTI also connecting to the ASYCUDA World System, current efforts of linking Customs with PQW) that should be replicated with DLS and DoF. However, as automated information exchange is almost non-existent between Customs and DLS/DoF, a broader coordination strategy should be prioritized for prompt implementation. NBR Customs involvement and collaboration with the government agencies is also key to suc￾cessful implementation of a Risk Management framework; its technical capabilities should be leveraged to spur definition of procedures for selectivity and to automate coordination/com￾munication of selection and inspection results, especially when several agencies are involved. B. National Single Window (NSW) should include initiatives to develop "transaction modules" (e.g., electronic payment of any fee, application process, queries on application progress, communication of partial application outcomes, request/submittal of missing/incomplete information, allowing document flow along all agencies involved), targeting agencies that have an urgent need to automate outdated manual procedures. 6.3 Initial Recommendations on BTF Effects on Beneficiaries A. Consider pilot programs to "privatize" (give concessions to private service providers) any or all functions (inspection, quarantine, testing) related to the issuing certificates. For example, C&F and Live & Chilled Exporters associations have observed that lab services and cold storage need improvement and that third-party contractors should complement government services. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 32 Specialized private contractors can provide improved level of service by using up-to-date equipment and expert staff; also, private companies have the flexibility to staff properly and serve around-the-clock the immediate needs of the importers/exporters. Pilot programs should target agencies that currently have the most serious limitations on complying timely with issuing the certifications (BAEC, DLS & DOF). For example, the pre￾certification process (Authorized Economic Operator) can be expanded aggressively through public-private partnerships defining the scope (sector prioritization), implementation plans and enlisting program participants. B. As with testing, all other function should have as much private participation as possible. Outsourcing process re-engineering and systems updates across similar agencies to a single private supplier with the proper expertise can dramatically accelerate development and implementation and deployment phases if supporting hardware and staff training is also planned and properly financed. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 33 ANNEXES Annex A. BTF Results Framework Source: Venture37 BTF proposal Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 34 Annex B. Table of key program indicators with baseline values Targets Year 1 Year 2 Year 3 Year 4 Year 5 Life of Project (LOP) Target Indicator Number Performance Indicator Standard/ Custom Baseline FY21 FY22 FY23 FY24 FY25 1 Value of agricultural imports (Context) Custom 9,906 million USD NA NA NA NA NA NA 2 Value of agricultural exports (Context) Custom 1,038 million USD NA NA NA NA NA NA 3 Volume of agricultural imports (Context) Custom 31,524 million USD NA NA NA NA NA NA 4 Volume of agricultural exports (Context) Custom 910 million USD NA NA NA NA NA NA 5 Percentage change (from baseline) in value of exports and imports of targeted agricultural products as a result of USDA assistance Custom 0% 0% 0% 1% increase 3% increase 5% increase 5% increase 6 Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance Standard #21 0 20 80 400 450 30 850 7 Number of individuals participating in USDA food security programs Standard #22 0 80 150 8,500 9,500 10,100 11,000 Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 35 Targets Year 1 Year 2 Year 3 Year 4 Year 5 Life of Project (LOP) Target Indicator Number Performance Indicator Standard/ Custom Baseline FY21 FY22 FY23 FY24 FY25 8 Number of individuals benefiting indirectly from USDA-funded interventions Standard #23 0 0 0 288,000 324,000 360,000 360,000 9 Number of individuals who have applied improved management practices or technologies with USDA assistance Custom 0 0 48 280 375 638 638 10 Number of WTO Trade Facilitation Agreement provisions supported by USG assistance (EG.2.1-1) Standard (EG.2.1-1) 0 0 0 0 2 4 4 11 Percentage change (from baseline) in Bangladesh export consignments of agricultural products rejected by import countries Custom 0% 0% 0% 10% decrease 15% decrease 25% decrease 25% decrease 12 Percentage change (from baseline) in volume of exports from Bangladesh of agricultural products as a result of USDA assistance Custom 0 0% 0% 1% increase 3% increase 5% increase 5% increase 13 Average time to release agricultural products at the ports Custom TBD 0% 0% 5% decrease 10% decrease 15% decrease 15% decrease 14 Percentage of agricultural import consignments inspected/tested Custom 100% 100% 100% 75% 50% 25% 25% Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 36 Targets Year 1 Year 2 Year 3 Year 4 Year 5 Life of Project (LOP) Target Indicator Number Performance Indicator Standard/ Custom Baseline FY21 FY22 FY23 FY24 FY25 15 Total increase in installed storage capa￾city as a result of USDA assistance18 Standard #16 0 0 0 0 0 30,000 m3 30,000 m3 16 Number of firms receiving USG assistance that have obtained certification with (an) international quality control institution(s) in meeting minimum product standards (EG2.2-2) Standard (EG2.2-2) 0 0 0 3 5 6 6 17 Number of organizations with increased performance improvement with USDA assistance Standard #12 0 0 5 15 15 20 20 18 Number of policies, regulations, and/or administrative procedures in each of the following stages of development as a result of USDA assistance Standard #17 0 0 10 20 20 10 60 19 Number of certificates issued through project-supported e certificate systems Custom 0 0 0 300,000 500,000 500,000 1.3 million 20 Number of public-private partnerships formed as a result of USDA assistance Standard #13 0 0 0 1 1 0 2 21 Value of new USG commitments and new public and private sector investment leveraged by USDA to support food security and nutrition Standard #14 $0 $0 $50,000 $450,000 $5 million $7 million $12.5 million 18 As of 2021, there are 429 cold storages (of which 395 are operational) that are member of Bangladesh Cold Storage Association (BCSA) with total cold storage capacity of 5.6 million MT. In addition, there are approximately 50 privately owned cold storage facilities, with a total storage capacity of 75,000 MT and 30 cold storages operated by Bangladesh Agricultural Development Corporation (BADC), with a cumulative capacity of 47,500 metric tons. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 37 Annex C. Systems-Level (Chapter 3) Methodological and Statistical Details The below sections provide: 1) a list of treated and untreated HS categories according to “Copy of The list of goods under the regulatory authority of agro trade-related Goods…_ (002)” shared with NORC by the BTF staff; 2) an overview of the historical pattern of Bangladesh’s imports and exports of goods falling within the priority agricultural product categories; 3) a description of the trade model approaches, and 3) some suggestions on trade indicators to measure impact that Venture37 and the USDA may wish to consider. C.1 List of Treated and Untreated HS Categories HS-Category Description Treated Categories 1 Live animals 2 Meat and edible meat offal 3 Fish and crustaceans, mollusks and other aquatic invertebrates 5 Products of animal origin, not elsewhere specified 6 Live trees and other plants; 7 Edible vegetables and certain roots and tubers 8 Edible fruit and nuts; peel of citrus fruit or melons 9 Coffee, tea, mate and spices 10 Cereals 11 Products of the milling industry; malt; starches; inulin 12 Oil seeds and oleaginous fruits 15 Animal or vegetable fats and oils 19 Preparations of cereals, flour, starch or milk; bakers' wares 20 Preparations of vegetables, fruit or nuts 21 Miscellaneous edible preparations 22 Beverages, spirits and vinegar 23 Residues and waste from the food industries 25 Salt; sulfur; earths and stone; plastering materials 28 Inorganic chemicals 29 Organic chemicals 30 Pharmaceutical products 32 Tanning or dyeing extracts 35 Albuminoidal substances; modified starches; glues; enzymes 38 Miscellaneous chemical products 52 Cotton 53 Other vegetable textile fibers; paper yarn and woven fabric of paper yarn Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 38 HS-Category Description Untreated Categories 4 Dairy produce; birds eggs; natural honey 13 Lac; gums, resins and other vegetable saps and extracts 14 Vegetable plaiting materials; vegetable products nes 16 Preparations of meat, of fish or of crustaceans 17 Sugars and sugar confectionery 18 Cocoa and cocoa preparations 24 Tobacco and manufactured tobacco substitutes 26 Ores, slag and ash 27 Mineral fuels, mineral oils and products of their distillation 31 Fertilizers 33 Essential oils and resinoids; perfumery, cosmetic or toilet preparations 34 Soap, organic surface-active agents 36 Explosives; pyrotechnic products; matches 37 Photographic or cinematographic goods 39 Plastics and articles thereof 40 Rubber and articles thereof 41 Raw hides and skins (other than fur skins) and leather 42 Articles of leather; saddlery and harness 43 Fur skins and artificial fur; manufactures thereof 44 Wood and articles of wood; wood charcoal 45 Cork and articles of cork 46 Manufactures of straw, of esparto or of other plaiting materials 47 Pulp of wood or of other fibrous cellulose material 48 Paper and paperboard; articles of paper pulp, of paper or of paperboard 49 Printed books, newspapers, pictures and other products of the printing indu ... 50 Silk 51 Wool, fine or coarse animal hair; horsehair yarn and woven fabric 54 Man-made filaments 55 Man-made staple fibers 56 Wadding, felt and non-wovens; special yarns, twine, cordage, ropes and cabl ... 57 Carpets and other textile floor coverings 58 Special woven fabrics; tufted textile fabrics; lace, tapestries; trimmings; ... 59 Impregnated, coated, covered or laminated textile fabrics 60 Knitted or crocheted fabrics 61 Articles of apparel and clothing accessories, knitted or crocheted 62 Articles of apparel and clothing accessories, not knitted or crocheted 63 Other made up textile articles; sets; worn clothing and worn textile articl ... 64 Footwear, gaiters and the like; parts of such articles 65 Headgear and parts thereof Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 39 HS-Category Description 66 Umbrellas, sun umbrellas, walking sticks, seat sticks, whips, riding-crops 67 Prepared feathers and down and articles made of feathers or of down 68 Articles of stone, plaster, cement, asbestos, mica or similar materials 69 Ceramic products 70 Glass and glassware 71 Natural or cultured pearls, precious or semi-precious stones 72 Iron and steel 73 Articles of iron or steel 74 Copper and articles thereof 75 Nickel and articles thereof 76 Aluminum and articles thereof 78 Lead and articles thereof 79 Zinc and articles thereof 80 Tin and articles thereof 81 Other base metals; cermets; articles thereof 82 Tools, implements, cutlery, spoons and forks, of base metal 83 Miscellaneous articles of base metal 84 Machinery and mechanical appliances; parts thereof 85 Electrical machinery and equipment and parts thereof; sound recorders and r ... 86 Railway or tramway locomotives, rolling-stock and parts thereof 87 Vehicles other than railway or tramway rolling stock 88 Aircraft, spacecraft, and parts thereof 89 Ships, boats and floating structures 90 Optical, photographic, cinematographic, measuring, checking, precision, med ... 91 Clocks and watches and parts thereof 92 Musical instruments; parts and accessories of such articles 93 Arms and ammunition; parts and accessories thereof 94 Furniture; bedding, mattresses, cushions and similar stuffed furnishing 95 Toys, games and sports requisites; parts and accessories thereof 96 Miscellaneous manufactured articles 97 Works of art, collectors' pieces and antiques 99 Commodities not specified according to kind Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 40 C.2 Historical Trade Trends19 Figure 2: Bangladesh’s value and volume of exports of priority agricultural and food products, 2016- 2019 Source: Authors’ calculations based on UN Comtrade Database. Notes: See Annex C.1 for a list of the trade categories comprising agricultural and food products. Figure 3: Bangladesh’s value and volume of imports of priority agricultural and food products, 2016- 2019 Source: Authors’ calculations based on UN Comtrade Database. Notes: See Annex C.1 for a list of the trade categories comprising agricultural and food products. 19 The 2020 trade statistics appears to be severely underreported by Bangladesh’s partners, possibly because of the following reasons: (i) delayed reporting due to COVID-19, and (ii) reduced economic activity by Bangladesh and its trading partners (again, due to COVID-19). At present it would not be possible to disentangle and correct for these two effects, so including 2020 values in the current version of the Baseline Report could create confusion and be distracting for the reader. We will download the revised and updated Comtrade data for 2020 at midline, which we would then use for comparison purposes as foreseen in the inception report. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 41 Figure 4: Total value (mln. USD) of Bangladesh’s agricultural and food exports, top 10 destinations, 2016-2019 Source: Authors’ calculations based on UN Comtrade Database. Notes: See Annex C.1 for a list of the trade categories comprising agricultural and food products. Figure 5: Bangladesh’s agricultural exports to its top trading partners, 2016-2019 Source: Authors’ calculations based on UN Comtrade Database. Notes: See Annex C.1 for a list of the trade categories comprising agricultural and food products. 160 203 242 254 258 281 367 670 761 998 - 200 400 600 800 1,000 1,200 Pakistan Belgium Netherlands USA Saudi Arabia Germany United Kingdom China Turkey India Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 42 Figure 6: Total value (mln. USD) of Bangladesh’s agricultural and food imports, top 10 origins, 2016- 2019 Source: Authors’ calculations based on UN Comtrade Database. Notes: See Annex C.1 for a list of the trade categories comprising agricultural and food products. Figure 7: Bangladesh’s agricultural and food imports from its top trading partners, 2016-2019 Source: Authors’ calculations based on UN Comtrade Database. Notes: See Annex C.1 for a list of the trade categories comprising agricultural and food products. C.3 Overview of the Methodological Approach Identification strategy. We will conduct a quasi-experimental impact evaluation to determine whether we can detect changes in quantity and value of imports and exports. This requires comparing actual trade with its counterfactual, that is, what trade would have been without the intervention. The second 1,562 1,694 1,833 1,844 1,916 2,257 3,087 3,284 11,057 13,090 - 2,000 4,000 6,000 8,000 10,000 12,000 14,000 Thailand Australia Brazil Canada Argentina Pakistan USA Indonesia India China Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 43 approach uses synthetic matching to construct the counterfactual which is then applied within a potential outcomes framework. Broadly, in this version for the “treatment” group we select the HS categories that reflect traded goods under BTF and for the comparison group we select other HS categories with (using selection criteria described below) that are not under BTF. We then compare how the quantity and value of these two groups change over time. Precision for both approaches is limited by the number of countries and subsectors with an adequate number of transactions per year among the treated and not-treated HS categories, as well as the degree to which the demand, supply, and transportation characteristics of the counterfactual categories are similar to those of the treated categories. We assess the precision of the two approaches by comparing within-sample “forecasts” to their actual pre-intervention values in this baseline report. Treatment variable. Given that transaction-level trade data are not available, we create a single treatment variable which is equal to 1, if the import or an export was within a two-digit HS category associated with the products under the BTF mandate, and 0 otherwise. Outcomes of interest. We create variables of total value and volume (i.e., net weight) for imports and exports at the two-digit HS level for priority agricultural and food products subjected to the BTF intervention. We also create similarly variables for a group of other importing and exporting products not yet thereby subjected. Annual data is or will become available (see below) over the pre-treatment and post-treatment periods. We discuss the data sources in more detail, below. The counterfactuals. Since it was not possible to roll out the project using randomization, we could not create a control group, so alternative approaches are used to construct the counterfactual – what agricultural trade would have been like without the BTF intervention. These are described, below. C.4 Data Sources for the Outcome Indicators The main outcome in our impact attribution models is: • Value (in millions of 2010 constant US Dollars) and volume (millions of kg.) of exports. • Value (in millions of 2010 constant US Dollars) and volume (millions of kg.) of imports Outcome data comes from the United Nations International Trade Statistics Database or UN Comtrade Database (https://comtrade.un.org/data/), the largest depository of international trade data, where data on the trade values and volumes (net weight) of various commodities are available. This database, going back as far as 1962, is maintained by the United Nations Statistics Division (UNSD), to whom over 170 countries report annual international trade statistics data detailed by partner countries and commodities/service categories. The commodities/services are disaggregated up to subcategories of products based on up to six-digit codes. For the purposes of our analysis, we extracted detailed annual data on exports (imports) of agricultural and food products for the last five available years from 2016-2020 for Bangladesh’s trading partners, as the recent trade data reported by Bangladesh are not available. For the purpose of our analysis, we used the list of treated products provided by the BTF team (“Copy of The list of goods under the Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 44 regulatory authority of agro trade-related Goods…_ (002)”). We computed the trade values of various commodities in terms of 2010 constant values by adjusting for inflation. The consumer price index data for 2010 and the years 2016-2020 were used to adjust the trade values for all years in terms of the 2010 values. C.5 Overview of the Methodological Approach for ITS Potential Outcomes Approach We implement the potential outcomes approach in two steps: (i) use statistical matching to create a counterfactual group of traded commodities not subject to BTF, but track the commodity categories subject to BTF oversight; and (ii) estimate an attribution equation to detect whether there is a statistical difference between the counterfactual and intervention groups four years after the start-up of BTF. For this baseline report, the task in this section is to determine the feasibility of just the first step. Implementation of the second step of estimating the attribution model must await the receipt of trade data at endline. Creating a counterfactual. We start the matching process by identifying a set of commodity categories that are not subject to BTF intervention but have some proximate characteristics that suggest they might, when taken as a group, roughly track the commodity categories that are subject to BTF intervention. While inclusion of a commodity category in the comparison group is ultimately subjective based on expert opinion, the pool may be populated broadly (we will conduct robustness tests for results sensitivity and statistical balance tests to ensure good matches). Then we use statistical matching to create a “synthetic” comparison group. This group, once constructed, should be both statistically similar (“balanced”) on characteristics of interest as well as not be subject to BTF intervention (products must be “untreated”). The idea is that once BTF begins its program, the comparison group will continue to mirror what the treated trade categories would have experienced had BTF not existed. That way, in the attribution step we can compare that counterfactual experience to that which the intervention group actually exhibited. Any statistical differences would be “impacts” that the analysis would attribute to the existence of BTF. Figure 7 illustrates conceptually the calculation of impact. We conduct synthetic matching over only the pre-treatment period to avoid any possibility that the later intervention could somehow affect the matching and compromise the results. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 45 Figure 8: Illustration of how ITSPOF depicts the effect of an intervention Detecting an impact. With the comparison group observations thereby constructed, we plan to estimate a difference-in-differences (DID) model specification over the evaluation period, t2 through t3: 𝑋𝑋𝑡𝑡,𝑐𝑐,𝑠𝑠,𝑖𝑖 = 𝛼𝛼0 + 𝛼𝛼1𝑇𝑇𝑡𝑡 + 𝛼𝛼2𝐺𝐺𝑐𝑐,𝑠𝑠,𝑖𝑖 + 𝛼𝛼3𝑇𝑇𝑡𝑡𝐺𝐺𝑐𝑐,𝑠𝑠,𝑖𝑖 + 𝒁𝒁𝑡𝑡,𝑠𝑠 (s) 𝜶𝜶4 ′ + 𝒁𝒁(c) 𝑡𝑡,𝑐𝑐𝜶𝜶5 ′ + {𝜉𝜉𝑠𝑠,𝑐𝑐} + 𝛼𝛼6𝑡𝑡 + 𝜈𝜈𝑡𝑡,𝑠𝑠 (1) where the 𝛼𝛼𝑛𝑛 are coefficients to be estimated, when i equals 1 then 𝑋𝑋𝑡𝑡,𝑐𝑐,𝑠𝑠,𝑖𝑖denotes its actual outcome data and when i equals 0 then 𝑋𝑋𝑡𝑡,𝑠𝑠,𝑖𝑖 denotes the predicted outcome data (in fact, 𝑋𝑋� 𝑡𝑡,𝑐𝑐,𝑠𝑠,𝑖𝑖) representing the counterfactual values of 𝑋𝑋𝑡𝑡,𝑐𝑐,𝑠𝑠,𝑖𝑖 for the period 𝑡𝑡 = [𝑡𝑡0,𝑡𝑡1,𝑡𝑡2,𝑡𝑡3], 𝑇𝑇𝑡𝑡 = 0 at baseline and 1 once BTF is running, Gs is the experimental group (treatment or comparison) and equal to 1 if i is a treated observation and 0 otherwise, {𝜉𝜉𝑠𝑠} is a HS category fixed effect. 20 Last, the vectors, 𝒁𝒁𝑡𝑡,𝑠𝑠 (s) and 𝒁𝒁𝑡𝑡,𝑐𝑐 (c) capture various sources influencing intertemporal variation in the outcome indicators specific to a sector or a country.21 At the macro level, there will be exchange-rate, and interest-rate fluctuations, and political and policy shocks (e.g., strikes, changes in government). The maintained assumption is that these would affect all sectors – and, therefore, treatment and comparison firms – roughly equally. Next there may be within-food-sector sources of variation, such as weather, exogenous wage and price shocks to food￾sector inputs (domestic or imported), and, of course, regulatory (BTF) influences. Taking derivatives with respect to 𝐺𝐺𝑐𝑐,𝑠𝑠,𝑖𝑖 and 𝑇𝑇𝑡𝑡, the gap between the treatment and comparison groups over the intervention period would be calculated as 𝛼𝛼�3. If we add the term 𝛼𝛼7𝑡𝑡𝑇𝑇𝑡𝑡𝐺𝐺𝑐𝑐,𝑠𝑠,𝑖𝑖 to Equation (3) then the gap would be calculated as 𝛼𝛼�3 + 𝛼𝛼�7𝑡𝑡 and would represent the estimated impact for the respec￾tive time period. This is a simple linear trend and is independent of the HS category. Alternatively and with adequate data, one could instead add the expression 𝛼𝛼7𝑡𝑡1𝑇𝑇𝑡𝑡𝐺𝐺𝑐𝑐,𝑠𝑠,𝑖𝑖 + 𝛼𝛼8𝑡𝑡2𝑇𝑇𝑡𝑡𝐺𝐺𝑐𝑐,𝑠𝑠,𝑖𝑖 + 𝛼𝛼9𝑡𝑡3𝑇𝑇𝑡𝑡𝐺𝐺𝑐𝑐,𝑠𝑠,𝑖𝑖 to Equation 1, where 𝑡𝑡𝑡𝑡 = 1 if Period t and 0 otherwise, and test the hypotheses 𝛼𝛼�𝑗𝑗 = 𝛼𝛼�𝑘𝑘 for j,k ∈ {1,2,3}, j≠k. 20 Note that there is no 𝐺𝐺𝑡𝑡,𝑠𝑠𝑇𝑇𝑡𝑡. 21 Note that this paragraph is not meant to list every possible potential covariate, but just to provide examples. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 46 We will explore at endline the feasibility of estimating more complicated versions of the approach described above. For example, we will specify and estimate versions that will contain additional interaction terms in Equation (1) involving variables subscripted by HS to examine whether factors hypothesized in the theory of change moderate the size of the gap (impact).22 Such specification enhancements would potentially cause the size of impact to be HS-dependent. Finally, while synthetic matching as described above, guarantees perfect balance between treatment and comparison groups, we plan to follow Abadie and Gardeazabal (2003) and carry out “placebo testing” to determine whether the gaps (impacts) could have been the result of chance.23 C.6 Statistical Matching Process We utilize the entropy matching approach of Hainmueller (2009) to compute a set of synthetic weights which, when applied in a linear combination with the comparison-group observations of the purposively selected HS categories not subject to BTF, result in a sample of “synthetic controls” which serve as counterfactuals for each treatment (intervention) HS category. What the reader may find conceptually challenging in our approach is that the covariates for the matching are not country or product characteristics as might normally be expected in the case in matching. Rather the covariates correspond to different years of the trade values for the groups to be matched. The matching process requires implementing a series of steps. Step 1: Select the HS comparison categories to include. We start with a list of all the two-digit HS trade categories covering goods subject to BTF regulation. We then use all the two-digit trade categories that are not directly affected by BTF to build a comparison group. The list of treated and comparison categories is provided in Annex C. 1. Step 2: Run entropy matching. While the dataset comprises treated and comparison trade transactions (“observations”) from the pre-treatment period (2016-2020), the matching approach requires that these observations be organized (“reshaped”) in a different way. For the HS-level matching, the set-up requires a dataset with a set of six variables. One such dataset must be constructed for each of the NT two-digit HS categories, h ϵ ℋ subject to the BTF intervention. The first variable in each dataset is the treatment group, Gh, which is equal to 1 if two-digit HS category h is subject to the BTF intervention and equal to 0 otherwise. The next five variables are the “covariates” of the matching algorithm (entropy balancing).24 These are named 𝑽𝑽ℎ,16, 𝑽𝑽ℎ,17, 𝑽𝑽ℎ,18, 𝑽𝑽ℎ,19, and 𝑽𝑽ℎ,20, one for each included pre-treatment year. For intervention HS category h, the year t covariate is 22 Examples of such factors, as illustrated in Equation (1) are intensity of regulatory enforcement actions, passage of additional regulations, and proportion of food sectors brought under regulatory oversight. 23 Abadie, A., and Gardeazabal, J. (2003), “The Economic Costs of Conflict: A Case Study of the Basque Country,” American Economic Review, 93 (1), 112–132. In essence, this test reruns the analysis for all the different HS codes in the sample and then calculates whether the change detected in the SITC codes of interest could have simply been the result of chance. 24We employ the command ebalance from Stata 15 for all statistical matching in this research. It produces “perfect” matching in the sense that the results lead to identical means and standard deviations for all covariates for both groups being compared. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 47 𝑽𝑽ℎ,𝑡𝑡 = ⎣ ⎢ ⎢ ⎢ ⎡ 𝑉𝑉ℎ,𝑡𝑡 𝑉𝑉1,𝑡𝑡 𝑉𝑉2,𝑡𝑡 … 𝑉𝑉𝑁𝑁C,𝑡𝑡⎦ ⎥ ⎥ ⎥ ⎤ where 𝑉𝑉𝑠𝑠,𝑡𝑡 refers to the constant-currency export value to all partners of the sth comparison HS category in Year t and NC is the number of comparison HS categories. We then use entropy matching to conduct statistical matching using the corresponding dataset constructed, above.25 For each HS category h we match HS Category h against HS Categories 1 to NC across all included countries. This generates a unique set of synthetic weights, Ⱳh. With the set of synthetic weights, Ⱳh in hand, we can now compute the synthetic control (counter￾factual) value for any year.26 We do this by computing linear combinations of comparison units and synthetic weights to generate the synthetic control covariates with the matched values for each treatment unit. In particular: 𝑃𝑃ℎ,𝑡𝑡 = � 𝑤𝑤ℎ,𝑠𝑠𝑉𝑉𝑠𝑠,𝑡𝑡 𝑁𝑁C 𝑠𝑠≠ℎ; 𝑠𝑠=1 where 𝑃𝑃ℎ,𝑡𝑡 is the predicted value of 𝑉𝑉ℎ,𝑡𝑡. Step 3: Test the quality of the matches. A straightforward way to compare the synthetic control variables to the treatment variables to ensure good matches is to examine for a given matching approach, m the mean in-sample forecast errors, 𝜀𝜀ℎ,𝑡𝑡 (𝑚𝑚) , t ϵ [2016,2020] in percentage-difference terms, namely, 𝜀𝜀ℎ,𝑡𝑡 (𝑚𝑚) ≡ 1 𝑁𝑁ℎ,𝑡𝑡 К � �𝑉𝑉ℎ,𝑐𝑐,𝑡𝑡 − 𝑃𝑃ℎ,𝑐𝑐,𝑡𝑡 (𝑚𝑚) � /𝑉𝑉ℎ,𝑐𝑐,𝑡𝑡 𝑐𝑐 We compute 𝜀𝜀ℎ,𝑡𝑡 (𝑚𝑚) for imports and for exports. The results are uniformly excellent. Figure 8 and Figure 9 present the within-sample forecast errors for exports and imports, respectively, for the treatment HS trade categories. These graphs indicate the degree to which matching reproduces the actual historical trade flows of our baseline period, indicating a high degree of reproducibility. 25 Entropy balancing is a data preprocessing procedure that allows users to reweight a dataset such that the covariate distributions in the reweighted data satisfy a set of specified moment conditions (see Hainmueller 2012 for details). This is useful to create balanced samples in observational studies analyses such as ours with a binary treatment where the comparison group data can be reweighted to match the covariate moments in the treatment group. 26 Note that 𝑤𝑤ℎ,𝑖𝑖 are time-invariant. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 48 Figure 9: HS-level matching of exports, 2016-2020 (mean-adjusted, by treatment HS) Source: Authors’ calculations Source: UN Comtrade Database. Notes: Forecast error is the percentage difference between the actual total of partner trade with Bangladesh for the given year in an HS category regulated by BTF and the associated values constructed through entropy matching for comparison HS categories. It is measured as the mean in-sample forecast errors in percentage-difference terms. Figure 10: HS-level matching of imports, 2016-2020 (mean-adjusted, by treatment HS) Source: Authors’ calculations based on UN Comtrade Database. Notes: Forecast error is the percentage difference between the actual total of partner trade with Bangladesh for the given year in an HS category regulated by BTF and the associated values constructed through entropy matching for comparison HS categories. It is measured as the mean in-sample forecast errors in percentage-difference terms. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 49 Annex D. Evaluation Scope of Work Bangladesh Trade Facilitation Baseline, Midterm, and Final Evaluation SOW INTRODUCTION AND JUSTIFICATION This contract is to hire an external evaluation firm to conduct the baseline, midterm evaluation (MTE), and final evaluation (FE) of the Bangladesh Trade Facilitation (BTF) Project. BTF is a five-year, $17.9 million project funded by the United States Department of Agriculture (USDA) Food for Progress (FFPr) Program and implemented by Land O’Lakes Venture37 (Venture37) from October 1, 2020 – September 30, 2025. The project aims to reach its goal of expanded trade of agricultural products by automating and streamlining the processes for import and export, instituting a risk management system, and improving handling of perishable goods at the border, including increasing capacity of cold storage. The contractor’s role in the baseline will be conducted in three overlapping phases: 1.) providing input into the comprehensive trade assessment (CTA) conducted by project partner Nathan Associates (Nathan), 2.) developing the trade model, and 3) use quasi-experimental or other design methods where feasible to detect intermediate outcomes (e.g., on laboratories, private sector users, inspectors) and test key assumptions necessary for trade impacts according to the theory of change. The baseline will set the baseline values for key indicators and facilitate an understanding of the current context in which the project will be implemented, while the MTE and FE will examine the results of the project and provide recommendations for improvements to the project or future similar projects. Externally conducted baseline, MTE, and FE are required deliverables under the USDA agreement with Venture37. This scope of work describes the objectives and payment schedule for various deliverables. Venture37 must give the contractor written approval to start prior to the contractor beginning work on the midterm evaluation or the final evaluation. The inception meeting held on Dec 11, 2020 is hereby approved. PROJECT OVERVIEW BTF will reach its goal of expanded trade of agricultural products (FFPr Strategic Objective 2: Expanded Trade of Agricultural Products) by automating and streamlining the processes for import and export, instituting a risk management system, and improving handling of perishable goods at the border, including increasing capacity of cold storage. Specifically, BTF will implement five interrelated activities: Activity 1. Increase Harmonization of Processes, Formalities, and Pre-Clearance Processing. Venture37 will support the GoB in updating, drafting, and approving laws, rules, and standard operating procedures (SOPs) that govern and regulate agricultural trade and will train and facilitate government agencies to simplify processes and reduce documentation for import and export of agricultural goods. Venture37 will work directly with GoB agencies responsible for overseeing the quality and safety of agricultural Fixed Price Contract Scope of Work Bangladesh Trade Facilitation Baseline, Midterm and Final Evaluation Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 50 products to simplify, automate and harmonize certificates and pre-clearance processes. Venture37 will support the GoB to engage with relevant private sector trade and business associations about rule￾making and help importers and exporters gain knowledge and access to the improved systems. Activity 2. Increase Transparency of Trade Information and WTO Notification Compliance. Venture37 will support GoB to establish and maintain coordinating bodies and provide training and technical assistance to focal points who collect and disseminate trade information within the GoB, update information, and consistently respond to enquiries on the country’s trade portal and enquiry points. Venture37 will facilitate GoB’s increased engagement with private sector stakeholders, inviting businesses and trade associations to review and comment on WTO notices and offering training and awareness campaigns to support business to use trade portals and enquiry points. Specifically, BTF’s Public and Private Sector Engagement Advisor will support the various GoB agencies to engage private sector associations and chambers and to implement a program of trainings or awareness seminars. Activity 3. Strengthen Risk Management Systems. Venture37 will strengthen risk management (RM) systems by upgrading and connecting IT systems, providing IT hardware when necessary, and by building the capacity of GoB agencies and staff to implement and enforce new sample-based testing procedures and by coordinating training in food safety, RM, and surveillance techniques. Venture37 will complement practical curriculum-based training by offering a study tour for GoB officials to examine and observe how other nations apply risk-profiling and use visual inspections and sampling tools. Venture37 will work with GoB agencies to update RM-related laws, rules, or administrative procedures introducing risk ranking and risk profiling practices for inspections of agricultural products. Venture37 will facilitate GoB to solicit input and inform businesses and trade associations about how to comply with updated RM protocols. Activity 4. Improve Laboratories and Testing Procedures. Venture37 will increase the capacity of government and private food and product testing laboratories to comply to international standards by updating SOPs, creating testing and e-certification that are linked with Automated System for Customs Data (ASYCUDA), and upgrading tools, equipment, and processes. Venture37 will provide IT expert services to develop and install software and hardware for an information system to automate submission of lab reports and test certificates, allowing GoB agencies to share information and access reports quickly. Venture37 will implement a training-of-trainers program, training each agency how to implement, operate, and maintain the system. Venture37 will support GoB to implement an accreditation system for private laboratories that sets sampling procedures that are harmonized across the GoB’s RM system and that meet ISO 17025 standards and the requirements of international standard-setting bodies, including International Plant Protection Convention, World Organisation for Animal Health and Codex Alimentarius. Venture37 will support the GoB and other donors to develop SOPs for testing priority agricultural products. Venture37 training interventions will build the capacity of labs to implement on-going staff training in sampling, testing, and lab operations. Venture37 will provide in-kind grants to GoB and private laboratories to upgrade testing and sampling equipment such as autoclaves, centrifuges, pH meters, and microscopes. Grants will require cost-sharing and will be awarded based on recommendations of an operational assessment of the facility and will require the lab’s participation in trainings and introduction of equipment maintenance plans. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 51 Activity 5. Enhance Cold Storage Infrastructure for Perishable Goods. Venture37 will promote, facilitate, and incentivize investment in the cold chain for agricultural products necessary for maintaining the safety and quality of high-value perishable goods. In collaboration with both GoB and industry stakeholders, Venture37 will develop and promote an investment plan to finance, build, and operate large-scale cold storage facilities at key air and sea ports. Venture37 will use subgrants from the project and/or facilitate public-private co-financing when supported by the results of an investment option analysis of financial returns and market conditions. Venture37 will support potential investors that have cleared a screening to conduct feasibility studies and to negotiate and secure financing and then will provide on-going design and engineering advice for the construction of the facility. Venture37 will provide technical advice to National Trade Facilitation Committee (NTFC) Working Group and GoB authorities and facilitate input from industry associations to update laws and rules relevant to ownership and operating standards and to implement an auditing system for refrigerated warehouses. Venture37 will guide GoB agencies to implement training programs and to draft policies and SOPs, audit requirements, and accreditation standards. Venture37 will provide training to GoB and private organizations that advise or certify cold storage facilities to implement food safety and management and quality control systems. EVALUATION DESIGN Purpose and Objectives of the Evaluation The purpose of the baseline, MTE, and FE is to analyze and document the extent to which the program has achieved its goals and objectives and to explain any deviations from the plan. Specific objectives for each evaluation are listed below: Baseline Objectives The baseline will: • Measure baseline values for outcome indicators in the approved performance monitoring plan (PMP); • Validate indicators and targets and provide recommendations on changes in the PMP and evaluation plan; • Review the project-level results framework and assumptions and recommend changes; and • Provide recommendations on implementation approaches to most efficiently achieve results. Midterm Evaluation Objectives The midterm evaluation will: • Measure the short-term progress made toward achieving key indicators, assessing attribution to the extent possible and explaining deviations; • Assess the relevance, effectiveness, efficiency, impact and sustainability of project activities; • Document lessons learned, challenges and unanticipated effects; • Identify enablers and constraints to progress that have supported or limited success; • Provide recommendations for necessary changes to strengthen project performance, efficiency and sustainability; and • Provide recommendations for areas of focus of the final evaluation, including reviewing and strengthening data collection systems and metrics in preparation for the final evaluation. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 52 Final Evaluation Objectives The final evaluation will: • Measure the longer-term progress made toward achieving key indicators, assessing attribution and explaining deviations from the plan; • Assess the relevance, effectiveness, efficiency, impact and sustainability of project activities; • Identify lessons learned through the duration of the project with an emphasis on identifying key strategies, methodologies and factors that contributed to and/or inhibited success; and • Draw conclusions and provide recommendations to Venture37, USDA, project participants, project partners and other stakeholders to strengthen future or similar projects. Preliminary Key Evaluation Questions The evaluation will answer the following key questions, drawing on an array of quantitative and qualitative evaluation techniques, depending on data availability. Table 1. Key Evaluation Questions. CRITERIA EVALUATION QUESTIONS Relevance • How well aligned is the project strategy and activities with the development goals, objectives and strategies of USDA, GoB and private sector priorities? How well is it aligned with the achievement of the standards outlined in the WTO TFA? Impact • How has the project affected imports and exports of priority agricultural products, and business investment? • How has support to the trade facilitation legal framework affected the progress of updating trade legislation, rules and SOPs at PQW, DLS, DoF, BSTI, BAEC and BFSA? • How have capacity development efforts affected the organizational capacity of PQW, DLS, DoF, BSTI, BAEC and laboratories? • How has the project affected the time to trade agricultural products, number of inspections required and number of rejections of Bangladeshi goods by the import country? Effectiveness • What internal and external factors have influenced the ability of the project to meet expected results and targets? • To what extent are the project targets and outcomes likely to be achieved by project end? • What key successes should be replicated, or key improvements made to the implementation to maximize the results? Efficiency • Were project activities carried out in a timely manner and with effective use of resources? • How well has the project been managed and MEL data used to make programmatic decisions? Sustainability • Which project benefits are likely to be sustained or not past the project lifespan and why? Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 53 SCOPE OF WORK The contractor will conduct a mixed-methods evaluation that will use a mixture of primary and secondary data. The project is expected to have an impact at the system level by making trade more efficient and safe in Bangladesh through more streamlined requirements and automated processes at the border. This system level impact will be evaluated through a trade model. The project is also expected to have an impact at the institutional level, increased the capacity of laboratories and government agencies through direct capacity building and infrastructure support. The institutional level impact will be evaluated through measuring performance changes in government agencies and laboratories. The contractor will also collect qualitative data through key informant interviews and focus group discussions with target government agencies, laboratories, private sector associations, project staff and other stakeholders to receive feedback on project implementation, challenges, lessons learned and where improvements can be made. The evaluation will also be informed by monitoring data provided by BTF on capacity building provided by the project; progress on laws, rules and SOPs; and regular reports from government agencies and laboratories. More detail on the system level and institutional level impact evaluation approaches is provided below. System Level. The evaluation of the project’s impact at the system level will use a quasi-experimental design to track changes in trade flows of agricultural products. The contractor will develop a trade model to look at trade outcomes for targeted agricultural products, accounting for other variables that affect trade27 and compare it to changes in other sectors that are not impacted by the project, which will serve as a synthetic counterfactual. Import and export volume and value of sales, type of products, export rejections, and factors that affect trade will come from secondary sources.28 The impact of the project on key system level indicators will be analyzed through an interrupted time series technique. Using secondary data for key agricultural product categories before and after the implementation, the contractor will compare changes in key outcomes for agricultural categories to outcomes in sectors that would not have been impacted by the project implementation, but that have similar capital and labor intensities. These other sectors will be used as the comparison group to form the quasi-experimental design. Key system level indicators that the contractor may include in their trade model include the following: • Import and export volume, by agricultural product type • Import and export value, by agricultural product type • Secular changes in the regional sources of imports and regional destinations of exports, by agricultural product type • Possibly, changes in the composition of imports and exports within the targeted agricultural sectors • Possibly, changes in the unit values (or other price proxy) of exports, by agricultural product type Since BTF may take some time to impact the higher-level trade outcomes, the contractor will also look at some intermediate sector-level outcomes that suggests “movement toward trade” as they analyze the 27 Other variables may include macro factors (inflation rates, exchange rate, interest rate, unemployment rates, political and policy changes in Bangladesh or its trading partners), supply-side factors (weather, wage, price) and demand-side factors (commodity prices, trading partner per￾capita income growth, weather, exchange rate ratios, other products imported by trade partners). 28 International Trade Centre, World Bank Doing Business Survey, COMSTAT Data Hub and the United Nations Industrial Development Organization are a few sources, and NORC will search for other sources. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 54 model. These institutional-level indicators refer to metrics of the performance of relevant government regulatory agencies, laboratories and sector trade associations. The list of such indicators will be finalized at baseline; they would include direct and indirect measures. Examples of direct performance would be the number, turn-around time, and variety of lab tests, efficiency of the inspection process, and port clearance times. Examples of indirect performance might be the self-reported impacts on exporter and importer costs incurred from utilizing and being subject to the various layers of inspection and regulatory compliance. It is expected that the contractor will use interrupted time-series techniques only for trade impact analysis. Specific intermediate measures may include: • Share of sectoral output that is exported by the agricultural product sector and for other exporting sectors with similar capital and labor intensities • Share of agricultural sector production of higher-value products and share of exports of these products • Number of firms exporting agricultural products • Employment growth in agricultural sector and in other exporting sectors with similar capital and labor intensities • Value and volume of critical imported inputs in the production of agricultural processing for export • Number of days to import or export agricultural products Institutional Level. The evaluation will assess BTF’s impact at the institutional level through measuring performance changes in government agencies and laboratories. For the GoB agencies, the contractor will use a non-experimental design. Since the government entities are unique within Bangladesh, there is no viable counterfactual with which to compare. To measure changes in capacity, NORC will work with Nathan during the initial assessment to develop organizational assessment scorecards for each of the six main government agencies that will track key capacities necessary to achieve project results. For the laboratories, the contractor may use a pretest-posttest quasi-experimental design or another design suggested by the contractor. The contractor will work with Nathan at baseline to develop an organizational assessment scorecard that lays out clear criteria that a laboratory must have to provide proper testing for agricultural products. The counterfactual could be created through 1) varying when the laboratories receive support; or 2) using a regression discontinuity, where a list of laboratories will be rated for involvement in the project, and those that just missed the cutoff will be the comparison group. The contractor may also evaluate some intermediate-level outcomes along the theory of change as well as test some key underlying assumptions so that, where impacts are not detected (or appear smaller than expected), the contractor could provide the empirical evidence to understand why this was. To achieve this, the contractor may suggest other components of the system besides laboratories to conduct an additional quasi-experimental assessment and/or the contractor may propose examining specific stakeholders as well. These additions to this SOW would be finalized during the second deliverable (in the payment schedule table). For example, the contractor could examine the following: • user (e.g., importer and exporter agents and freight forwarder) knowledge and taking account of the new systems, procedures, and regulations put into place with project assistance; and Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 55 • knowledge and experience of inspectors and port agents regarding the use of the new procedures and regulations put into place with project assistance. The specific tasks that the contractor will carry out are detailed below: Development of Evaluation Methodology: Support Venture37 to design an appropriate evaluation methodology for the BTF project, particularly related to the quasi-experimental design for assessing the laboratories, which will include participation in brainstorming phone calls, providing written input, and giving feedback to the draft evaluation plan. Meeting with USDA: Conduct an introductory in-person or virtual meeting with USDA prior to conducting fieldwork for the baseline, MTE, and FE. If requested by USDA, conduct an in-person or virtual meeting with USDA after the draft report is completed to share key findings from the baseline, MTE, and/or FE. Review of Documents: Undertake a review of the BTF program documents and other relevant documents that are available at the time, including, but not limited to, the following: • Project agreement with USDA • BTF Evaluation Plan • BTF Performance Management Plan • BTF Internal Monitoring, Evaluation and Learning Plan • BTF Workplan • Inception Report for Comprehensive Trade Assessment (CTA) • Preliminary CTA tools • Comprehensive Trade Assessment (CTA) (when available) • Time Release Study (at Midterm and Final Evaluation) • Monitoring data (at Midterm and Final Evaluation) • Baseline report & data collection tools (at Midterm and Final Evaluation) • Midterm report & data collection tools (at Final Evaluation) • Semi-annual reports submitted by Venture37 to USDA (at Midterm and Final Evaluation) • Any other relevant program documents • Relevant Government of Bangladesh reports and documents • Secondary data relevant to trade model and food industry performance in Bangladesh, such as data from COMSTAT, the International Trade Centre, and Food and Drug Administration and E.U. import refusal databases Develop methodology and trade model and provide input into CTA data collection tools (Baseline): The contractor, in close collaboration with Venture37, will do the following at Baseline: • Provide input into the CTA, led by project partner Nathan Associates, specifically noting questions that the CTA will need to ask to ensure the correct data is collected for the baseline; • Develop/revise a methodology to carry throughout the three evaluations, including trade model(s), sample frame, sample technique and sizes for qualitative primary data collection that is consistent with the desired methodology in this SOW; • Identify sources of secondary data to inform a trade model for international trade; • Develop data collection tools for qualitative data collection during the CTA, and review and provide feedback on the integration of those tools into the CTA by Nathan; Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 56 • Review and provide feedback on the organizational scorecards for the government agencies and laboratories developed by Nathan; • Based upon a reading of the program documents, propose any additional topics or issues for analysis prior to conducting each evaluation; and • Develop and submit an inception report that includes all of the above for review and approval by Venture37. Conduct web-surveys and develop a supplemental report: The contractor will conduct web-surveys with the actors listed below and develop a supplemental report with the results of those surveys, and incorporate any additional data and analysis as appropriate after the baseline report has been completed. Venture37 will review and approve the web-survey tools, content of the questions, and the specific participants targeted by the survey prior to implementation of the web-survey. • GoB staff survey: perception survey of GoB staff (maximum 10 minutes of length covering approximately 10 questions related to perception of challenges and opportunities of trade). • Private sector firm survey: perception and experience survey of importers/exporters and forwarding agents (maximum 15 minutes of length covering approximately 20 questions) for capturing their familiarity with the current regulatory requirements (including documentation and pre-clearance process) and of SPS compliance (testing) as well as information on rejections (rate and reason). Review of Year 1 and Year 3 Progress (between MTE and FE): The contractor will do the following between the MTE and FE: • Review progress materials provided by Venture37. • Attend progress meetings with Venture37 and others, such as Nathan, as appropriate. Refinement of methodology and data collection tools (MTE and FE): The contractor, in close collaboration with Venture37, will do the following at MTE and FE: • Revise the methodology developed at baseline, as needed, including revising the trade model(s), sample frame, sample technique and sizes for qualitative primary data collection that is consistent with the desired methodology in this SOW; • Identify additional sources of secondary data, as appropriate, to inform the trade model for international trade; • Revise data collection tools for qualitative data collection. Additional questions should be added or new tools created at MTE/FE to explore participation in project-supported activities, and to ensure all evaluation questions described in this SOW are answered; • Based upon a reading of the program documents, propose any additional topics or issues for analysis prior to conducting each evaluation; and • Develop and submit an inception report that includes all of the above for review and approval by Venture37. Data entry, analysis and reporting: • Synthesize, analyze, and interpret qualitative data at Baseline and enter, clean, synthesize, analyze, and interpret qualitative data at MTE and FE using approved qualitative data analysis packages, as needed; analyze and interpret secondary data and monitoring data; • Prepare a draft evaluation report that addresses the objectives and questions of the evaluation outlined in this SOW and makes recommendations for the BTF project or for potential similar future projects for review by Venture37 staff and stakeholders; Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 57 • Develop a PowerPoint presentation of evaluation findings and present and submit to Venture37 and stakeholders, as arranged by Venture37; and • Prepare a final evaluation report that includes revisions based on feedback on the draft report and presentation. Timeframe The timeline for the baseline is detailed below. The anticipated timeline for the midterm is March – July 2023, and the final evaluation is May – September 2025. Detailed timelines for the midterm and final evaluation will be included as an amendment to this contract. # BASELINE EVALUATION ACTIVITY RESPONSIBILITY DATES 1 Review of relevant documents, identify sources of secondary data, prepare preliminary methodology for discussion, including options for quasi-experimental design with the laboratories Evaluator Nov. 20-30, 2020 2 Inception meeting with Venture37 to discuss protocol, methodology, sampling, tools, and timeline, and to make a decision regarding selected methodology Evaluator and Venture37 Dec. 1, 2020 3 Read CTA SOW and discuss with Nathan and LixCap, as needed, to ensure understanding of scope of CTA Evaluator Nov. 30 – Dec. 4, 2020 4 Develop an inception report and preliminary trade model in line with selected methodology Evaluator Dec. 1, 2020 – Jan. 15, 2021 5 Inception report due to Venture37 Evaluator Jan. 15, 2021 6 Venture37 reviews inception report and provides feedback, comments, and suggestions to evaluator Venture37 Jan. 18-22, 2021 7 Develop questions for the CTA, specifying information needed from the different stakeholders during the CTA and other data collection needs, if applicable Evaluator Jan. 18 – Feb. 5, 2021 8 Finalize trade model plans based on Venture37 feedback Evaluator Jan. 25-29, 2021 9 Finalized trade model plans due to Venture37 Evaluator Jan. 29, 2021 10 Draft CTA questions due to Venture37 Evaluator Feb. 5, 2020 11 Venture37 reviews CTA questions and provides feedback, comments, and suggestions to evaluator Venture37 Feb. 8-12, 2021 Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 58 # BASELINE EVALUATION ACTIVITY RESPONSIBILITY DATES 12 Finalize CTA questions based on Venture37 feedback Evaluator Feb. 15-19, 2021 13 Finalized CTA questions due to Venture37 Evaluator Feb. 19, 2021 14 Develop trade model on secondary data Evaluator Feb. 2-26, 2021 15 Analysis of qualitative data from CTA and report writing Evaluator Mar. 10-31, 2021 16 Baseline report outline is submitted to Venture37 Evaluator Jul. 2, 2021 17 Venture37 reviews draft Baseline report outline and provides evaluator with comments and suggestions for revisions for Baseline report outline Venture37 Jul. 3-8, 2021 18 Draft Baseline report is submitted to Venture37 Evaluator Jul. 2, 2021 19 Venture37 reviews draft Baseline report and provides evaluator with comments and suggestions for revisions for Baseline report Venture37 Jul. 3-8, 2021 Final Baseline report is submitted to Venture37 Evaluator Jul. 13, 2021 Venture37 reviews final report for submission￾readiness and provides evaluator with any final comments and requests for revisions Venture37 Jul. 13-14, 2021 20 If necessary, respond to final comments on the Baseline report based on Venture37 feedback and prepare all deliverables Evaluator Jul. 15, 2021 by noon EST 21 Presentation of evaluation findings to Venture37 Evaluator Jul. 20, 2021 23 Draft Web-survey tools submitted to Venture37 Evaluator Jul. 23, 2021 24 Venture37 reviews draft web-surveys and provides evaluator with comments and suggestions for revisions Venture37 Jul. 23-29, 2021 22 All Final Deliverables Due (Final Baseline report, non￾technical summary, clean data, and PPT presentation) Evaluator Jul. 28, 2021 25 Draft Supplemental report submitted to Venture37 Evaluator Aug. 30, 2021 26 Venture37 reviews draft supplemental report and provides evaluator with comments and suggestions for revisions Venture37 Aug. 31-Sep.7, 2021 Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 59 # BASELINE EVALUATION ACTIVITY RESPONSIBILITY DATES 27 Finalize Supplemental report based on Venture37 feedback Evaluator Sep. 8-15, 2021 28 Final Supplemental report submitted to Venture37 Evaluator Sep. 15, 2021 Required Deliverables The deliverables under this contract for each of the baseline, MTE, and FE are listed below. Due dates are provided for the baseline. Due dates for the MTE and FE will be included as an amendment to this contract. DELIVERABLE DESCRIPTION DUE DATE Inception Report The Inception Report should describe the following: • Understanding of the project based on project documents and literature review • Finalized methodology, including trade model • Communication protocol • Finalized timeline Jan 15, 2021 Input into CTA Tools Provide draft CTA questions that includes information needed from the different stakeholders during the CTA and other data collection needs, if applicable Jun 30, 2021 Draft Outline Draft detailed outline that provides a breakdown of how the results and findings will be reported. Report should include at least following sections: • Acknowledgements • List of Acronyms and abbreviations • Table of Contents • Executive Summary • Background of Project and Evaluation • Methodology and Implementation • Results and Findings • Recommendations • Annex: Table of key program indicators with baseline values • Annex: Results Frameworks • Annex: Conflict of Interest Statement • Annex: Scope of Work for the evaluation • Annex: Inception Report for the evaluation • Annex: Final Survey Instruments: questionnaire(s), survey(s), interview protocol(s), focus group discussion protocol(s) Jul. 2, 2021 Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 60 DELIVERABLE DESCRIPTION DUE DATE Draft baseline report The report should be submitted in English addressing all the evaluation objectives and questions listed in the scope of work. Jul. 2, 2021 Final version of the baseline report Electronic copy of the final evaluation report should be submitted in English in both Microsoft Word and PDF. The final version should be free of personally identifiable information, and use the USDA cover and title page template. Jul. 13, 2021 Presentation on Findings Presentation should include an abbreviated list of the baseline findings that can be presented to relevant internal and external stakeholders Jul. 20, 2021 Web-survey tools Web-survey tools for GoB staff and Private Sector Firms Jul. 23, 2021 Non-technical summary of evaluation findings A short (2-4 pages) non-technical summary of the evaluation findings, easily understood by non-experts, and include graphics such as infographics, charts and/or graphs. Jul. 28, 2021 Final Data Collection Tools Electronic copies of all clean and final English-version of data collection tools Jul. 28, 2021 Final Cleaned Data Clean and final English versions of: • Quantitative data sets in approved format (Excel, Stata, SPSS) • Secondary data sets • Qualitative transcripts, field and interview notes, complete list of key informant interviews and FGDs in Microsoft Word Jul. 28, 2021 Presentation on Findings Presentation should include an abbreviated list of the baseline findings that can be presented to relevant internal and external stakeholders Jul. 28, 2021 Draft Supplemental report The supplemental report should be in English addressing all the objectives listed in the scope of work. Aug. 30, 2021 Final Supplemental report The supplemental report should be in English addressing all the objectives listed in the scope of work. Sep. 15, 2021 Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 61 Annex E. Evaluation Inception Report Microsoft Word Document . Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 62 Annex F. WTO February 2021 Notification Detail Provision Heading/Description Category Indicative date for implementation (for categories B and C) Definitive date for implementation (for categories B and C) Assistance and Support for Capacity Building Required for Implementation (for category C) Article 5 Other Measures to Enhance Impartiality, Non-Discrimination and Transparency Article 5.3 Test Procedures C 30 June 2030 To be determined The Department of Plant Quarantine, Bangladesh Standards & Testing Institute, Bangladesh Atomic Energy Commission, Department of livestock, Department of Fisheries do not have sufficient laboratories for 2nd tests. It has now become a challenge for them. Technical assistance is required to establish testing laboratories and other logistics and also capacity building for Test Procedures. Development partners' assistance is required in the following areas: 1. Equipment 2. Calibration 3. New lab 4. Training and capacity building Article 7 Release and Clearance of Goods Article 7.4 Risk Management C 30 June 2030 To be determined The National Board of Revenue is currently working to implement this measure with technical assistance from the USAID and the World Bank Group (WBG). Assistance required: 1. Support for legislation 2. Consultation 3. Training 4. Capacity development 5. Need Assessment The Plant Quarantine Wing has previously received training from the WBG. It requires assistance from development partners for: 1. Capacity development of surveillance mechanism 2. Establishment of pest Risk Analysis (PRA) procedure 3. Technical support on risk assessment and mitigation 4. Documentary and awareness development program in social media for moral and ethical responsibilities 5. preparation of some SOPs and manuals for risk management issues The Bangladesh Standards and Testing Institute (BSTI) is working to implement this measure with technical assistance from the World Bank Group. It requires assistance for: 1. Stabilization of the risk management system following international best practices. Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 63 Provision Heading/Description Category Indicative date for implementation (for categories B and C) Definitive date for implementation (for categories B and C) Assistance and Support for Capacity Building Required for Implementation (for category C) 2. Training for the Risk Management Committee members. 3. On-site visit for Risk Management Committee that have successfully established the system. No development partner is currently working with Bangladesh Atomic Energy Commission in this regard. It requires assistance from development partners in the area of risk management. For Developing laboratory facilities, accredited process is going on. The Plant Quarantine Wing of DAE needs official assistance. The automation process of import and export permits has already been started. Article 7.9 Perishable Goods C 30 June 2030 To be determined No development partner has so far offered any assistance in the area of perishable goods. Assistance required: 1. Support for legislation 2. Consultation with stakeholders through workshops, seminars and round table meetings 3. Training 4. Capacity development 5. Infrastructure Support Again, Port authorities require both technical (feasibility study, drawing, design etc.) and financial (infrastructure, equipment, maintenance etc.) assistance. Article 10 Formalities Connected with Importation, Exportation and Transit Article 10.1.1(a) Formalities and Documentation Requirements C 30 June 2030 To be determined No development partner has so far offered any assistance in the area of perishable goods. Assistance required: 1. Training 2. Capacity development Again, Plant Quarantine Wing requires assistance for- (a) the preparation of a valid organigram of a Plant Quarantine Authority in Bangladesh, and (b) the establishment of E-Phyto System. Article 10.1.1(b)- 10.1.2 Formalities and Documentation Requirements A - - - Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 64 Annex G. WTO Notifications & Indicative/Definitive FTA Articles Date for Implementation 2/22/2021 notification 2/22/2021 notification 1/15/2021 notification Provision Heading/description Category Indicative date for implementation Definitive date for implementation Indicative date for implementation Article 1.3 Enquiry Points C June 30, 2026 To be determined Article 1.4(c) Notification C June 30, 2026 To be determined Article 5.3 Test Procedures C June 30, 2030 To be determined To be determined Article 7.1 Pre-arrival Processing C June 30, 2025 To be determined Article 7.4 Risk Management C June 30, 2030 To be determined To be determined Article 7.5 Post-clearance Audit C June 30, 2026 To be determined Article 7.7 Trade Facilitation Measures for Authorized Operators C June 30, 2026 To be determined Article 7.8 Expedited Shipments C June 30, 2026 To be determined Article 7.9 Perishable Goods C June 30, 2030 To be determined To be determined Article 8 Border Agency Cooperation C June 30, 2030 To be determined Article 10.1.1(a) Formalities and Documentation Requirements C June 30, 2030 To be determined To be determined Article 10.4 Single Window C June 30, 2030 To be determined Article 11.5 Freedom of Transit C June 30, 2030 To be determined Article 11.9 Freedom of Transit C June 30, 2030 To be determined Article 11.16 Freedom of Transit C June 30, 2030 To be determined Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 65 Annex H. Government Scorecards Data Summary Sl N o Cooperati ng Agency Legal Framewo rk Trade Relate d Servic es Informati on Technolog y WTO Notificati on Processes Private Sector Engageme nt Institution al Capacity Risk Manageme nt Laboratori es & Testing Procedure Prioritizati on of Perishable Goods Averag e Score 1 PQW 1.60 1.67 2.33 2.00 4.00 2.00 0.63 2.00 3.00 2.14 2 BSTI 1.50 1.67 0.33 2.50 4.00 2.00 0.13 3.00 1.00 1.79 3 DLS 1.20 1.67 1.33 2.00 3.00 1.33 0.25 2.00 2.00 1.64 4 DoF 1.60 1.33 0.00 2.00 3.00 1.33 0.00 3.00 1.00 1.47 5 BAEC 1.40 1.33 0.33 1.50 2.00 2.00 0.38 2.00 1.00 1.33 6 BFSA 0.80 NA 0.00 1.00 NA 1.33 0.00 NA NA 0.63 Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 66 Annex I. Laboratory Scorecards Data Summary Sl No Organizations IT Systems, Level of Automation and Online Reporting Sampling Procedures and Capabilities Testing Capabilities Implementation of Management Systems & Accreditation Labs Institutional Capacity Average Score 1 BSTI 3.00 3.5 3.5 3.5 3.17 3.11 2 Waffen Private Lab 2.5 2.67 3.75 3.5 2.83 3.05 3 DLS QC Lab 3.2 3.5 3.25 2.5 2.67 3.02 4 DOF 0.25 3.33 3.25 3 3.4 2.71 5 BAEC 0.25 0 3 1 2.60 1.71 6 DLS - PRTC 1.5 - 2.67 0 2.2 1.59 7 DLS - BLRI 0.2 3 2.5 0 2 1.54 8 PQW HSIA (airport) 0 2 1.25 0 1.33 0.92 9 PQW - Mongla 0 2 1.25 0 1.17 0.88 10 PQW - Chittagong 0 2 1.25 0 1.33 0.92 Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 67 Annex J. BTF Indicators by Categories Indicator No. Performance Indicator Category (a) - related to supporting public/private institutions (e.g. regulations, procedures, partnerships, increasing certificates) and improving human resources (e.g. training). 6 Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance 7 Number of individuals participating in USDA food security programs 8 Number of individuals benefiting indirectly from USDA-funded interventions 9 Number of individuals who have applied improved management practices or technologies with USDA assistance 10 Number of WTO Trade Facilitation Agreement provisions supported by USG assistance (EG.2.1-1) 16 Number of firms receiving USG assistance that have obtained certification with (an) international quality control institution(s) in meeting minimum product standards (EG2.2-2) 17 Number of organizations with increased performance improvement with USDA assistance 18 Number of policies, regulations, and/or administrative procedures in each of the following stages of development as a result of USDA assistance 19 Number of certificates issued through project-supported e certificate systems 20 Number of public-private partnerships formed as a result of USDA assistance Category (b) - related to value or volume of trade. 5 Percentage change (from baseline) in value of exports and imports of targeted agricultural products as a result of USDA assistance 12 Percentage change (from baseline) in volume of exports from Bangladesh of agricultural products as a result of USDA assistance Category (c) - related to the quality improvement of the goods traded (e.g. release of goods, rejection ratios, investment in facilities). Preparations of vegetables, fruit or nuts Venture37 Bangladesh Trade Facilitation Evaluation: Baseline Report BASELINE REPORT | 68 Indicator No. Performance Indicator 11 Percentage change (from baseline) in Bangladesh export consignments of agricultural products rejected by import countries 13 Average time to release agricultural products at the ports 14 Percentage of agricultural import consignments inspected/tested 15 Total increase in installed storage capacity as a result of USDA assistance 21 Value of new USG commitments and new public and private sector investment leveraged by USDA to support food security and nutrition