Paraguay Food for Progress Trade-Facilitating Agricultural Systems and Technology (T-FAST) Project Midterm Evaluation 6/1/2022 Revised 7/16/2022 DISCLAIMER: This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Midterm Evaluation of the T-FAST Project in Paraguay Program: Food for Progress Agreement Number: FCC-526-2019/010-00 Funding Year: Fiscal Year 2019 Project Duration: 2019-2023 Implemented by: International Executive Service Corps (IESC) Evaluation Authored by: LINC with partner Instituto Desarrollo Rich Fromer (LINC), Jose Molinas (Instituto Desarrollo) LINC LLC Page i T-FAST Project Instituto Desarrollo Midterm Evaluation Report Table of Contents List of Acronyms.............................................................................................................................................v Executive Summary .......................................................................................................................................1 Project Background....................................................................................................................................1 Evaluation Design and Methods................................................................................................................1 Findings and Conclusions...........................................................................................................................2 Relevance...............................................................................................................................................3 Effectiveness..........................................................................................................................................3 Efficiency................................................................................................................................................3 Sustainability..........................................................................................................................................3 Impact....................................................................................................................................................4 Challenges..............................................................................................................................................4 Recommendations.....................................................................................................................................4 Overall Project Recommendations........................................................................................................4 Recommendations for NTFC Support....................................................................................................5 1. Introduction and Purpose......................................................................................................................7 1.1. Project Context..............................................................................................................................7 1.2. Project Description ........................................................................................................................9 1.3. Results Framework ......................................................................................................................12 1.4. Purpose of the Evaluation ...........................................................................................................12 2. Evaluation Design and Methodology...................................................................................................14 2.1. Evaluation Questions...................................................................................................................14 2.2. Evaluation Design ........................................................................................................................15 2.3. Data Collection and Analysis Methods and Target Respondents................................................16 2.3.1. Electronic Survey .................................................................................................................16 2.3.2. Focus Group Discussions .....................................................................................................18 2.3.3. Key Informant Interviews ....................................................................................................18 2.3.4. Document Review................................................................................................................19 2.4. Evaluation Limitations .................................................................................................................19 3. Findings................................................................................................................................................21 3.1. Findings from the Survey .............................................................................................................21 3.1.1. Respondents........................................................................................................................21 LINC LLC Page ii T-FAST Project Instituto Desarrollo Midterm Evaluation Report 3.1.2. Relevance.............................................................................................................................22 3.1.3. Effectiveness........................................................................................................................23 3.1.4. Sustainability........................................................................................................................25 3.1.5. Challenges............................................................................................................................26 3.2. Findings from the KIIs..................................................................................................................26 3.2.1. Relevance.............................................................................................................................26 3.2.2. Effectiveness........................................................................................................................28 3.2.3. Efficiency..............................................................................................................................30 3.2.4. Sustainability........................................................................................................................30 3.2.5. Impact..................................................................................................................................31 3.2.6. Challenges............................................................................................................................31 3.3. Findings from Project Documents...............................................................................................32 3.3.1. Effectiveness........................................................................................................................32 3.3.2. Efficiency..............................................................................................................................39 3.3.3. Challenges............................................................................................................................40 4. Conclusions..........................................................................................................................................41 4.1. Relevance.....................................................................................................................................41 4.1.1. To what extent have T–FAST expected results and activities to date addressed the core issues of target beneficiaries? .............................................................................................................41 4.1.2. Is the project relevant in addressing goals and objectives of the Trade Facilitation Agreement (TFA), particularly in assisting Paraguay with its Category C notifications?.....................41 4.1.3. Is the project coherent with other projects that support the TFA? Are the projects mutually supporting or duplicating efforts and if so, how? ................................................................41 4.2. Effectiveness................................................................................................................................42 4.2.1. How effectively is the project reaching its targets, both timewise and per the work plan? Is T–FAST on track to achieve the specific targets and results established? ......................................42 4.2.2. Which activity or combination of activities has/have the highest potential to achieve T– FAST’s higher-level result (expanded trade of agricultural products)?...............................................43 4.2.3. How effective is T–FAST at reaching women beneficiaries (particularly via capacity building activities) and what more could be done to improve women’s engagement / leadership? 43 4.2.4. Which activities were most effective in contributing to a reduction of average time of trade? .............................................................................................................................................43 4.3. Efficiency......................................................................................................................................44 4.3.1. To what extent are staffing, management, and oversight costs suitable, given the number and scope of activities carried out and the outputs obtained? ..........................................................44 LINC LLC Page iii T-FAST Project Instituto Desarrollo Midterm Evaluation Report 4.4. Sustainability................................................................................................................................44 4.4.1. What factors contribute to the sustainability of project results and how has T–FAST focused on these to date? To what extent is progress being made towards project sustainability/graduation, and what could be done to strengthen the sustainability of project activities and results? ..........................................................................................................................44 4.5. Impact..........................................................................................................................................45 4.5.1. Identify both intended and unintended intermediate results that are expected to contribute to / or hinder medium and long-term effects? Given the project trajectory so far, is it feasible to achieve the expected medium and long-term effect? How can these results be enhanced or mitigated to ensure expected medium and long-term results are achieved? ...............................45 4.6. Challenges....................................................................................................................................45 4.6.1. What are the external forces that negatively and positively impacted the project implementation, what were the effects on different stakeholders, and how did the project attempt to overcome, mitigate, or capitalize its impact? .................................................................................45 4.6.2. What are the key challenges in collecting data and information and how did the project overcome or mitigate? ........................................................................................................................46 4.6.3. Have recommendations from the baseline evaluation been incorporated into the project and if not, why not?.............................................................................................................................46 5. Recommendations...............................................................................................................................48 5.1. Overall Project Recommendations..............................................................................................48 5.2. Recommendations for NTFC Support..........................................................................................50 Annexes........................................................................................................................................................52 Annex 1: Results Framework ...................................................................................................................53 Annex 2: Indicator Table..........................................................................................................................54 Annex 3: Data Collection Instruments.....................................................................................................57 Draft Online Survey..............................................................................................................................57 Key Informant Interviews: Semi-Structured Interview Guide.............................................................62 Annex 4: Midterm Evaluation Scope of Work .........................................................................................65 PURPOSE AND SCOPE ..........................................................................................................................65 AUDIENCE AND KEY STAKEHOLDERS ...................................................................................................65 KEY EVALUATION QUESTIONS .............................................................................................................65 METHODOLOGY...................................................................................................................................66 EVALUATION MANAGEMENT AND COORDINATION...........................................................................68 ETHICAL GUIDELINES ...........................................................................................................................68 CONTRACTOR TASKS............................................................................................................................69 DELIVERABLES AND EXPECTED DUE DATES.........................................................................................72 LINC LLC Page iv T-FAST Project Instituto Desarrollo Midterm Evaluation Report Annex 5: Status of Actions Planned by Activity and Sub-Activity............................................................74 Annex 6: Articles of Trade Facilitation Agreement Supported by T-FAST...............................................81 Annex 7: IESC Comments and Responses to Draft Midterm Evaluation Report...... Error! Bookmark not defined. List of Figures Figure 1: Type of Support Received by Survey Respondents......................................................................22 Figure 2: T-FAST FFPr Results Framework ...................................................................................................53 List of Tables Table 1: Individual and Institutional Participation in Survey and KIIs by Sector and Gender.......................2 Table 2: Top Agricultural Export Products in 2021........................................................................................7 Table 3: Top Agricultural Import Products in 2021 .......................................................................................8 Table 4: Example Key Public and Private Sector Institutions ......................................................................12 Table 5: Evaluation Criteria and Questions .................................................................................................14 Table 6: Individual and Institutional Survey Participation by Sector and Gender.......................................17 Table 7: Individual and Institutional KII Participation by Sector and Gender .............................................18 Table 8: Individual and Institutional Survey Respondents by Gender and Institution Type.......................21 Table 9: How Survey Respondents Learned of T-FAST................................................................................22 Table 10: Survey Respondents’ Perception of Alignment of T-FAST to Priorities.......................................23 Table 11: Survey Respondents' Perceived Effectiveness of Specific T-FAST Activities ...............................24 Table 12: Survey Respondents’ Perception of T-FAST Effectiveness at Reaching Women and Reducing Time to Trade...............................................................................................................................................24 Table 13: Survey Respondents’ Perception of Sustainability of T-FAST Results to Date ............................25 Table 14: Survey Respondents' Perception of the Impact of External Forces ............................................26 Table 15: Progress vs. Plan by Sub-Activity .................................................................................................34 Table 16: Indicator Progress and LOP Projections.......................................................................................37 Table 17: Cost Effectiveness Analysis by Activity and Action Status...........................................................39 Table 18: Summary of Baseline Recommendations and T-FAST Responses...............................................46 Table 19: Indicator Table with Target and Actual for Years 1 & 2...............................................................54 Table 20: Status of Actions Planned by Activity and Sub-Activity (Year 2) .................................................74 Table 21: Articles of WTO TFA that T-FAST Supports..................................................................................81 LINC LLC Page v T-FAST Project Instituto Desarrollo Midterm Evaluation Report List of Acronyms Acronym Full Term AEO Authorized Economic Operator CAPECO Chamber of Exporters and Traders of Cereals and Oilseeds CIF Cost, Insurance, Freight CIP Center for Paraguayan Importers CIPE Center for International Private Enterprise DINAVISA National Health Surveillance Directorate DNA National Customs Directorate EU European Union FAS Foreign Agricultural Service FFPr Food for Progress FOB Free on Board GDP Gross Domestic Product ICC International Chamber of Commerce ICT Information and Communication Technology ID Instituto Desarrollo IESC International Executive Service Corps INAN National Institute of Food and Nutrition INTN National Institute of Technology, Standardization, and Metrology IT Information Technology KII Key Informant Interview KSU Kansas State University LOP Life of Project MAG Ministry of Agriculture MEL Monitoring, Evaluation, and Learning MIC Ministry of Industry and Trade MRE Ministry of Foreign Affairs MSPyBS Ministry of Public Health NCM MERCOSUR Common Nomenclature LINC LLC Page vi T-FAST Project Instituto Desarrollo Midterm Evaluation Report NTFC National Trade Facilitation Committee OPI Organizational Performance Index PMP Performance Monitoring Plan SENACSA National Livestock Quality and Health Service SENAVE National Vegetable and Seed Quality and Health Service SOW Scope of Work TFA Trade Facilitation Agreement T-FAST Trade-Facilitating Agricultural Systems and Technology TOC Theory of Change UIP Paraguayan Industrial Union UN United Nations UNCTAD UN Conference on Trade and Development USD U.S. Dollar USDA U.S. Department of Agriculture USG U.S. Government VUE Single Export Window VUI Single Import Window WTO World Trade Organization LINC LLC Page 1 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Executive Summary Project Background The 4-year (2019-2023), $8.9 million Trade-Facilitating Agricultural Systems and Technology (T-FAST) Project, funded by the U.S. Department of Agriculture (USDA) and implemented by the International Executive Service Corps (IESC), aims to simplify, modernize, and harmonize processes for the export, import, and transit of agricultural goods in Paraguay. T-FAST works with the private and public sectors to reduce nontariff barriers to trade with a target of achieving a 30% decrease in release time of agricultural goods. As Paraguay joined the World Trade Organization’s Trade Facilitation Agreement (TFA) in 2016, the project focuses on supporting Paraguay with key priorities related the facilitation of trade: collaboration, capacity, and the Category C Notifications of the TFA (articles of the TFA that Paraguay has asked for help to achieve, see Annex 6 for more information). The primary stakeholders of the project are the leadership and members of the National Trade Facilitation Committee (NTFC), established in 2017 to support trade facilitation and achieve full compliance with the TFA. The Ministry of Foreign Affairs oversees the NTFC and its members are representatives of relevant public and private sector institutions. Beyond the NTFC, T-FAST project participants include civil servants at the public sector institutions responsible for foreign trade policy and services, and staff or members of trade associations that support or engage in import and export. T-FAST works through five key activities to achieve its goals: (1) Capacity building for the NTFC; (2) Capacity building of government institutions; (3) Capacity building of trade associations; (4) Provision of market and trade infrastructure (primarily information and communications technology); and (5) On￾demand support to flexibly meet needs as they arise. IESC has hired LINC and Paraguayan partner Instituto Desarrollo as independent evaluators. The same team conducted the baseline study. This report is for the midterm evaluation which has been designed to review the project’s implementing experience and environment, assess the relevance and effectiveness of the project, assess to what extent the project is on track to achieve its goals and objectives, and identify recommended mid-course corrections. Evaluation Design and Methods The midterm evaluation is a performance evaluation, primarily designed to inform IESC and USDA about the performance of the T-FAST project to date. The evaluation focuses on key questions related to five important evaluation criteria of relevance, effectiveness, efficiency, sustainability, and impact. To effectively address the evaluation criteria, the evaluation team employed a mixed-methods approach, using both quantitative and qualitative data. Different data sources were required to capture the variety of experiences and engagement of a diverse set of stakeholders. The mixed-methods approach allowed for triangulation of findings using data from multiple sources and to maximize input contributing to understanding how the T-FAST project responds to each of the criteria. The evaluation team originally planned to collect data from document review, an online survey, key informant LINC LLC Page 2 T-FAST Project Instituto Desarrollo Midterm Evaluation Report interviews (KIIs), and focus group discussions (FGDs). However, due to feedback from crucial stakeholders on interest and availability to participate and indicating that participation in multiple data collection processes would be a high burden on their time, the team removed FGDs and replaced them with additional KIIs. The survey and KIIs target the participants in the project and in particular the members of the National Trade Facilitation Committee (NTFC) which includes representatives from public institutions and from private sector associations. Additionally, each data collection method has included other representatives of international cooperation or other agencies supporting trade facilitation to incorporate their perception of Paraguay’s priorities, complementarity and effectiveness of T-FAST activities, and other information to support the evaluation. The online survey was primarily composed of closed-ended and Likert scale perspective questions allowing analysis of responses. Open-ended questions added flexibility to collect individual perspectives on project performance and opportunities for improvement. The results provided knowledge that supplement and complement the qualitative KII data. The online survey was launched the week of March 21, with reminders sent electronically during the following weeks. Additionally, the evaluation team followed up by phone with institutions that had not responded. The survey was closed on April 23 with responses from 33 individuals representing 28 institutions. The profile of respondents to the survey is included in the table below. The KIIs use a semi-structured format, allowing the evaluation team to adapt questions and areas of focus to each respondent’s experience with trade facilitation and with the T-FAST project. This format also provided the opportunity for the evaluation team to probe for further depth on responses provided where appropriate and to supplement data collected through the survey. The evaluation team coded qualitative KII responses against evaluation criteria for further analysis. A total of 25 KIIs were conducted with individuals or small groups of representatives from a diverse set of stakeholder institutions and agencies. The profile of KII participants is included in the table below. To supplement and augment the survey and KII data, the evaluation team conducted a review of project documentation including workplans, progress reports, indicator tables and backup data, and financial reporting. This additional data contributed to understanding the effectiveness and efficiency of the T￾FAST project to date, and added context to the data collected through the survey and KIIs. Table 1: Individual and Institutional Participation in Survey and KIIs by Sector and Gender Online Survey Key Informant Interviews Sector Individuals Institutions Individuals Institutions Public Sector 21 (11M/10F) 18 24 (11M/13F) 15 Private Sector 11 (5M/6F) 9 4 (0M/4F) 4 Other 1 (0M/1F) 1 12 (6M/6F) 6 Total 33 (17M/16F) 28 40 (17M/23F) 25 Findings and Conclusions The main findings and conclusions of the evaluation on the performance of the T-FAST project presented below fall into six categories investigated by the evaluators. LINC LLC Page 3 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Relevance T-FAST’s expected results and activities to date address the core issues of target beneficiaries. The top priorities of project participants are digitalization of foreign trade procedures and paperwork, simplification of processes and procedures, and increased coordination across relevant public institutions. T-FAST’s stated objectives, plans, and activities align well with these priorities, and with the goals of the TFA. Additionally, T-FAST has coordinated closely with other projects that support trade facilitation, often filling gaps that those projects are not able to address. Stakeholders have also indicated some priorities for foreign trade that are outside of the scope of T￾FAST and go beyond the requirements of the TFA. The most important of those is infrastructure, primarily to address Paraguay’s challenges as a landlocked country. Effectiveness T-FAST is making effective progress towards its targets and objectives. Based on both stakeholder input and document review, the project is performing well and is likely to achieve its objectives. The most valued strengths of the project are the expertise of staff and consultants, flexibility of the project to address changing needs, responsiveness of the project and its staff to questions and concerns raised by participants, and the approach to engaging embedded staff within the Ministry of Foreign Affairs to support the NTFC. The project activities that are most effective in contributing to T-FAST’s goals are those that directly and indirectly address the processes, procedures, rules, and paperwork required for import or export. Those include: • Digitalization of trade processes and paperwork including provision and expansion of IT hardware and software. • Streamlining of procedures including analysis of workflows to identify areas of duplication, unnecessary steps, or opportunities for improvement. • Consultancies with high priority public institutions involved in trade policy, regulation, or services to identify and address their constraints. • Expansion of the Authorized Economic Operator program. Efficiency T-FAST stakeholders indicate that the project is providing substantial benefit to support foreign trade in general, to the NTFC, and to the institutions it supports directly. They feel that the value is proportionate to or exceeds expectations for the resources employed. A comparison of the annual workplans and semi-annual reports highlight the flexibility of the project, again one of the characteristics most valued by stakeholders. While the project has not completed all of the activities planned in a given project year, the ability to adapt has been a key factor in the effectiveness of the project to date. Sustainability Project partners highlight several factors that contribute to the likely sustainability of T-FAST activities and results. Foremost among those factors is the value of digitalization, simplification, and related IT systems for the processes and procedures of foreign trade. The improved processes benefit both public sector institutions providing services and private sector actors involved in trade. It is in none of those LINC LLC Page 4 T-FAST Project Instituto Desarrollo Midterm Evaluation Report stakeholders’ interests to return to the previous way of doing things. Stakeholders also note that there are ample opportunities to continue this work and note that increased reliance on the new processes with increase the likelihood of sustainability and future efforts to improve further. The other key mechanism for sustainability of the project’s activities and results is the strength of the NTFC. T-FAST has been supporting NTFC leadership directly, the coordination of the committee, increasing transparency and accountability mechanisms, and strengthening the working groups as an efficient structure. Stakeholders indicate that there are many more opportunities to strengthen the NTFC and that this will increase the likelihood of sustainability of efforts. Such opportunities include continuous improvement to the operational structure, engaging the National Economic Cabinet, and continued improvement in communications. Impact The most important result of the activity to date has been the uptake of technology to support digitalization, simplification, and inter-institutional coordination and data sharing. The IT systems and related improvement of processes, paperwork, procedures, and data management contribute greatly to the medium and long-term goals of reduction of average time to trade and sustainability of the project’s support. Challenges There are three primary external forces that negatively impacted project implementation: the COVID-19 pandemic, the turnover in key positions in the public sector, and slow processes due to bureaucracy. The project has adapted to the situations as they have arisen. Some specific planned project activities are still behind their implementation timeline due primarily to pandemic restrictions. Interestingly, the pandemic has also positively impacted the project. Due to safety-related restrictions, there was increased urgency and willingness to adopt IT systems and digitalization. Additionally, NTFC meetings moved online which facilitated participation by physically dispersed and busy members. Recommendations The recommendations to the T-FAST project, based on lessons learned from the fieldwork and findings of the midterm evaluation, fall into two categories: Overall Project Recommendations • Continue and expand the most effective project activities. These activities are already in T￾FAST’s project plans, and there are many other activities planned that are not meant to be diminished by this recommendation. Still, T-FAST should continue to invest resources and efforts in the most effective and valued project activities. Digitalization is perhaps the most valued support that T-FAST is providing. Streamlining foreign trade processes is also highly valued by LINC LLC Page 5 T-FAST Project Instituto Desarrollo Midterm Evaluation Report project participants and stakeholders. Institutional strengthening for high priority public institutions has been effective for the reduction of time to trade. • Maintain flexibility and responsiveness of the project. This recommendation is to continue to focus on one of the strengths of the project to date. Flexibility and the ability to adapt has been a key factor in the effectiveness of the project. • Support coordination by public institutions with the appropriate mandate. T-FAST is providing specific support and institutional strengthening to several individual public institutions. There is an opportunity to localize at least some of that support within government and create a more sustainable coordination structure. Two specific examples come from the fieldwork. Engage the Central Laboratory of the Ministry of Public Health as a coordinating and support institution for the various labs engaged in clearance of products for trade. Engage the Ministry of Agriculture in consultancies with public institutions that are related to agricultural products. • Consider adding time to the project. Given that the pandemic caused some project delays in Year 1 and slowed implementation of some activities, the project and its beneficiaries could benefit from an extension. • Review and strengthen the communications strategy and plan as appropriate. Project and NTFC communications were highlighted as a strength by some stakeholders and as a weakness by others. T-FAST has already invested in improving and formalizing communications and communications planning, and a review of the strengths and opportunities would help ensure a more consistently positive perception of communications. • Review and strengthen procurement procedures as appropriate. While responsiveness of T￾FAST to stakeholder needs and requests is a strength, the feedback from some stakeholders on the slow speed of procuring some equipment or services by the project is worth further investigation. Recommendations for NTFC Support • Further develop and strengthen the structure of the NTFC. The establishment of NTFC working groups has improved the performance and results of the overall committee. However, there are opportunities for continued improvement of the NTFC structure, including: o Support increased autonomy for the working groups. o Work to decrease overlap in membership across working groups to increase efficiency of people’s time. o Establish a regular review for NTFC leadership and members to reflect on the structure of working groups and make decisions to adapt. o Establish a coordinating body like a steering committee or a secretariat comprised of members of multiple institutions and including public and private sectors. • Facilitate regular review of the NTFC decree and legal framework. The decree has also already been updated once, based on the experience and needs of the NTFC. T-FAST can support a LINC LLC Page 6 T-FAST Project Instituto Desarrollo Midterm Evaluation Report regular review of the decree to propose revisions as appropriate, based on changing priorities, emerging constraints or opportunities, and NTFC needs. • Increase private sector representation. The private sector has been very actively involved in the NTFC, the working groups, and other activities of the project. However, they have limited time and budget for participation in activities. Additional private sector members would reduce the burden on existing members. • Engage the government’s National Economic Cabinet (Equipo Economico Nacional). The National Economic Cabinet is a coordination body comprised of eight cabinet ministers. This group’s support may be necessary for political will to make changes proposed by the NTFC, and for the long-term financial sustainability of the NTFC. • Review the mandate and scope of the NTFC for possible expansion. Per decree, the NTFC’s mandate is quite wide, however it has been primarily focused on meeting the requirements of the TFA. That is an understandable and important focus but may impede the NTFC from becoming a more sustainable entity with continued activities in the long term. As indicated in the next recommendation bullet, analysis or other support that can lead to necessary infrastructure development is an example of a potential opportunity for the NTFC with an expanded scope. • Support NTFC with analysis of infrastructure needs. It could be cost effective and beneficial to have T-FAST support or conduct some exploratory analysis of the challenges related to physical infrastructure needed to expand and facilitate trade. In addition to supporting trade and the NTFC, this would help the NTFC and its members identify upcoming challenges and shift focus to a longer-term future. LINC LLC Page 7 T-FAST Project Instituto Desarrollo Midterm Evaluation Report 1.Introduction and Purpose 1.1. Project Context Paraguay has enjoyed a dynamic and growing economy over the last decade, with low unemployment. Economic growth averaged 4.3% from 2010-2019 and unemployment averaged 5.5%.1 The agricultural sector decreased as a portion of GDP during this time, from 13% in 2010 to 10% in 2019.2 This is still high compared to neighbors such as Brazil (3%) and Argentina (5%).3 The economic contraction due to the coronavirus pandemic in 2020 has been smaller than initially expected, in part due to the resilience of the agricultural sector.4 In terms of foreign trade, Paraguay is an agricultural, export-oriented economy with more than 50% of its export markets in nearby South American countries.5 Two-way trade between the U.S. and Paraguay initially decreased during the pandemic, from $2.3 billion in 2019 to $1.4 billion in 2020, before rebounding to $2.2 billion in 2021.6 Paraguay’s exports are primarily concentrated in its main trading partners of Brazil (34% by value) and Argentina (26% by value).7 Exports are also primarily composed of agricultural products, with unprocessed primary products representing 38% and processed agricultural products accounting for 35% of total by value. 8 Soybean is the top export product by value (28%), followed by beef (16%), and electric energy (15%). This represents a lack of diversification of exports in terms of both products and markets.9 Paraguay’s imports originate from a wider variety of trade partners, with the two largest being MERCOSUR (35%) and China (30%). Import products are diverse, including an emphasis on manufactured goods. Machinery, equipment, and electrical materials account for 16.8% of total imports, followed by fuels (12.9%), boilers and machinery (12.4%), and vehicles (8.6%). 10 Tables 1 and 2 below show the top agricultural export and import products, values, and top trading partners for 2021. Table 2: Top Agricultural Export Products in 2021 NCM Description US$ FOB Value % of total exports Top Destination 1201.90.00 Soy beans - others 2,975,059,493 28.20% Argentina 1 Source: Central Bank of Paraguay, Statistics Department 2 Ibid 3 Source: U.S. Census Bureau, foreign trade statistics 4 Source: Central Bank of Paraguay, Statistics Department 5 Ibid 6 Source: U.S. Census Bureau, foreign trade statistics 7 Source: Central Bank of Paraguay, Statistics Department 8 Ibid 9 Ibid 10 Ibid LINC LLC Page 8 T-FAST Project Instituto Desarrollo Midterm Evaluation Report 0201.30.00 Meat of bovine animals, fresh or refrigerated - boneless 788,747,461 7.48% Chile 0202.30.00 Meat of bovine animals, frozen - boneless 771,271,210 7.31% Russia 2304.00.10 Cakes and other solid residues, from the extraction of soybean oil (soy), even grounded or in pellets - Flour and Pellets 762,123,828 7.22% Chile 1507.10.00 Soybean oil and its fractions, even refined, but not chemically modified - Crude oil, including degummed 620,501,814 5.88% India 1005.90.10 Corn - grain 409,073,143 3.88% Brazil 1006.30.21 Rice - Semi-milled or blanched rice, whether or not polished or glazed - Not parboiled 121,522,686 1.15% Brazil 1001.99.00 Wheat and meslin - Others 89,153,096 0.85% Brazil Source: Central Bank of Paraguay (2022) Table 3: Top Agricultural Import Products in 2021 NCM Description US$ CIF Value % of total imports Top Origin 3105.20.00 Mineral or chemical fertilizers, with two or three of the fertilizing elements; nitrogen; phosphorus and potassium; the other fertilizers; products of this chapter in tablets or similar forms or in containers of a gross weight less than or equal to 10 kg - Mineral or chemical fertilizers with the three fertilizing elements: nitrogen, phosphorus and potassium. 152,462,545 1.22% Brazil 3105.40.00 Mineral or chemical fertilizers, with two or three of the fertilizing elements; nitrogen; phosphorus and potassium; the other fertilizers; products of this chapter in tablets or similar forms or in containers of a gross weight less than or equal to 10 kg - Ammonium dihydrogen orthophosphate (monoammonium phosphate), even mixed with diammonium hydrogen orthophosphate (diammonium phosphate) 138,755,692 1.11% China 3808.92.99 Insecticides, rodenticides, fungicides, herbicides, germination inhibitors and regulators of the plant growth, disinfectants and similar products, presented in forms or packages for retail sale, or as preparations or in articles, such as tapes, sulfured wicks and spark plugs, and fly killer papers - others. 88,542,055 0.71% Brazil 1806.90.00 Chocolate and other food preparations containing cocoa - others. 15,749,674 0.13% Brazil 2106.90.30 Food preparations not expressed or included elsewhere - other - food supplements 13,266,161 0.11% USA Source: Central Bank of Paraguay (2022) LINC LLC Page 9 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Following the drought that hit Paraguay in 2019 and the adverse effects of COVID-19 in 2020, the Paraguayan economy recovered strongly in 2021. Real GDP reached year-on-year growth of 4.2% in 2021, explained by a rebound in fixed investment, both in private and public construction works, as well as growth in private consumption. Despite high commodity prices, net exports slowed growth, as the drought negatively impacted the export value of soybeans and hydroelectric power, two of the country's main export products. The drought is expected to slow GDP growth in 2022, which would mitigate growth in exports and private consumption.11 1.2. Project Description In 2016, Paraguay was the second South American country to ratify the World Trade Organization’s (WTO) Trade Facilitation Agreement (TFA). The TFA aims to help Paraguay promote economic growth by improving the processes and transparency of trade facilitation to reflect international best practices and norms. The 4-year (2019-2023), $8.9 million Trade-Facilitating Agricultural Systems and Technology (T-FAST) Project, funded by the U.S. Department of Agriculture’s (USDA’s) Food for Progress (FFPr) program, aims to simplify, modernize, and harmonize processes for the export, import, and transit of agricultural goods in Paraguay. The project is implemented by the International Executive Service Corps (IESC). Working with the private and public sectors, the project is designed to reduce nontariff barriers to trade resulting in a 30 percent decrease in release time of agricultural goods. The project will also create improvements in predictability through the increased transparency and automatization of processes. T-FAST supports five key activities to improve import and export processes in Paraguay and support Paraguay to fulfill TFA obligations. These activities consist of: 1. Capacity building for the National Trade Facilitation Committee (NTFC). The project is building the capacity of Paraguay’s NTFC to promote an improved policy and regulatory framework for agricultural trade facilitation. The Paraguay NTFC was created by executive decree in April 2017, with the goal of developing a national strategic agenda for achieving full compliance through coordination and implementation of trade facilitation efforts. Direct beneficiaries include NTFC staff and cross-agency working groups. T-FAST conducted an initial assessment to identify core capacity gaps or needs, including structural and organizational, monitoring and evaluation, donor coordination, and communications capacity. This Activity started in Year 1 and is planned to conclude in Year 4. Example sub-activities undertaken in the first two years of the project include: • Establish and Coordinate Steering Committee: T-FAST organized a project Steering Committee comprised of representatives from the in-country T-FAST leadership, US-based T-FAST Home Office team, USDA field-based regional team, USDA US-based program team, 11 Source: Central Bank of Paraguay LINC LLC Page 10 T-FAST Project Instituto Desarrollo Midterm Evaluation Report and the US Embassy in Asuncion, Paraguay. The committee meets quarterly and provides feedback and input as well as rapid responses on strategic topics that arise during implementation. • Strengthen and Support NTFC Working Groups: T-FAST has supported the establishment, planning, monitoring, and organization/re-organization of thematic working groups within the NTFC to focus on specific areas of high priority such as process simplification, agricultural goods, and transparency. • Support NTFC to Fulfill Its Role: T-FAST has provided a variety of support to the NTFC and its coordinators directly. T-FAST seconded two staff full-time to the Ministry of Foreign Affairs (MRE) who coordinate the NTFC. Through those two staff and other support from the T-FAST office, the project has provided technical assistance for the management of the NTFC and its working groups, public and internal communications on the NTFC’s actions, monitoring of NTFC work against indicators and development of a “project dashboard” for the NTFC, and other capacity building and training activities. 2. Capacity building of government institutions. The project is responding to requests from Paraguay’s public institutions for capacity building related to agriculture import and export processes important to fulfilling TFA obligations and realizing TFA benefits. T-FAST is building technical and managerial capacity for select government institutions involved in the regulation and facilitation of trade in agricultural products. This technical assistance supports government institutions in their efforts to implement Paraguay’s 11 Category C notifications of the TFA. This Activity started in Year 1 and is planned to conclude in Year 4. Example sub-activities undertaken in the first two years of the project are aligned with the Articles of the TFA that the project is supporting the NTFC to achieve (see Annex 6 for more information) and include: • Support to the NTFC and Its Public Sector Members Related to Communications: T-FAST is helping the NTFC and public institutions to follow through on their planned actions to comply with Articles 1 and 2 of the TFA, in particular for publication of information, public notification of policy changes, and capacity to receive public comments and policy proposals. • Support to the NTFC and Its Public Sector Members Related to Laboratories: To support compliance with Article 5 of the TFA, T-FAST has conducted assessments and provided training and equipment to improve laboratory testing in public and private labs. • Support to the NTFC and Its Public Sector Members to Decrease Time for Foreign Trade: To support progress towards Article 7 of the TFA, T-FAST has worked with key public institutions (see Table 4 below) to assess constraints to trade, simplify processes, digitalize documentation and approvals, identify potential information and communications technology (ICT) needs and options, and extend the Authorized Economic Operator (AEO) program which seeks to reduce release time for foreign trade shipments. 3. Capacity building of trade associations. T-FAST provides technical and change management capacity building to enable trade associations to effectively advocate for trade-related reforms. The project has begun working with selected trade associations to understand the relevance of LINC LLC Page 11 T-FAST Project Instituto Desarrollo Midterm Evaluation Report the TFA to their business objectives, to analyze the impact of related policy changes on their sector, and to effectively communicate with government about these issues. This Activity started in Year 2 and is planned to conclude in Year 4. Example sub-activities undertaken in the first two years of the project include: • Build Capacity of Private Sector to Engage in Trade Facilitation Reform: T-FAST has conducted a training needs assessment and provided training to key private sector associations (see Table 4 below). • Promote AEO Program with Trade Associations: T-FAST has collaborated with the National Customs Directorate (DNA) AEO Coordinator to promote AEO certification. 4. Provision of market and trade infrastructure. The project provides the necessary infrastructure to implement the policy, procedure, and process reforms developed in other project Activities, including infrastructure, equipment, and information and communication technology (ICT) systems. T-FAST works with NTFC staff and members, other government agencies, and private sector stakeholders to identify infrastructure needs that are not duplicative. The project is procuring the infrastructure and ensuring that NTFC staff and member users are trained to use the equipment and systems procured. This Activity started in Year 2 and is planned to conclude in Year 4. Example sub-activities undertaken in the first two years of the project include: • Develop ICT Infrastructure Related to Trade Facilitation Communications: T-FAST is working with the NTFC and its members to develop and launch a “platform” for communications and notification for the public and receipt of public comment on policy proposals per Article 2 of the TFA. • Develop ICT Infrastructure Related to Reducing Release Time of Foreign Trade Shipments: T-FAST has worked with key public institutions on infrastructure to support process simplification and digitalization, including conducting needs assessments, procuring equipment and servers, negotiating licensing, installing software, and providing training to users. 5. On-demand support to flexibly meet needs as they arise. The project also implements a flexible, on-demand mechanism to support the Government of Paraguay and project objectives by responding to agricultural trade issues as they arise. Requests are evaluated against alignment with project objectives, potential impact, and level of institutional buy-in. This Activity started late in Year 2 and is planned to conclude in Year 4. The T-FAST project approach centers on three pillars: motivating and incentivizing collaboration, building organizational and human capacity, and focusing on Category C Notifications of the TFA. To do this, the primary stakeholders of the project include the leadership and members of the NTFC, which is overseen by the Ministry of Foreign Affairs (MRE, in Spanish) and comprised of a secretary and representatives of public and private sector institutions. Beyond the NTFC and its participants, T-FAST stakeholders include civil servants at the main public sector institutions which have responsibilities related to foreign trade policy and processes, and staff or members of trade associations that support or directly engage in import and export. LINC LLC Page 12 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Table 4: Example Key Public and Private Sector Institutions Example Key Public Institutions Example Key Private Sector Associations • National Health Surveillance Directorate (DINAVISA) • National Livestock Quality and Health Service (SENACSA) • National Vegetable and Seed Quality and Health Service (SENAVE) • National Institute of Technology, Standardization, and Metrology (INTN) • National Institute of Food and Nutrition (INAN) • Center for Paraguayan Importers (CIP) • Paraguayan Industrial Union (UIP) • Chamber of Exporters and Traders of Cereals and Oilseeds (CAPECO) • International Chamber of Commerce (ICC) 1.3. Results Framework The project results framework is included in Annex 1. The project’s Theory of Change (TOC) states that: “By developing a culture of continuous improvement, building organizational and human capacity to institute trade related improvements, and supporting the relevant physical infrastructure, Paraguay will fully implement the World Trade Organization’s (WTO) Trade Facilitation Agreement in both letter and spirit. This will help make Paraguay more competitive in the global marketplace and increase the potential for regional and international agricultural trade.” The project is based on several critical assumptions which could impact its ability to achieve the TOC: • The government of Paraguay will maintain its present commitment to the WTO regarding implementation of the TFA. • Trade associations will maintain interest in engaging in trade policy reforms and provide comments on proposed reforms. • The government of Paraguay will continue its planned adoption of automated technologies, including those supported by T-FAST. • The COVID-19 pandemic does not prevent continued operation of the project or involvement of key stakeholders. 1.4. Purpose of the Evaluation The T-FAST project design includes a requirement for an independent third-party contractor to conduct a baseline study, midterm evaluation, and final evaluation to better understand the T-FAST project’s performance over time. LINC was selected by IESC to implement the three evaluations, supported by Paraguayan partner Instituto Desarrollo (ID). This report provides the results of the midterm evaluation, focusing on approximately the first two years of project implementation. In compliance with the USDA’s monitoring and evaluation policy, IESC’s midterm evaluation will “critically and objectively review and take stock of the project’s implementing experience and environment, assess whether targeted beneficiaries are receiving services as expected, assess to what extent the project is on track to achieve its stated goals and objectives, review the results frameworks and assumptions, document initial lessons learned, and discuss necessary modifications or mid-course LINC LLC Page 13 T-FAST Project Instituto Desarrollo Midterm Evaluation Report corrections that may be necessary to effectively and efficiently meet the stated goals and objectives.” The evaluation examines both administrative and programmatic aspects of T-FAST. The midterm evaluation has primarily been designed to inform IESC (as the implementer) and USDA (as the donor) about the performance of the T-FAST project to date. The purpose of the evaluation is to identify priorities and strategies for the remainder of the project and to improve overall project performance and outcomes. The evaluation is also designed to provide USDA with material for reporting and other internal purposes, and to provide an assessment of the reliability and accuracy of IESC’s reported data. Beyond making the midterm evaluation data available to USDA for their utilization, IESC will also share relevant results of the midterm to stakeholders in the public and private sector, and IESC’s partners on the project, Kansas State University and the Center for International Private Enterprise (CIPE). The T-FAST project focuses on implementation of the trade facilitation agreement, a newer area of emphasis under the USDA FFPr Program. This project and the midterm evaluation represent an opportunity to further expand USDA’s body of knowledge related to trade facilitation. The FFPr Learning Agenda is focused on traditional USDA project activities and work related to agricultural products rather than trade facilitation including non-agricultural products. As such, this evaluation does not seek to directly answer FFPr Learning Agenda questions. Still, the work of T-FAST is relevant to some components of the FFPr Learning Agenda, in particular, within the topics of Value Creation and Market Linkages within the Market Systems focus area. With additional research or reflection, USDA can therefore learn from T-FAST’s interventions and results and use that learning to inform future projects, including by: • targeting leverage points for agricultural trade, • supporting policies and governing mechanisms that could improve market expansion, • identifying opportunities to leverage the role of international platforms, and • utilizing ICT to increase agricultural actors’ access to critical market information. LINC LLC Page 14 T-FAST Project Instituto Desarrollo Midterm Evaluation Report 2.Evaluation Design and Methodology The LINC evaluation team conducted a performance evaluation to understand how the T-FAST project’s design, objectives, activities, and results respond to the needs for facilitation and growth of Paraguay’s foreign trade. The team used a mixed-methods approach to collect and analyze qualitative and quantitative data to gain a broad view of T-FAST’s relevance, effectiveness, efficiency, impact, and sustainability. This section highlights the evaluation questions that guided the evaluation, the data collection and analysis methods, target respondents, and limitation of the evaluation. The Midterm Evaluation Scope of Work (SOW) can be found in Annex 4 and the data collection instruments are provided in Annex 3. 2.1. Evaluation Questions The T-FAST midterm evaluation answers key questions related to the USDA standard evaluation criteria of relevance, effectiveness, efficiency, sustainability, and impact. The table below highlights the evaluation questions within each criterion. Table 5: Evaluation Criteria and Questions Evaluation Criteria Evaluation Questions Relevance RQ1.1 To what extent have T-FAST expected results and activities to date addressed the core issues of target beneficiaries? RQ1.2 Is the project relevant in addressing goals and objectives of the Trade Facilitation Agreement (TFA), particularly in assisting Paraguay with its Category C notifications? RQ1.3 Is the project coherent with other projects that support the TFA? Are the projects mutually supporting or duplicating efforts and if so, how? Effectiveness RQ2.1 How effectively is the project reaching its targets, both timewise and per the work plan? Is T–FAST on track to achieve the specific targets and results established? RQ2.3 Which activity or combination of activities has/have the highest potential to achieve T–FAST’s higher-level result (expanded trade of agricultural products)? RQ2.4 How effective is T–FAST at reaching women beneficiaries (particularly via capacity building activities) and what more could be done to improve women’s engagement / leadership? RQ2.5 Which activities were most effective in contributing to a reduction of average time of trade? Efficiency RQ3.1 To what extent are staffing, management, and oversight costs suitable, given the number and scope of activities carried out and the outputs obtained? Sustainability RQ4.1 What factors contribute to the sustainability of project results and how has T– FAST focused on these to date? LINC LLC Page 15 T-FAST Project Instituto Desarrollo Midterm Evaluation Report RQ4.2 To what extent is progress being made towards project sustainability / graduation, and what could be done to strengthen the sustainability of project activities and results? Impact RQ5.1 Identify both intended and unintended intermediate results that are expected to contribute to / or hinder medium and long-term effects? Given the project trajectory so far, is it feasible to achieve the expected medium and long-term effect? How can these results be enhanced or mitigated to ensure expected medium and long-term results are achieved? Challenges RQ6.1 What are the external forces that negatively and positively impacted the project implementation, what were the effects on different stakeholders, and how did the project attempt to overcome, mitigate or capitalize its impact? RQ6.2 What are the key challenges in collecting data and information and how did the project overcome or mitigate? RQ6.3 Have recommendations from the baseline evaluation been incorporated into the project and if not, why not? 2.2. Evaluation Design The T-FAST midterm evaluation is a non-experimental performance evaluation. While impact evaluations are also valuable, the work that T-FAST is implementing at a national level with all or nearly all of the most relevant stakeholders makes an appropriate counterfactual control group extremely difficult to identify. Performance evaluations such as this midterm evaluation are often undertaken during implementation or at the conclusion of a project, valuable for drawing out lessons to improve design and implementation strategies to produce desired outcomes and sustainability. Given the goals of learning for improvement not only of the T-FAST project, but USDA’s growing number of projects supporting trade facilitation, a performance evaluation is appropriate. To effectively address the evaluation criteria and questions, the team employed a mixed-methods approach. Different data sources are required to capture the variety of experiences and engagement of a diverse set of stakeholders. A mixed-methods approach allowed the team to triangulate findings using data from multiple sources and maximize input, contributing to understanding how the T-FAST project responds according to each of the criteria. Data collection consisted of three methods: • Electronic survey • Key Informant Interviews (KIIs) • Document review The evaluation team originally planned to collect data from focus group discussions (FGDs) as well. However, due to feedback from crucial stakeholders on interest and availability to participate, including indicating that participation in multiple data collection processes would be a high burden on their time, the team removed FGDs and replaced them with additional KIIs. The survey allowed for primarily closed LINC LLC Page 16 T-FAST Project Instituto Desarrollo Midterm Evaluation Report ended and Likert scale perception responses with some additional flexibility from open ended responses. The KIIs allowed for more flexibility utilizing a semi-structured interview guide where interviewers could adapt questions based on the respondent and probe responses for more detail and nuance. Additionally, the evaluation team observed a monthly meeting of the NTFC and conducted a site visit to one port to observe import and export procedures and speak with staff in the customs, SENAVE (National Vegetable and Seed Quality and Health Service), and SENACSA (National Livestock Quality and Health Service) offices. The team designed and implemented evaluation activities intentionally to balance the gathering of evidence with the protection of human subjects, including safeguarding the dignity, rights, safety, and privacy of participants. The team applied ethical principles in all stages of the evaluation, including raising and clarifying ethical matters with stakeholders during data collection. Survey and KII respondents were provided explanation of the use of data and the assurances and limitations of confidentiality, and they were informed of their right to refuse to answer any of the questions or not to participate. No findings are attributed to respondents by name. The survey and KIIs targeted the participants in the project and in particular the members of the National Trade Facilitation Committee (NTFC), which includes representatives from public institutions and from private sector associations. The evaluation team intentionally focused on public and private institutions that have directly participated in T-FAST activities, and incorporated NTFC members or other stakeholders as well. Additionally, each data collection method included representatives of international cooperation or other agencies supporting trade facilitation with familiarity of Paraguay’s trade facilitation needs and the T-FAST project’s response (e.g., UNCTAD and the Global Trade Helpdesk). 2.3. Data Collection and Analysis Methods and Target Respondents The evaluation team employed three data collection methods to gather necessary information from key stakeholders and other sources. Analysis and findings from each method allowed the team to tie together its inquiries to gain a broad understanding of T-FAST’s performance. The data collection methods supplement one another on evaluation criteria where each method is appropriate, and complement one another where one method is more appropriate to fill in gaps of the other methods. Evidence from the surveys, KIIs, and document review were organized following the principles of contribution analysis to come to reasonable and robust conclusions about the contribution being made by T-FAST to the observed results. 2.3.1. Electronic Survey The online survey comprised 27 questions in 6 sections, including respondent information, background knowledge of T-FAST, project relevance, project effectiveness, project sustainability, and challenges for the project. Among the background information collected respondents provided the type of institution they represent (public, private, or other) and their gender in order to disaggregate responses as appropriate. The survey was primarily composed of closed-ended and Likert scale perspective questions to provide targeted information for answering the evaluation questions and to provide additional supporting data to the qualitative KII results. The Likert scale questions seeking respondent perceptions of priorities, objectives, activities, results, and other areas of T-FAST’s work used a scale from 1 (Extremely Poor) to 7 (Excellent) with 4 as the neutral response, and included a “Do not know” option where appropriate. For the analysis, the evaluation team considered scores of 1 and 2 as “Low”; 3, 4, LINC LLC Page 17 T-FAST Project Instituto Desarrollo Midterm Evaluation Report and 5 as “Medium”; and 6 and 7 as “High.” Open-ended questions asked for the alignment of project activities, objectives, and results with organizational priorities of respondents or recommendations on potential activities and improvements. The survey is included in Annex 3. The online survey was launched the week of March 21, with reminders sent electronically during the following two weeks. Additionally, the evaluation team followed up by phone with institutions that had not responded to further increase the response rate. The evaluation team stopped accepting new responses on April 23, with responses from 33 individuals representing 28 institutions (26 institutions that are members of the NTFC and 2 international agencies/projects). The profile of respondents to the survey is included in the table below. T-FAST provided the evaluation team with information on potential beneficiaries that the survey could be sent to. These beneficiaries included 139 individual contacts from 41 public and private sector institutions. Of those, 105 individuals were from 27 public sector institutions and 34 individuals were from 14 private sector entities. As the electronic survey was sent to all beneficiaries identified by IESC for potential response, it is considered a census methodology. This is distinct from a sampling method, which targets a representative cross-section of a typically larger beneficiary population to meet a particular confidence level target. As with most censuses, the evaluation team did not expect a 100% response rate. Based on the experience at baseline, the team expected an approximately 70% response rate from the 41 institutions (responses from a total of approximately 28 institutions). The total number of responses from beneficiaries was lower than expected, however, at 63% (a difference of two fewer institutional members than expected). The evaluation team was able to draw inferences from both responding and non-responding members of the population. Six of the institutions not responding to the survey were included in KIIs though, which provided the evaluation team with more confidence that their feedback and input is included in the conclusions. Note that this survey was different from the survey implemented at baseline and which has been repeated by T-FAST for custom indicator 1.1 (Percent of stakeholders who perceive the NTFC is enacting their agenda to strengthen trade facilitation in Paraguay). Table 6: Individual and Institutional Survey Participation by Sector and Gender Sector Individuals Institutions 12 Public Sector 21 (11M/10F) 18 Private Sector 11 (5M/6F) 9 Other 1 (0M/1F) 1 Total 33 (17M/16F) 28 The evaluation team analyzed the results of the survey primarily reviewing the responses to closed￾ended questions. The data were examined for distribution of responses and patterns based on the type of institution the respondent represented and the gender of the respondent. For open-ended questions, 12 Some individuals represent more than one institution. This table only shows the representation they indicated as their primary institution in the survey response. LINC LLC Page 18 T-FAST Project Instituto Desarrollo Midterm Evaluation Report data were reviewed and grouped around emergent themes identified by the analyst. Findings from the survey were supplemented by and compared with findings from the KIIs to drive evaluation conclusions. 2.3.2. Focus Group Discussions The evaluation team had originally planned to conduct a total of five FGDs with NTFC representatives of the public and private sectors. The objective of FGDs was to gain additional insight through the interaction of multiple project stakeholders during the discussions. In the initial effort to organize the FGDs, the evaluation team had low response rates and some direct unfavorable feedback from NTFC leadership and members indicating that they did not wish to participate in such group discussions, primarily due to time constraints. As a result, the team reassessed the data collection plan and needs and adapted them, deciding to eliminate FGDs and replace them with additional KIIs. 2.3.3. Key Informant Interviews The evaluation team designed the KIIs to be conducted as semi-structured interviews with key stakeholders to the project and other experts with knowledge of the Paraguayan context, needs for improved trade facilitation, and the T-FAST project. The KII guide instrument included 12 open ended questions seeking to elicit responses related to how T-FAST responds to the evaluation criteria. The questions largely followed the evaluation criteria, including a question on engagement of women in the project and in the NTFC to ensure an understanding of gender issues. The semi-structured format allowed interviewers to adapt, add, or skip questions, or probe for more details and nuance, as appropriate for the respondents’ knowledge and experience. Most of the interviews were conducted with two interviewers to further allow for flexibility in probing respondent answers for clarification. Additionally, several of the interviews included multiple individuals from the institution or office where the respondents felt their colleagues could add detail or had experience with T-FAST that they could contribute to the conversation. The KII semi-structured interview guide is included in Annex 3. The evaluation team had originally planned to conduct approximately 10 KIIs with representatives of the public and private sectors and of other stakeholders such as international cooperation and donor projects. With the decision to replace FGDs with additional KIIs, the evaluation team conducted more than twice the initial target for KIIs. The team also conducted an interview with T-FAST staff and with some non-NTFC representatives who were staff of public institutions participating in T-FAST activities. Table 7: Individual and Institutional KII Participation by Sector and Gender Sector Individuals Institutions Public Sector 24 (11M/13F) 15 Private Sector 4 (0M/4F) 4 Other 12 (6M/6F) 6 Total 40 (17M/23F) 25 The evaluation team analyzed the qualitative data collected through the KIIs using a process to code, compare, and synthesize the information. The evaluation team documented narrative responses at a sufficient level of detail to allow a systematic content analysis of these qualitative data. Narrative reviews of interview responses provided an in-depth understanding of project stakeholder experiences and perceptions. Notes from each interview were collected as excerpts which were then coded against the evaluation criteria and emergent sub-criteria. This approach allowed for both within interview and LINC LLC Page 19 T-FAST Project Instituto Desarrollo Midterm Evaluation Report cross-interview comparisons. In this way, key pieces of evidence from the various interviews were compared and triangulated to identify the main findings from the KIIs. As the KIIs were semi-structured, content of responses included some variation across interviews in depth and themes. 2.3.4. Document Review The evaluation team reviewed the project’s annual workplans and semi-annual reports, indicator reports and supporting data, financial reports, the revised Performance Monitoring Plan (PMP), and other documents provided by the T-FAST team. An initial phase of the document review provided the team with sufficient background knowledge about the project and its activities to inform the design of the other data collection instruments, namely the survey and the semi-structured interview guide. A more detailed document review during and after the fieldwork allowed the team to fill in gaps in information that could not be collected through the survey or KIIs, and was used in some cases to supplement, confirm, or contextualize the data and findings captured through other sources. The team conducted a structured comparison of the annual workplans with the semi-annual progress reports. The indicator tables and supporting data informed the team’s review of the use of data compared to PMP descriptions and to determine the potential scenarios for achieving the targets set for each indicator. The financial reports allowed a comparison of budget and expenditure. The three sets of documents combined gave insights into the effectiveness and efficiency of the project. 2.4. Evaluation Limitations The evaluation team has noted some limitations that may be relevant to the review of findings of this midterm evaluation report. • Elimination of Focus Group Discussions. The evaluation team had designed the midterm evaluation to include FGDs. Facilitating discussions with multiple stakeholders often allows new themes to emerge from the interaction that may be missed with individual interviews. The team attempted to mitigate the impact of this change by conducting more than double the planned number of KIIs, and in many of the KIIs multiple representatives of the same institution participated which did allow for some additional interaction. • Low Response Rate for Survey. The evaluation design included use of the survey as a census rather than a sample. The team had expected a response rate of approximately 70% of institutions (28 institutional members of the NTFC). After sending out the survey and multiple reminders, the evaluation team made phone calls to follow-up and encourage responses. Through this process, the response rate increased from an initial 27% up to 63%, still below the target. The team felt that the effect of the lower-than-expected response rate was mitigated by including in the KIIs six of the key institutions that were not represented in the survey. • Reliance on Perception Data. Due to the nature of the T-FAST activity, much of the data collected and analyzed through the survey and KIIs was based on stakeholder perceptions of the project. Stakeholder perceptions are established through their experience with T-FAST and past experience, and the questions posed sought to draw out that experience. The responses on perception were probed by KII interviewers and supplemented by data from the document review. • Some Project Performance Indicators are Collected and Reported Infrequently. The evaluation team did not directly measure any of the project’s indicator values from its PMP, and several LINC LLC Page 20 T-FAST Project Instituto Desarrollo Midterm Evaluation Report indicators, such as Average Release Time for Agricultural Goods and Percentage of Shipments Assigned to the Green Channel for Imports, had not been measured frequently. However, the evaluation team did conduct a review of the indicator data collected by the project to confirm accuracy. Additionally, as mentioned in the previous bullet, the evaluation team probed stakeholders for their perception of progress against key indicators such as the one on Average Release Time. LINC LLC Page 21 T-FAST Project Instituto Desarrollo Midterm Evaluation Report 3.Findings This section presents the analysis and main findings from quantitative and qualitative data collected during the midterm fieldwork. The findings are presented by data collection method, including a subsection on findings from the survey, a subsection on findings from the KIIs, and a subsection on findings from the document review. The three methods supplement and complement one another on the evaluation criteria of relevance, effectiveness, efficiency, sustainability, impact, and challenges. The survey questions specifically covered the respondents’ perception of the relevance, effectiveness, sustainability, and challenges of the project. The KIIs asked interviewees about relevance, effectiveness, efficiency, sustainability, impact, and challenges. The document review focused on effectiveness, efficiency, and challenges of the project. The main objectives of the midterm data collection were to gather information and understanding of the salience, performance, and results of the T-FAST project for the key public and private sector stakeholders of foreign trade in Paraguay. The analysis focused on how the project is performing in relation to the highest perceived priorities to facilitate foreign trade, including meeting the requirements of the WTO TFA and alignment with priorities of public institutions and private sector associations. Additionally, the evaluation team reviewed the T-FAST project’s progress against its plans, indicator targets, and the baseline recommendations. The data analysis process supported a contribution analysis methodology to understand associations and links between project activities and observed results using the evidence generated from the evaluation. 3.1. Findings from the Survey The evaluation team designed and employed the midterm evaluation online survey to provide a foundation on the perceived value, sustainability, and challenges of the T-FAST project. The survey targeted representatives from all of the institutional members of the NTFC (N=41) but the response rate did not reach 100%, with 26 NTFC member institutions represented. While this is lower than expected, the findings are informative and supplemented by data from the KIIs (in the next section), including KIIs from six NTFC member institutions that did not respond to the survey. 3.1.1. Respondents The survey respondents included a mix of representatives from public institutions and the private sector, as well as a mix of women and men. The male/female breakdown was close to even. While there were more public institutions represented than private sector entities, the NTFC has more public sector members than private sector members (approximately 66% and 34% respectively in the NTFC, the survey proportions were 65% and 35%). Table 8: Individual and Institutional Survey Respondents by Gender and Institution Type Individuals Type Male Female Total Institutions Public Sector 11 (33%) 10 (30%) 21 (64%) 18 (64%) Private Sector 5 (15%) 6 (18%) 11 (33%) 9 (32%) Other 0 (0%) 1 (3%) 1 (3%) 1 (4%) Total 16 (48%) 17 (52%) 33 (100%) 28 (100%) LINC LLC Page 22 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Respondents indicated they were aware of the work of the T-FAST project, with the majority (64%) having been introduced to the project through a formal project presentation. Other channels (primarily NTFC meetings or by email) were cited by 27% of respondents, and informal conversations by 21%. Some respondents indicated learning of T-FAST through multiple channels. Table 9: How Survey Respondents Learned of T-FAST Channel Percentage Formal presentation of the project by the project team 64% Read the official project document 9% Social media, website 9% Media 3% Informal conversations about the project with colleagues 21% Other (primarily NTFC or email) 27% Most of the respondents (78%) reported that they or their institution had received at least one type of support from the T-FAST project, with 22% indicating no direct support. Training was the most common type of support received (44%) with trade process support (25%) and other support (28%, primarily through support to the NTFC or other organizational strengthening) as other common responses. Note that some respondents (25% of all respondents) indicated multiple types of support. For this question, one respondent did not answer, so n=32. Figure 1: Type of Support Received by Survey Respondents 3.1.2. Relevance Closed-ended survey questions on the relevance, effectiveness, efficiency, sustainability, and challenges of the project used a seven-point Likert scale (1=Extremely Poor, 4=Neutral, 7=Excellent). For the LINC LLC Page 23 T-FAST Project Instituto Desarrollo Midterm Evaluation Report analysis, the evaluation team considered scores of 1 and 2 as “Low”; 3, 4, and 5 as “Medium”; and 6 and 7 as “High.” Respondents were also given the option to answer “I don’t know” to any question. Four questions focused on relevance of T-FAST. The first two asked to what extent the results and completed activities (respectively) of the project align with the challenges faced by the respondent’s institution. The third and fourth asked to what extent the T-FAST project’s activities aligned with the TFA requirements and complemented or supported other relevant projects (respectively). Almost all responses indicated that the project aligns either partially or completely. In terms of alignment with the TFA and complementarity to other projects, 67% and 70% of respondents respectively indicated “high.” In general, public sector respondents were more likely to rate the project’s alignment and complementarity as “high” than private sector respondents. Table 10: Survey Respondents’ Perception of Alignment of T-FAST to Priorities Overall Public Sector Private Sector Low Med High Don't Know Low Med High Don't Know Low Med High Don't Know Alignment of results to organizational challenges 0% 39% 45% 15% 0% 33% 48% 19% 0% 55% 36% 9% Alignment of activities to organizational challenges 3% 39% 42% 15% 5% 38% 48% 10% 0% 45% 27% 27% Alignment of activities to TFA needs 0% 21% 67% 12% 0% 24% 67% 10% 0% 18% 64% 18% Complementarity of activities to other projects 0% 18% 70% 12% 0% 24% 71% 5% 0% 9% 64% 27% In open-ended questions asking respondents about the relationship between T-FAST expected results and activities and the priorities of the respondents’ organizations, several themes were frequently cited. Those include digitalization of processes and paperwork, simplification of procedures, reduced time for clearance in foreign trade, improved accessibility to information, and improved coordination across public institutions and with the private sector. 3.1.3. Effectiveness Respondents answered closed-ended questions regarding the effectiveness of the project. The first was a series of responses to gauge the perceived effectiveness of 10 specific activities of T-FAST. The majority of respondents indicated that all of the selected activities were either highly effective or partially effective. The activities most consistently rated as high effectiveness were the implementation of the digital platform for the NTFC (67%), training of laboratory professionals (64%), and jointly developed action plans with key public sector NTFC member institutions (61%). Responses were similar for public and private sector institutions, however, the private sector even more frequently rated the digital platform (73%) and joint action plans (73%) as highly effective. It is also important to note that a relatively high percentage of respondents indicated not knowing about the effectiveness of some activities, in particular the coordination with binational chambers of commerce (27%). LINC LLC Page 24 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Table 11: Survey Respondents' Perceived Effectiveness of Specific T-FAST Activities Overall Public Sector Private Sector Low Med High Don't Know Low Med High Don't Know Low Med High Don't Know Digital platform to support NTFC members' coordination, publication, etc. for public comment 0% 21% 67% 12% 0% 24% 62% 14% 0% 18% 73% 9% Jointly developed action plans with NTFC members 0% 18% 61% 21% 0% 19% 52% 29% 0% 18% 73% 9% Training courses for NTFC members 0% 27% 58% 15% 0% 33% 52% 14% 0% 18% 64% 18% Support to NTFC working groups 0% 24% 58% 18% 0% 24% 52% 24% 0% 27% 64% 9% Development and implementation of ICT software and collaborative process optimizations with selected institutions 3% 24% 58% 15% 5% 24% 52% 19% 0% 27% 64% 9% Coordinating activities with 13 binational chambers of commerce, and the largest business associations of the country 0% 27% 45% 27% 0% 29% 38% 33% 0% 27% 55% 18% Training to laboratory professionals 0% 15% 64% 21% 0% 14% 67% 19% 0% 18% 55% 27% T-FAST facilitated electronic payments and installed ICT server hardware on which digitalized processes are being programmed and implemented 0% 15% 61% 24% 0% 14% 62% 24% 0% 18% 55% 27% T-FAST developed the first NTFC monitoring plan and a communications plan that addresses the needs of the NTFC’s 45+ members 0% 27% 52% 21% 0% 24% 52% 24% 0% 36% 45% 18% T-FAST supported the process to achieve complete adoption of the Authorized Economic Operator (AEO) program at regulatory institutions 0% 27% 48% 24% 0% 29% 43% 29% 0% 27% 55% 18% Survey respondents largely indicated that they did not know if the T-FAST project was effective at reaching women (55%), though for respondents who provided a rating, responses indicated high (27%) and medium (18%) effectiveness. Respondents indicated moderate to high confidence that T-FAST would be effective at achieving its target for reducing time to trade (39% high, 39% medium, and 21% did not know). Table 12: Survey Respondents’ Perception of T-FAST Effectiveness at Reaching Women and Reducing Time to Trade Low Med High Don't Know Effectiveness of T-FAST at reaching women 0% 18% 27% 55% Effectiveness of T-FAST for reducing average time to trade 0% 39% 39% 21% In open-ended questions, respondents were asked to list up to three activities that T-FAST could invest in to improve the reduction of average time to trade. The most cited activities were continued digitalization of documentation and processes, streamlining of procedures and reduction of bureaucracy, improved availability of information and transparency in communication, improved IT LINC LLC Page 25 T-FAST Project Instituto Desarrollo Midterm Evaluation Report infrastructure, and continued expansion of the Authorized Economic Operator (AEO) program (see text box below). 3.1.4. Sustainability Respondents answered closed-ended questions on the extent to which the T-FAST project’s results to date will be sustained in the long-term. Survey responses indicate that the majority of respondents believe it is highly likely that the results will be sustained (61% overall, 62% of public sector, 55% of private sector). Table 13: Survey Respondents’ Perception of Sustainability of T-FAST Results to Date Low Med High Don't Know To what extent will T-FAST results be sustained 0% 21% 61% 18% In open-ended questions, respondents were asked to list up to three factors that would indicate or increase the likelihood of sustainability of results after the project ends. The responses were quite diverse, falling into several thematic areas: • Strength of the NTFC – including capacity building for the NTFC and its leadership, strengthening or reviewing the legal framework of the NTFC, improving systems and tools available to the NTFC, etc. • Availability of Financial Resources – for the NTFC and for the sector overall, including commitments from public and private institutions and identification of other resources from other sources. • Political Will and Commitment – including written and demonstrated commitments from public institutions and sector leaders, as well as more consistent participation and reduction of turnover. • Continued Digitalization and Investment in IT – indicating that more availability and use of IT systems will increase reliability on those systems, making IT improvements more sustainable. • Transparency and Improved Communications – including continued work to develop and expand communications platforms and to make data and information more accessible to interested parties. Authorized Economic Operator Program According to the T-FAST project, the AEO program is a modern and smart control strategy that develops and fosters security and compliance through the move from transaction-based controls to system-based controls based on strong partnerships with stakeholders throughout the supply chain. Processes, procedures, systems, competencies and security measures are verified by Customs at the company level and benefits are provided to traders to facilitate their transactions. AEO certified companies are given a preferential treatment for trade operators, including a guaranteed 95% green channel for all their Customs releases. The AEO program requirements are described in Article 7.7 of the TFA. LINC LLC Page 26 T-FAST Project Instituto Desarrollo Midterm Evaluation Report 3.1.5. Challenges Survey respondents answered two closed-ended questions to determine the extent to which external factors negatively or positively impacted the project. A relatively high percentage of respondents indicated that they did not know whether any external factors influenced the project (24% for each question), while 42% of respondents answered neutrally to each question. A small number of respondents (6%) felt that external factors did not negatively impact the project. Table 14: Survey Respondents' Perception of the Impact of External Forces Low Med High Don't Know To what extent have external forces NEGATIVELY impacted project 6% 42% 27% 24% To what extent have external forces POSITIVELY impacted project 0% 42% 33% 24% In open-ended questions, respondents were asked to list up to two external factors that had a negative impact on the project and up to two factors that had a positive impact on the project. The most commonly cited factors with a negative impact were the pandemic, turnover of key public sector staff or NTFC representatives, and low use or access to IT infrastructure. The factors with a positive impact were more diverse, but several commonly cited factors were the move to a virtual environment due to the pandemic, increased international demand and economic rebound as the pandemic continued or restrictions eased, and the high interest and participation of the private sector in trade facilitation. 3.2. Findings from the KIIs The evaluation team conducted KIIs using a semi-structured interview approach to gain a deeper understanding and supplement the data collected through the online survey. The KIIs targeted both the public and private sector institutions most directly participating in T-FAST and some public and private institutions and external agencies that are well-aware of the project but who do not directly benefit from the activities. 3.2.1. Relevance Priorities and T-FAST Response Interviewees consistently indicated several top priorities for trade facilitation in Paraguay. Those included digitalization of processes, simplification of processes and procedures, and increased coordination across public institutions. As described by interviewees, digitalization of processes refers to accessibility to the documentation and processes required for import or export of goods and to transparent information about trade and trade processes. Interviewees reported that this was an area in which T-FAST has been providing substantial support, in particular, through the provision of Information Technology (IT) hardware and software and relevant training. Interviewees noted that many of the processes and procedures for foreign trade, including the pre￾customs and customs processes, were unnecessarily complicated and lengthy. In describing the priority for simplification of processes and procedures, interviewees often described reviewing existing process flows and streamlining the steps. Some interviewees specifically mentioned steps that appear to be LINC LLC Page 27 T-FAST Project Instituto Desarrollo Midterm Evaluation Report unnecessary or that are duplicative. Some interviewees also indicated their perception that individual steps in the import or export procedures for some goods are sometimes controlled by different offices and that some steps could be combined into another step for simplification. In other words, interviewees indicated that steps in the import and export processes can be reduced – a perception highlighted by both private sector and public sector key informants. Interviewees reported that T-FAST has conducted studies, made recommendations, and supported improvements for simplification. Interviewees indicated that increased and improved coordination across public institutions was a high priority for facilitating trade, with some interviewees also indicating that increased involvement of private sector in the coordination was a priority as well. The NTFC as a body and the working groups within the NTFC are the primary vehicle through which interviewees felt the priority is being addressed and noted that T-FAST has provided support to the NTFC on coordination and organization of efforts. Several additional priorities were also named, though less frequently, including reduction of clearance times, improved communications, and awareness raising for the private sector. In general, interviewees who highlighted those priorities also indicated that T-FAST activities have supported those as well. A subset of interviewees highlighted additional priorities for facilitation of Paraguay’s foreign trade that are outside of the TFA priorities and outside of the scope of T-FAST. One of those is the considerable infrastructure needs unique to a landlocked country, in particular, waterway infrastructure and land border infrastructure. A second priority that does not align directly with the TFA requirements is a focus on improving facilitation of export rather than only import, with some interviewees noting that the TFA appears to be more concerned with import. That said, the interviewees who noted that priority also felt that most improvements to trade facilitation would support both export and import. T-FAST Relevance to Other Activities Beyond the perceived and stated priorities of the key stakeholders to foreign trade in Paraguay, the WTO TFA has its own priorities and there are other projects from international agencies to support foreign trade in Paraguay. Interviewees reported that the TFA Category C Notifications are generally aligned with stakeholder priorities, in particular, around transparency of information and communications, prior consultation of the private sector, and average release time. Interviewees indicated that T-FAST has been actively supporting the NTFC and specific members to respond to the TFA requirements. The NTFC communications platform that is currently under implementation, the digitalization and process streamlining, and the support to fully implement the Authorized Economic Operator (AEO) program were all mentioned as relevant T-FAST support. Interviewees, including those from NTFC member institutions and from international agencies, indicated that T-FAST has closely coordinated with other projects that are supporting trade facilitation or the NTFC. In particular, T-FAST has been able to fill in gaps where other projects were not operating (such as supporting career path analysis and HR improvements), build on the activities and recommendations of other projects (such as the Empowerment project supported by the UN Conference on Trade and Development, UNCTAD which proposed the formation of working groups that T-FAST has subsequently supported), and to complement the activities of other projects. Interviewees noted that the flexibility of LINC LLC Page 28 T-FAST Project Instituto Desarrollo Midterm Evaluation Report T-FAST has allowed it to avoid duplication and provide that complementary support, for example, providing HR support requested by some institutions to complement other projects’ assistance. T-FAST staff have suggested that three mechanisms may be driving the flexibility that interviewees reported. The language of the annual workplan is less prescriptive than other donor-funded projects working on trade in Paraguay allowing for more ability to adapt an activity to the needs of the project participants. The T-FAST project Steering Committee, composed of representatives from the project, IESC, USDA, and the US Embassy, meets every three months and gives the project leadership an opportunity to discuss shifting needs and quickly adapt to address those needs. Activity 5 of T-FAST, the on-demand mechanism, is designed to give the project flexibility to respond to issues that arise during implementation. While Activity 5 had not been used during the first half of the project (the period covered by this Midterm Evaluation), participants are aware of it. 3.2.2. Effectiveness Perceived Strengths and Weaknesses Overall, interviewees indicated that the T-FAST project has a wide range of strengths and very few weaknesses. Key strengths named by interviewees included expertise of the program staff and consultants, support provided to the NTFC13 , transparent operations, and activities that have supported digitalization, streamlined procedures, installation of IT hardware and software, organizational structure and human resources, and training. Additionally, interviewees pointed out that while it is unusual to see a project support such a diverse set of activities, that has been a strength of T-FAST, allowing the project to build momentum across multiple institutions and priorities. Interviewees also noted that while T-FAST has provided direct support to only a subset of public institutions (e.g., DINAVISA, SENACSA, SENAVE, INAN, INTN), those have been the highest priority institutions to focus on. Interviewees also noted a key perceived strength as the flexibility of the project to address needs as they arise. This includes adapting activities and plans based on changing needs, identifying additional needs and opportunities to support the NTFC and specific institutions, and responding to requests for support. In general, interviewees also stated that the project is extremely responsive to participants and their requests or communications. Though interviewees indicated rapid responses for most requests, one interviewee noted that some procurement processes were slower than expected. Two weaknesses were noted, though only by a small number of interviewees. One is the length of the project in which some interviewees indicated concern that the project might end at an inopportune time and before the NTFC will be able to fully take on the responsibility that T-FAST currently has. Accompanying the comments on time, some interviewees also noted that the onset of the COVID-19 pandemic slowed the start of T-FAST and delayed some activities. Mitigating this stated weakness, some interviewees noted that the challenge of turnover of NTFC representatives and leadership, for example after elections which are set to take place near the close of the project, is beyond the control of T-FAST. Some also noted that there have been projects supporting the NTFC and trade facilitation before T-FAST 13 As summarized in section 1.2 Project Description, T-FAST support to the NTFC includes two staff embedded at the MRE a variety of technical assistance related to NTFC meetings and planning, development of communications plan and monitoring plan including a monitoring dashboard, establishment of thematic working groups within the NTFC, and organizing and support regular meetings. LINC LLC Page 29 T-FAST Project Instituto Desarrollo Midterm Evaluation Report and there is reason to believe there will be opportunities for other international support to continue after T-FAST ends. The second weakness indicated by some interviewees was also reported as a strength by other interviewees: communications. As a weakness, interviewees highlighted the need to raise communications to a higher level within the public sector and at times a lack of communication from the representatives on the NTFC to their colleagues (superiors, peers, and subordinates) who might be affected by or influential on the activities of the NTFC and the project. As strengths, interviewees noted the improvement in regular communication from the NTFC, transparency in the operations and budget of T-FAST, and the focus on the NTFC platform to increase information flow and opportunities for consultation. Average Time to Trade Interviewees consistently reported that the ambitious target of 30% reduction in time to trade is achievable within the period of the T-FAST project. Interviewees from both private and public sectors indicated that time to trade had already been reduced appreciably, and even that some of the already implemented activities would continue to reduce time further than has been observed so far.14 Some interviewees with a “client facing” responsibility also noted that they had received positive feedback from companies and individuals directly involved in foreign trade about the improvement in procedures and digitalization. Interviewees pointed to several activities in particular as contributing to a reduction of time to trade, and highlighted opportunities for further reduction. The support to digitalization15 was the most named activity in supporting this objective, with many interviewees noting that despite great progress, there are still many opportunities for further digitalization of processes and documentation, including numerous documents and approvals that can still be automated or moved online. The IT hardware and software that has contributed to this activity can also be further expanded. Other activities that interviewees indicated have contributed to reduction in time to trade include: • The roll-out of the AEO program. 16 • Studies on process flows and subsequent efforts to address inefficiencies. • Human resource studies and consulting to improve organizational structure and career paths within specific public institutions.17 Some interviewees also highlighted that while the T-FAST project and the NTFC are on track to achieve the target reduction in time to trade, the results are still quite dependent on political will and 14 This stakeholder perception cannot be compared to actual results per the indicator data as the project performance indicator 2.1 “Average release time for agricultural goods” was measured at baseline and was next scheduled to be measured in Year 3. 15 As summarized in section 1.2 Project Description, T-FAST support to digitalization includes conducting assessments, developing action plans, providing equipment and servers, installing software, funding training, and negotiating licensing fees. 16 The Authorized Economic Operator program is a risk management strategy that develops and fosters security and compliance through partnerships with stakeholders along the supply chain for import and export goods. 17 Studies on processes and on human resources issues were conducted directly by T-FAST staff or by consultants engaged by the project. LINC LLC Page 30 T-FAST Project Instituto Desarrollo Midterm Evaluation Report commitment from the relevant public institutions. In particular, the current momentum of the NTFC is high, but interviewees felt that is a potential risk for the future, especially if there is turnover in key positions. Also, some important improvements will require support through legal framework changes or policy changes that depend on factors outside of the control of T-FAST. Additionally, some improvements will require continued focus and willingness on the part of the specific public institutions they relate to, including the National Customs Authority (DNA), the Single Import and Export Windows (VUI and VUE respectively), and others that have been very involved in the project to date. Inclusion of Women Interviewees consistently reported that women were well-represented in trade facilitation in Paraguay. Many pointed to the large number of women in key positions for foreign trade, including in the relevant public sector institutions, in private sector associations, and in private companies. Interviewees specifically noted the active leadership of women representatives in the NTFC and in their institutions, and their involvement in decision-making. One area where this is not the case, as noted by one interviewee, is the IT sector. T-FAST has provided considerable support for IT in several institutions, and there are fewer women involved in IT in Paraguay, thus their perception is that this is an area where women are less represented within the project. 3.2.3. Efficiency Cost/Benefit of Activities and Results Interviewees generally noted that they felt the project is providing value proportionate to the resources available, accomplishing a lot for the total budget. The transparency of budget and allocations gave interviewees insight into the efficiency of the use of resources. Interviewees noted their perception that many of the activities of T-FAST are managed efficiently and that the project follows through on its planned tasks. Interviewees also highlighted the flexibility of the project to adapt to needs, identify new opportunities, and/or respond to institutional requests as an example of efficiency. Two specific critiques arose to efficiency from individual interviews. One was the limitation of the scope of the T-FAST project to priorities closely aligned to the TFA when there are some other priorities that could be addressed to improve facilitation of trade (e.g., infrastructure needs). The other was the sometimes-long process for some procurements to be approved. 3.2.4. Sustainability Interviewees indicated that they believe that most if not all of the activities and results already achieved through T-FAST and the NTFC are sustainable. Interviewees stated that this belief is primarily due to the types of activities and results seen to date. In particular, the technology improvements for digitalization and the streamlining of processes or organizational structures have reduced time and cost for those involved in foreign trade and made the work of the supported institutions easier. The importers and exporters involved in foreign trade and public institutions with responsibilities for customs or other foreign trade processes have no reason to change their processes or procedures back to the way they were before, so the efforts are seen as sustainable. Interviewees noted that the support to the NTFC itself, through formation of working groups, development of plans and indicators, and other operational aspects, has also improved the work and LINC LLC Page 31 T-FAST Project Instituto Desarrollo Midterm Evaluation Report relationships of representatives. While the work of the NTFC does require a time commitment from those representatives, interviewees felt that the improvements have made their involvement more efficient and thus they are likely to be sustained. Training, capacity building, and knowledge sharing are helping to internalize many of the improvements and support sustainability as well, though the training activities and results are more difficult to sustain due to required budgets and potential for turnover of individuals trained. To maximize sustainability, interviewees highlighted a need for further commitment from leadership in the government and from each of the public institutions. Interviewees noted opportunities for sustainability planning, financial or other resource commitments from stakeholders, and monitoring efforts that can help with NTFC management and activity follow-through. Key informants cited concerns that without a stronger commitment from government leadership to maintaining the NTFC and commitments from stakeholders, the NTFC will have a more difficult path to sustainability. Further, some informants suggested that the current legal framework under which the NTFC operates and its mandate may need to be revised and updated from time to time to support long-term sustainability. This type of review and revision has already taken place once, so there is precedent to continue the practice. Many of the interviewees expressed confidence that the project will achieve sustainable results through the capacity building activities in each key institution. 3.2.5. Impact Interviewees generally felt that the T-FAST project is on target to achieve a long-term impact on trade facilitation and on trade of agricultural goods. As of the midterm evaluation, most of the responses of interviewees are focused on potential opportunities to contribute to and improve impact over the long￾term. These key contributors and opportunities include: • Capacity building at key institutions (public and private). • Capacity building of the NTFC overall, in particular the leadership training in progress. • Continued support for IT infrastructure and equipment, across institutions. • Continued support to improve coordination between the NTFC, the working groups, and the institutional members. 3.2.6. Challenges The most cited challenge to the T-FAST project is the COVID-19 pandemic. Interestingly, interviewees highlighted a number of positive and negative effects of the pandemic on the project. On the positive side, they noted that it increased the urgency and momentum for digitalization and generally moving more of the processes and information online. Interviewees also believed that it increased the participation in NTFC meetings and other events because it was easier to join without a commute. This belief included a perception that participation came from a higher level of representation within relevant institutions than was likely had the meetings been in person. Interviewees perceived negative effects of the pandemic as well. They felt the project was delayed overall and that some specific efforts were delayed or slowed due to the inability to visit sites and meet with key counterparts in person. Some of the initial delay was due to the project responding to the changes in context and restrictions put in place due to the pandemic, and the project did work to adapt to the new reality. One specific negative effect of the delay was noted by several interviewees, the LINC LLC Page 32 T-FAST Project Instituto Desarrollo Midterm Evaluation Report timing at start up may have affected the project’s coordination with the UNCTAD Empower project which was conducting some relevant studies. Interviewees did note that the two projects’ coordination greatly improved later. The other often cited challenge to the project and to trade facilitation was turnover in key positions, primarily in the public sector. This has included some key counterparts at MRE, including those very involved in the NTFC, that affected the continuity of knowledge and activities. The project has been able to immediately work with newly appointed or hired staff, but the delays or in some cases setbacks were noted as difficult to manage smoothly even when the replacement is knowledgeable, experienced, and capable. Interviewees noted several other important external factors that have had negative effects on foreign trade overall, and on trade facilitation or the project more specifically: • Pandemic restrictions and shutdowns in Paraguay and other countries have affected the volume of trade. • The drought in Paraguay has had a large effect on agricultural production and trade. • Bureaucracy and corruption were both named as challenges to the effectiveness of the project. • Changes in diplomatic relations, including issues like the Russia-Ukraine war have posed challenges for trade. 3.3. Findings from Project Documents The evaluation team conducted a detailed review of several project documents in order to fill in gaps in information that could not be collected through the survey or KIIs. In particular, the team reviewed the project’s annual workplans and semi-annual reports, the indicator reports and data, financial reports, and revised Performance Monitoring Plan (PMP), as well as several other documents shared by the T￾FAST project team or created by the project team at the request of the evaluators. 3.3.1. Effectiveness Activity Progress, Challenges, and Adaptation Based on reviews of the workplans and semi-annual reports, during the first two years of the project, T￾FAST consistently made progress on activities. The project reported regular communications with key stakeholders, preparing the foundation for future work through knowledge transfer and training, collecting and analyzing information on trade facilitation needs and processes, and implementing ICT hardware and software procurement and installations. Examples of specific project actions progressing effectively per the reports include: • Assessments of trade facilitation environment for agricultural goods, and assessments of seven (four public and three private) quality control laboratories for agricultural goods. • Training and certification on laboratory protocols and best practices. • Action plan for design and development of an online platform for the NTFC to publish policy proposals for key public institutions and receive comments from the private sector.18 18 This platform is part of the effort for meeting the requirements of Article 2 of the TFA and the T-FAST project has supported its development and roll out including compatibility assessments, participatory workshops and technical design, procurement of hardware and software, and training. LINC LLC Page 33 T-FAST Project Instituto Desarrollo Midterm Evaluation Report • Training of technical staff in communication, change management, and analysis of comments on policy proposals. • Training assessment and provision of training for private sector associations to improve engagement in trade facilitation reform. • Promotion and expansion of the AEO program. • Execution of reorganization of the NTFC structure and establishment of working groups. The project also reported challenges in executing several specific actions but was generally able to adapt to mitigate the challenges or achieve goals through alternative approaches. The primary challenges requiring adaptation of activities and sub-activities arose due to the COVID-19 pandemic, bureaucratic or institutional obstacles, or assessments that identified unexpected obstacles and needs. Examples of specific project actions that adapted or arose from challenges include: • Delivering certain training courses using a hybrid in-person/virtual approach. • Recognizing a challenge of human resources attrition in several institutions, the project has supported institutions with reorganizations and career plans. • Support to SENAVE for cross-border relations to speed the control process ran into an obstacle when legal counsel decided to cancel the request. T-FAST was able to adapt by reallocating support to other activities. Planned vs. Executed Activities The evaluation team analyzed and compared the activities planned in each annual workplan with those executed according to the semi-annual reports. The analysis focused on the Year 2 plan and reports as in Year 1 no action was reported as completed (in large part due to delays from the onset of the pandemic). The analysis considered three stages of progress for activities: completed, in progress, and not initiated. Completed actions are those completed as planned for the project year in question. Actions in progress are those initiated but not concluded. Actions not initiated are those that are still in the planning process or not reporting any progress. The evaluation team determined level of progress based on the progress described in the T-FAST semi-annual reports. For this purpose, the team identified all actions by sub-activity according to the Year 2 Annual Work Plan and compared them to the progress of each action based on each semi-annual report. The level of progress is based on the criteria of the evaluation team and focuses specifically on the progress reported in Year 2, given that in Year 1 no action was reported as completed. Further, if the T-FAST semi-annual reports for Year 2 did not provide an update on a planned activity, the evaluation team considered this not progressed beyond the status from the previous report (if any). Therefore, the progress noted in Table 15 is an estimate by the evaluation team based on the data available from the document review. Nevertheless, this midterm evaluation presents an overview of the level of completion of planned actions as of the close of Year 2 of the project for the purpose of gaining additional perspective on the effectiveness and efficiency of T￾FAST. As of the end of Year 2, the sub-activities of Activity 1 varied in terms of their level of completion. The NTFC working groups reorganization and the assessment of trade facilitation environment for agricultural goods had the lowest proportion of completed actions. The sub-activities of Activity 2 showed a higher percentage and number of actions completed. Activity 3 shows more than 40% of actions completed with more than 40% in progress and a small amount to be initiated. The majority of LINC LLC Page 34 T-FAST Project Instituto Desarrollo Midterm Evaluation Report sub-activities of Activity 4 have been completed with the procurement of technological hardware and software for INTN, INAN, SENAVE and DINAVISA. The full set of actions and their status as presented in Table 15 below is included in Annex 5. Table 15: Progress vs. Plan by Sub-Activity19 Activity Sub-Activity Actions (in % of completion) Total # Completed In Progress Not initiated 1. Strengthen the National Trade Facilitation Committee (Capacity Building: Promote Improved Policy and Regulatory Framework) 1.1. Establish and Coordinate Steering Committee 2 100% 0% 0% 1.2. Conduct Initial Assessment of the Trade Facilitation Environment for Food and Agricultural Goods 5 20% 20% 60% 1.3. NTFC Working Groups Re-organization 16 19% 19% 63% 1.4. Build Core Capacity of the NTFC 19 53% 16% 32% 2. Support Process Modernization with Government Institutions and Agencies in Agriculture Import/Export Processes 2.1. Support NTFC related to Publication/Opportunity to Comment 4 100% 0% 0% 2.2. Support NTFC related to Laboratory Testing 12 42% 42% 17% 2.3. Support NTFC related to Time Release and Expedited Shipping 44 43% 16% 41% 2.4. Support NTFC related to Border Cooperation 6 33% 50% 17% Other activities 2 50% 50% 0% 3. Engage Trade Associations to Promote Improved Trade Facilitation 3.1. Build Capacity of Private Sector to Engage in Trade Facilitation Reform 7 43% 43% 14% 4. Procure Software and Equipment to Improve Agriculture Import/Export Processes 4.1. Develop Infrastructure to Implement NTFC Priorities related to Publication/Opportunity to Comment 1 100% 0% 0% 4.2. Develop Infrastructure to Implement NTFC Informal Producers to Formal Market 1 100% 0% 0% 19 As Activity 5 had not started during the period evaluated by this midterm evaluation it is not included in this table. LINC LLC Page 35 T-FAST Project Instituto Desarrollo Midterm Evaluation Report 4.3. Develop Infrastructure to Implement NTFC Priorities related to Time Release and Expedited Shipping 9 78% 22% 0% Summary of Challenges for Activities Performed vs. Planned Activity 1: Strengthen the National Trade Facilitation Committee (Capacity Building: Promote Improved Policy and Regulatory Framework) 1.2. The sub-activity “Conduct Initial Assessment of the Trade Facilitation Environment for Food and Agricultural Goods” had a primary objective to conduct a Paraguay Agriculture Laboratory Capacity Assessment. KSU completed the technical evaluations of seven Paraguayan agricultural product control laboratories (four public and three private). As a result, a laboratory technical subcommittee was planned to be established to assist in the implementation of action plans. However, activities planned to conduct an air transportation import/export process could not be carried out due to restrictions implemented as a consequence of the COVID19 pandemic. 1.3. Regarding the sub-activity “Reorganization of the NTFC Working Groups,” T-FAST worked in close coordination with UNCTAD to establish three working groups, each led by a team of one public institution and a representative of a private sector association. Other actions such as facilitating a lab learning trip and facilitating an expedited shipping learning trip could not be achieved as planned due to restrictions imposed by COVID. T-FAST organized virtual learning opportunities via Zoom instead, and postponed the planned border integration learning trip to 2022. 1.4. For the sub-activity “Build Core Capacity of the NTFC,” one of the deliverables is to build NTFC Staff Structural and Operational Capacity. Among other actions within this sub-activity, the project hired a communication specialist who started working with the monitoring specialist. The monitoring specialist developed specific indicators to measure progress towards reaching objectives and maintained an updated project dashboard, tracking TFA-relevant working group initiatives against the TFA’s Category C Notifications for Paraguay. Activity 2: Support Process Modernization with Government Institutions and Agencies in Agriculture Import/Export Processes 2.2 Regarding the sub-activity “Support NTFC related to Laboratory Testing,” T-FAST has successfully addressed the majority of the tasks to be carried out. Actions completed included assessments of eight agricultural laboratories, training, and planning for ICT solutions. Several actions are ongoing, including implementation of action plans and provision of technical assistance. However, follow-up training and technical assistance, had not yet been initiated. 2.3 Under the sub-activity “Support NTFC related to Time Release and Expedited Shipping,” a number of technological support actions have been completed. Additionally, in response to an observation on human resource challenges for INTN, T-FAST supported the development of a Career, Job, and Salary plan that INTN has committed to adopt as policy and with budgetary support. Several actions, such as studies or the introduction of electronic systems, were slowed or delayed due to COVID or bureaucratic obstacles. LINC LLC Page 36 T-FAST Project Instituto Desarrollo Midterm Evaluation Report 2.4 The sub-activity “Support NTFC related to Border Cooperation” has made progress on almost all planned actions, with some studies and contracting of technical support completed. Only the effort to support SENAVE with legal and technical assistance related to reform of integrated control areas had not been initiated due to SENAVE’s decision to cancel the request. Other activities that supported T￾FAST Activity 2 overall, including signing agreements with various NTFC stakeholders were progressing or completed during Year 2. Activity 3: Engage Trade Associations to Promote Improved Trade Facilitation 3.1 With reference to the sub-activity “Build Capacity of Private Sector to Engage in Trade Facilitation Reform,” the project has made progress on several fronts. T-FAST assessed trade association training needs and began a series of Workshops that included technical and legal guidance for trade association staff on how to effectively propose new policies. The project also developed and supported implementation of a plan to promote the AEO program in coordination with the private sector. Activity 4: Procure Software and Equipment to Improve Agriculture Import/Export Processes All planned actions under the sub-activities of T-FAST Activity 4 were completed or in progress as of the end of Year 2. Activity 5: Manage On-Demand Mechanism for Improvements in Agriculture Import/Export Processes Activity 5 had not started during the period covered by this midterm evaluation, so no actions were planned or reported on by the project. Performance Indicator Analysis The evaluation team reviewed the T-FAST indicator tables, the semi-annual reports, and the backup documentation to analyze three aspects of the performance indicator progress: • Comparison between the results of the indicator table and the semi-annual reports. • Control of the means of verification that support the results reported in the implementation of the T-FAST Project. • Projection of expected results at the close of the T-FAST Project. Indicator Table and Semi-Annual Reports In a comparison of the indicator tables and the results reported in semi-annual reports, some small discrepancies were found that are clearly explained, have been corrected, or were changed based on guidance from USDA. The custom indicator 1.1 “percent of stakeholders who perceive the NTFC is enacting their agenda to strengthen trade facilitation in Paraguay” was not reported in Year 1 but appears to have the correct data included for both Year 1 and Year 2 in the Year 2 Annual Report. This was reported in accordance with USDA’s instructions. Custom indicator 2.1 “average release time for agricultural goods” was not reported in Year 1, but appears to have the correct data included for Year 1 in the Year 2 Annual Report. This indicator was also reported in accordance with USDA’s instructions. Custom indicator 2.2 “average cost of trade for agricultural goods” was removed based on recommendation from the baseline study, and indicator C.3 “percent of shipments assigned to the green channel for imports” was added based on recommendation from the baseline study. LINC LLC Page 37 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Review of Means of Verification All indicators reported to date have a means of verification. The only observation of the evaluation team on the data is to note that some indicators have no progress to date. These include indicator 4.1 “number of laboratories with improved infrastructure to better support international protocols for food testing/safety as a result of USDA assistance” and indicator 3.1 “number of private and public sector comments regarding trade reform and TFA implementation received by government through the T-FAST mechanism.” These are both expected to have data to report in future periods and T-FAST has already made progress on indicator 4.1 in the current period, reported in the most recent semi-annual report after the cutoff date for review in this midterm evaluation. Projections of Expected Indicator Results Compared to Targets The evaluation team analyzed the data on indicator progress to date, including through December 2021 (Quarter 1 of Year 3) to estimate potential progress of the project through Year 4. Three scenarios were constructed for each indicator. A pessimistic scenario assumes that the project will not achieve any more progress beyond the results through December 2021. Indicators that had already achieved their Life of Project (LOP) target meet the pessimistic scenario. An expected scenario and an optimistic scenario assume that Year 3 and Year 4 progress will continue either based on the expected targets set by the project for each year, or based on a continuation of the average quarterly progress achieved to date. The evaluation team also notes that some indicators are behind in progress due to delays caused by the pandemic and still have the opportunity to catch up even quicker than the optimistic scenario. This analysis simply helps to identify results for which the project might need to scale or adapt activities to meet their targets or where T-FAST might consider reviewing targets. T-FAST has ample opportunity in Years 3 and 4 to make progress towards achieving or exceeding targets as pandemic restrictions continue to ease, and as some activities build momentum and naturally pick up speed. Based on the three scenarios, the indicators are classified into four categories: those that will meet their targets under the pessimistic scenario (green cells in Table 16: Indicator Progress and LOP ProjectionsTable 16 below), those that will meet their targets under the expected scenario (yellow cells), those that will meet their targets under the optimistic scenario (yellow cells), and those that T-FAST may want to review to ensure they will move at a pace in the remaining two years to meet their targets (red cells and red text). Table 16: Indicator Progress and LOP Projections Projections # Indicator LOP Target Pessimistic Scenario Expected Scenario Optimistic Scenario 1 1. Number of host government or community derived risk management plans formally proposed, adopted, implemented, or institutionalized with USDA assistance 8 8 36 43 2 2. Number of organizations with increased performance improvement with USDA assistance 9 0 0 4 3 3. Value of new U.S. assistance commitments and new public and private sector investment leveraged by USDA to support food security and nutrition (USD) $1,456,000 $3,222,995 $12,562,419 $15,675,225 LINC LLC Page 38 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Projections # Indicator LOP Target Pessimistic Scenario Expected Scenario Optimistic Scenario 4 4.1 Number of laboratories with improved infrastructure to better support international protocols for food testing/safety, as a result of USDA assistance 6 0 0 4 5 5. Number of policies, regulations, and/or administrative procedures in each of the following stages of development as a result of USDA assistance 75 61 247 285 6 6. Number of individuals who have received short￾term agricultural sector productivity or food security training as a result of USDA assistance 1,006 457 929 996 7 7. Number of individuals participating in USDA food security programs 1,006 457 929 996 8 8. Number of individuals benefiting indirectly from USDA-funded interventions 19,396 12,407 54,143 54,311 9 Context: C 1.1 Value of agricultural imports $1.04B $1.00B $1,03B $1.04B 10 Context: C 1.2 Volume of agricultural imports 1.27M 1.26M 1.27M 1.49M 11 Context: C 2.1 Value of agricultural exports $5.22B $5.19B $5.22B $6.47B 12 Context: C 2.2 Volume of agricultural exports 13.47M 13.41M 13.47M 13.72M 13 Context: C 3 Percent of shipments assigned to the green channel for imports20 16% 10% 13% 16% 14 1.1 Percent of stakeholders who perceive the NTFC is enacting their agenda to strengthen trade facilitation in Paraguay 85% 78% 81% 85% 15 2.1 Average release time for agricultural goods21 324 463 410 324 16 2.3 Number of individuals in the agricultural trade system who have applied techniques or technologies that were provided with USDA assistance 970 600 2,689 3,200 17 3.1 Number of private and public sector comments regarding trade reform and TFA implementation received by gov’t through the T– FAST mechanism 57 0 0 7 18 Number of World Trade Organization Trade Facilitation Agreement provisions supported with USG assistance (EG.2.1-1) 15 6 19 24 As of the midterm evaluation, Indicators numbered 2, 4, 6, 7, and 17 were progressing at a slower rate than necessary to achieve their targets based on the optimistic scenario. As of December 2021, when the most recent indicator data was collected and presented for analysis in the midterm evaluation, T￾FAST had not yet completed an organizational capacity assessment process using the Organizational 20 Context indicator C 3 “Percent of shipments assigned to the green channel for imports” was added to the PMP after the baseline study in response to recommendations from the baseline. It has only been measured once to date and will next be measured in Year 3. 21 Custom indicator 2.1 “Average release time for agricultural goods” has only been measured at baseline and will next be measured in Year 3. LINC LLC Page 39 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Performance Index (OPI) which will directly impact indicator 2. Indicator 4 has seen some progress since the midterm, already meeting the expected target for Year 3. Indicators 6 and 7 were projected to come within 1% of the target under the optimistic scenario, and indeed in the first half of Year 3 (after the period studied in this midterm evaluation) the project has already increased its pace above the expected and optimistic scenarios and is on track to exceed targets. The NTFC platform for publishing policy proposals to receive comments had not yet been launched as of December 2021, which will directly impact indicator 17. 3.3.2. Efficiency To contribute to understanding the efficiency of the project, the evaluation team reviewed financial reports and expenditures to apply in comparing planned actions against executed actions under each T￾FAST activity. For the calculation of progress of actions under each activity over two years, the evaluation team used the work breakdown spreadsheet provided by T-FAST with self-assessed status (completed, in progress, not initiated). Direct costs attributed to each activity in years 1 and 2 (from start through September 30, 2021) were considered with indirect costs added based on the distribution of direct cost. The total cost for these years was $3,105,707 with 179 planned actions, of which 46% were completed, 15% were in progress, and the remaining 39% had not been initiated. Activity 2 executed the highest budget ($1.1M), planned the most actions (89), and started or completed the majority of those actions (56%). Activity 4 executed the second highest budget ($1.1M) with fewer planned actions (11) and completed the highest percentage of those actions (82%), primarily purchase of IT hardware and software. Table 17: Cost Effectiveness Analysis by Activity and Action Status Actions Activity Budget Total % Completed % In progress % Not initiated 1. Strengthen the National Trade Facilitation Committee $742,828 70 54.3% 10.0% 35.7% 2. Support Process Modernization with Government Institutions and Agencies in Agriculture Import/Export Processes $1,117,239 89 34.8% 21.3% 43.8% 3. Engage Trade Associations to Promote Improved Trade Facilitation $143,024 9 44.4% 11.1% 44.4% 4. Procure Software and Equipment to Improve Agriculture Import/Export Processes $1,102,616 11 81.8% 0.0% 18.2% 5. Manage On-Demand Mechanism for Improvements in Agriculture Import/Export Processes22 $0 0 0.0% 0.0% 0.0% Total $3,105,707 179 45.8% 15.1% 39.1% 22 The project includes this flexible mechanism to support unexpected needs that arise or evolve during implementation. It had not been started during the period covered by this midterm evaluation. LINC LLC Page 40 T-FAST Project Instituto Desarrollo Midterm Evaluation Report 3.3.3. Challenges The project reported several critical challenges to implementation. Two particular challenges that also resonated in the reported progress against activities were the COVID-19 pandemic and the turnover of public sector representatives to the NTFC. The pandemic had negative and positive reported effects on the project. It created delays with planned activities that required an in-person component or travel, slowed development or progress of some activities, and disrupted many of the underlying foreign trade activities that T-FAST was working to improve. At the same time, the shift to virtual meetings for the NTFC is believed to have increased participation through the pandemic, and the closure of many public offices to visitors created urgency for digitalization of processes and documentation. Despite the challenges of the pandemic, the project was able to adjust plans, including by using virtual meetings and remote training, providing additional support for IT solutions needed by public institutions, and shifting the timing of some activities. The project’s adaptation through the pandemic also allowed for additional time in planning events and activities that could be increased as the pandemic restrictions eased. Public sector staff participating in the project include those from MRE leading the NTFC, representatives to the NTFC, and participants in training, institutional capacity building, streamlining processes, implementing IT solutions, and planning efforts. Per reporting, public sector staff are occasionally reassigned to new responsibilities or leave their role through other types of human resource attrition. This has been a challenge for T-FAST, as even when a replacement is quickly identified and has the skills and capacity to take on their role, the change itself is disruptive and the new staff member often needs some time to adapt. As the project reports, these changes are largely out of the control of T-FAST, a statement which aligns with feedback heard in KIIs and in the survey. The project has documented its efforts to adapt, changing plans in response to those shifts in staffing, which stakeholders confirmed in KIIs and surveys as well. LINC LLC Page 41 T-FAST Project Instituto Desarrollo Midterm Evaluation Report 4.Conclusions 4.1. Relevance 4.1.1. To what extent have T–FAST expected results and activities to date addressed the core issues of target beneficiaries? Based on the survey, KIIs, and document review, T-FAST’s expected results and activities to date address the core issues of target beneficiaries. The top priorities of project participants are digitalization of foreign trade procedures and paperwork, simplification of processes and procedures, and increased coordination across relevant public institutions. T-FAST’s stated objectives, plans, and activities align well with these priorities. Additionally, stakeholders indicated their perception that the project is addressing the most important issues for facilitating foreign trade. There are priorities named by a subset of stakeholders that are outside of the stated scope of T-FAST and beyond the requirements of the TFA. The most important of which is the considerable infrastructure needed for Paraguay as a landlocked country. In particular, Paraguay’s foreign trade is heavily dependent on waterway infrastructure which has seen increased pressure with the recent drought. Additionally, Paraguay relies on good relationships at land border crossings with its neighbors, primarily Brazil and Argentina. For that reason, land border infrastructure, such as bridges and buildings, and equipment, such as scanners, that can facilitate clearance of goods at the border, are seen as a priority as well. 4.1.2. Is the project relevant in addressing goals and objectives of the Trade Facilitation Agreement (TFA), particularly in assisting Paraguay with its Category C notifications? Triangulating information from the document review, surveys, and interviews, T-FAST’s activities and expected results also support Paraguay in achieving completion of its Category C notifications. The focus of the project aligns directly with the TFA objectives, including activities to increase access to information, allow public consultation and commenting on policy proposals, reduction of average time for trade, training and equipment for laboratory professionals, and the AEO program. The project participants indicated their belief that the project is well-aligned to the objectives of the TFA. The general priorities for Paraguay’s foreign trade as cited by participants also align closely with the TFA objectives. 4.1.3. Is the project coherent with other projects that support the TFA? Are the projects mutually supporting or duplicating efforts and if so, how? The project is coordinating and complementing other projects that support the TFA in Paraguay, including several ongoing or recently completed projects supported by UN agencies and the EU or agencies from EU countries. T-FAST has demonstrated an interest and ability to coordinate closely with other projects not only to avoid a duplication of efforts, but to leverage one another’s work. For example, the UNCTAD Empowerment project conducted studies and assessments and proposed implementation of working groups for the NTFC. T-FAST coordinated closely with the effort, and supported the establishment of the working groups. In other cases, within specific key public institutions receiving support from other projects, T-FAST filled in gaps in equipment, training, or consultancy LINC LLC Page 42 T-FAST Project Instituto Desarrollo Midterm Evaluation Report activities. The ability of T-FAST to avoid duplication and complement other projects is credited largely to the flexibility of the project’s planning and budget. In conversations with T-FAST and comparison with data from interviews and surveys, several factors have contributed to T-FAST’s flexibility. The approval of annual workplans that identify objectives and results with less prescriptive description of project actions has allowed T-FAST to adapt specific support to fit the changing situation. The project Steering Committee allowed T-FAST to discuss the shifting context and specific areas of opportunity or potential duplication, identify solutions, and receive quick approval for changes as necessary. Key informants also noted the project team’s subject matter and local context knowledge combined with their responsiveness to questions and concerns as contributors to flexibility. 4.2. Effectiveness 4.2.1. How effectively is the project reaching its targets, both timewise and per the work plan? Is T–FAST on track to achieve the specific targets and results established? Based on the surveys and KIIs, the project is making effective progress towards its targets and objectives. Project participants are satisfied with the support the project is providing, believe the project is effective, and have highlighted some specific attributes of the project that are most valued: expertise of the staff and consultants, flexibility of the project to address changing needs, responsiveness of the project and project staff to questions and concerns raised by beneficiaries, and the approach to engaging embedded staff within the MRE to support the NTFC. The document review highlights that, overall, the project is progressing, with many activities, objectives, and indicator results on track to meet plans and targets. However, there are some gaps and challenges for the project. In terms of activities, analysis of the T-FAST team’s reported status of planned activities during Years 1 & 2 indicates that approximately 46% of planned actions through Year 2 have been completed, with another 15% in progress, leaving approximately 39% of actions not yet initiated. Most of the project’s indicators are on track to meet targets, assuming that the project can continue to achieve similar progress during Years 3 and 4 as achieved during Years 1 and 2. The pandemic has been the largest factor in challenges to achieving targets and completing activities per the work plan. The initial delay due to restrictions and necessary adjustments slowed progress up front, and unpredictability of changing restrictions or waves of pandemic challenges forced further adaptation of the project. Other challenges have affected the ability of T-FAST to execute on its workplans and achieve its targets, including bureaucratic delays in the public institutions receiving support and turnover of key public institution counterparts at MRE and in NTFC member representation. Taking into consideration the challenges, despite some differences between workplan and execution, the project has shown great resilience and adaptability to maintain effectiveness. In fact, that flexibility has been more important to effectiveness in results and satisfaction of beneficiaries than simply completing all activities as described in the workplan given the dynamic context. Additionally, the delays due to the pandemic have raised the question from stakeholders of whether the time of the project can be extended to ensure that the support of T-FAST is fully utilized and realized. LINC LLC Page 43 T-FAST Project Instituto Desarrollo Midterm Evaluation Report 4.2.2. Which activity or combination of activities has/have the highest potential to achieve T–FAST’s higher-level result (expanded trade of agricultural products)? T-FAST activities are focused on foreign trade facilitation more generally and not on specific aspects of agricultural production, processing, or domestic sales. However, trade of agricultural products will be expanded by T-FAST’s support for improvements to foreign trade processes with some additional focus on processes specific to agricultural goods and/or public institutions involved in various aspects of trade of agricultural goods. According to partner institutions, several activities have the potential for high impact on T-FAST’s goal of expanded trade of agricultural products. Easing access to the Paraguayan market and other markets from Paraguay through the digitalization of processes and paperwork and the simplification of trade procedures are the two activities with the highest potential for expanding trade. The efforts have a higher potential for expanding agricultural trade by continuing to focus on improving the processes of key public institutions for agricultural trade such as SENAVE, SENACSA, INAN, and any laboratories involved in clearing agricultural products. 4.2.3. How effective is T–FAST at reaching women beneficiaries (particularly via capacity building activities) and what more could be done to improve women’s engagement / leadership? Based on interviews and supported by the surveys, T-FAST is effective at reaching women beneficiaries. This is primarily due to the high participation of women in foreign trade, both in the public and private sectors. Women are reported to have leadership and management positions and be actively involved in decision-making. The composition of the NTFC includes more women than men, so project activities working with the NTFC adequately include and reach women. Additionally, the leadership of the working groups has high participation of women, and counterparts of many of the project activities at public institutions are women. Women are well-represented as participants in training courses. 4.2.4. Which activities were most effective in contributing to a reduction of average time of trade? According to partner institutions, the activities that are most effective in contributing to a reduction of average time to trade are those that are most directly addressing the processes, procedures, rules, and paperwork required for import and export. Those include work on digitalization of processes and paperwork (including provision and expansion of IT hardware, software, and systems), streamlining of procedures, and support to specific public institutions on their bottlenecks or other constraints (including knowledge and skills, human resources, equipment, planning, and institutional strength). Additionally, stakeholders indicated that the expansion of the AEO program has contributed to a reduction of average time to trade. As noted elsewhere, the relevant PMP indicator (custom indicator 2.1 “Average release time for agricultural goods”) was calculated as part of the baseline study but has not been updated yet. It will next be calculated in Year 3 to show whether average time of trade has been reducing. LINC LLC Page 44 T-FAST Project Instituto Desarrollo Midterm Evaluation Report 4.3. Efficiency 4.3.1. To what extent are staffing, management, and oversight costs suitable, given the number and scope of activities carried out and the outputs obtained? To answer this question on efficiency, the evaluation team has taken into account the project stakeholders’ perceptions of cost and value as well as understanding how the results of the project relate to deviations or adaptations from its plans, in terms of budget, activities, and staffing. Stakeholders indicated that the project is providing substantial value to foreign trade overall, to the NTFC, and to the individual institutions involved, and that this value is proportionate to or even exceeds expectations for the resources employed. Based on the review of plans, progress reports, and indicators, the project has adapted many of its activities and timing. In some cases, the project has experienced delays, while in others, the project is achieving more than expected. The annual project expenditures have generally remained in line with budgets. The key challenges that have instigated project adaptations from plans are complex, including the pandemic, turnover of key public institution stakeholders, and bureaucracy in beneficiary institutions. Considering these challenges and the value stakeholders have placed on the flexibility of the project, the project has been efficient in its implementation. An emphasis on remaining flexible and continuing to adapt to the context is a key factor in the effectiveness and efficiency of the project to achieve results and to meet the needs of project beneficiaries. 4.4. Sustainability 4.4.1. What factors contribute to the sustainability of project results and how has T–FAST focused on these to date? To what extent is progress being made towards project sustainability/graduation, and what could be done to strengthen the sustainability of project activities and results? Partner institutions highlight several factors that they believe contribute to sustainability of both results and activities. Foremost among the factors are the value of simplification and automation of processes which benefit private sector importers and exporters as well as public institutions managing those processes. Given that it would take additional effort to return to the previous and generally agreed to be inferior way of doing things, it is likely that these changes will be maintained beyond the life of the project. Additional factors for sustainability include the monitoring system and NTFC project dashboard making actions transparent and accountable, human and institutional capacity within individual institutions to continue activities and even improve on them, structure and management of the NTFC, commitment of institutional authorities, continued active participation of the private sector, and availability of financial resources. T-FAST has worked extensively on digitalization, simplification, IT systems, the project dashboard, and capacity development at individual institutions and with the NTFC management. There is still ample opportunity to continue implementing digitalization and simplification activities and IT systems which are likely to be maintained. There are also still opportunities and project plans to strengthen institutional capacity and the structure and management of the NTFC. To strengthen the LINC LLC Page 45 T-FAST Project Instituto Desarrollo Midterm Evaluation Report sustainability of project activities and results, further T-FAST support could be invested in the commitment and capacity of the NTFC. In particular, T-FAST can work with the NTFC and/or its members to identify and access financial resources for sustainability and to continue to develop the structure and leadership of the NTFC. Several key informants suggested that long-term sustainability and potential shifts in the country’s trade facilitation needs and the role of the NTFC to address those may require review and update to the legal framework of the NTFC. 4.5. Impact 4.5.1. Identify both intended and unintended intermediate results that are expected to contribute to / or hinder medium and long-term effects? Given the project trajectory so far, is it feasible to achieve the expected medium and long-term effect? How can these results be enhanced or mitigated to ensure expected medium and long-term results are achieved? The most important result of the activity to date has been the uptake of technology to support digitalization, simplification, and inter-institutional coordination and data sharing. The IT systems and related improvement of processes, paperwork, procedures, and data management contribute greatly to the medium and long-term goals of reduction of average time to trade and sustainability of the project’s support. The IT systems supported by T-FAST include creating electronic versions of paper based forms, eSignature approval processes, and electronic payments, among other changes. These upgrades have improved processes and efficiency for both public and private sector stakeholders, increasing the likelihood of their sustainability. With expanded use and reliability on those systems, additional individuals and institutions are adopting them as well. For the longer-term effects, important contributing activities and results include training of personnel, institutional capacity strengthening, and support to the NTFC. The NTFC is the long-term avenue to continuous improvement of trade facilitation through coordination, consultation, and policy management. 4.6. Challenges 4.6.1. What are the external forces that negatively and positively impacted the project implementation, what were the effects on different stakeholders, and how did the project attempt to overcome, mitigate, or capitalize its impact? There are three primary external forces that negatively impacted project implementation: the COVID-19 pandemic, the turnover in key positions in the public sector, and slow processes due to bureaucracy. Each of the factors caused delays and changes to many of the planned activities for the project, primarily affecting project participants in the way and time that they were able to engage with the project. Delays due to the pandemic near the start of the project also affected coordination with other projects such as the UNCTAD Empowerment project who noted that their consultancy regarding the structure of the NTFC and working groups launched as T-FAST was adapting to COVID and thus did not participate in meetings. However, T-FAST stayed flexible and adapted to the changing conditions. Some activities are still lagging behind their plans, but the project has maintained momentum. The primary external force that positively impacted project implementation is also the pandemic. Due to safety-related restrictions, the pandemic increased the urgency and willingness on the part of public LINC LLC Page 46 T-FAST Project Instituto Desarrollo Midterm Evaluation Report institutions involved in trade to adopt IT systems and digitalization. It also moved NTFC meetings online, which is believed to have increased regular attendance and the participation of many high-level officials at member institutions. The project capitalized on this by staying responsive to project participants’ requests for support, including for IT systems, analysis of processes for improvement, and digitalization. 4.6.2. What are the key challenges in collecting data and information and how did the project overcome or mitigate? In reviewing project reports and data, the primary challenge in collecting information has been bureaucracy in approvals. Paraguay has strong databases for information on foreign trade, in particular the customs authority’s SOFIA system, first established with support from the private sector. The project has received extracts from SOFIA and other databases with approval through bureaucratic channels. The project has also conducted a variety of studies and assessments that have required interviews, surveys, and other data collection. In the relevant public institutions, this has usually required similar bureaucratic approval processes. Based on project reports and stakeholder feedback, the project has used two primary strategies to mitigate these challenges. T-FAST has planned for the bureaucracy, setting better expectations and budgeting more appropriate timelines for approvals. The project’s leadership has also maintained pressure to speed the processes through regular follow-up on requests. 4.6.3. Have recommendations from the baseline evaluation been incorporated into the project and if not, why not? T-FAST has responded to all of the recommendations from the baseline. Some recommendations have been directly incorporated into the project, some have been adapted, and some have informed the project’s planning, implementation, and monitoring. A summary is provided in Table 18 below. Table 18: Summary of Baseline Recommendations and T-FAST Responses Baseline Recommendation T-FAST Action or Response Recommendation 1: Adjust the targets for Custom Indicator 1.1: Percent of stakeholders who perceive the NTFC is enacting their agenda to strengthen trade facilitation in Paraguay. The indicator definition and targets have been updated and approved in the revised PMP. Recommendation 2: Consider other measures of NTFC effectiveness. As included in the Year 2 workplan, the project MEL Specialist prepared and implemented a monitoring plan for the action plans of the NTFC working groups, a control panel/project dashboard for the NTFC, and a system to measure participation by NTFC members. Recommendation 3: Adjust the disaggregation of Custom Indicator 2.1: Average release time for agricultural goods. The indicator disaggregation has been updated and approved in the revised PMP. Recommendation 4: Consider other possible measures of efficiency in customs procedures. T-FAST added a new indicator to measure the “proportion of shipments assigned to the green channel at customs for import products,” approved in the revised PMP. Recommendation 5: Adjust or remove Custom Indicator 2.2: Average cost of trade for agricultural goods. T-FAST removed custom indicator 2.2, approved in the revised PMP. LINC LLC Page 47 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Baseline Recommendation T-FAST Action or Response Recommendation 6: Consider using some complexity aware monitoring tools for T-FAST. T-FAST is planning to incorporate two complexity aware monitoring tools: Outcome Logging and Most Significant Change. The MEL Specialist has worked to incorporate the tools into the NTFC monitoring system. Recommendation 7: To the extent possible, focus T-FAST activities on several specific needs for foreign trade in Paraguay. T-FAST has focused efforts and invested resources into several of the recommended areas, including digitalization of data, the monitoring system for NTFC working group action plans, and transparent accessibility of trade information. LINC LLC Page 48 T-FAST Project Instituto Desarrollo Midterm Evaluation Report 5.Recommendations The recommendations to the T-FAST project, based on lessons learned from the fieldwork and findings of the midterm evaluation, fall into two categories: • Overall Project Recommendations are actions or considerations for the project to take which may include areas of focus, activities, or changes to the project SOW. • Recommendations for NTFC Support are actions or considerations specific to T-FAST’s support of the NTFC. 5.1. Overall Project Recommendations Recommendation 1. Continue and expand the most effective project activities. These activities are already in T-FAST’s project plans, and there are many other activities planned that are not meant to be diminished by the recommendation. Still, these are some of the most effective and valued project activities and T-FAST should continue to invest resources and effort in them. • Digitalization is considered to be perhaps the most valued support that T-FAST is providing the agricultural sector. Public and private sector representatives indicate that there are still ample opportunities to improve and expand digitalization of processes, paperwork, payments, etc. • Streamlining foreign trade processes is also highly valued by project participants and stakeholders. The project has been successful at assessing existing workflows and processes to identify areas for improvement, and stakeholders indicate that there are still many processes that can be improved. • Similarly, institutional strengthening for high priority public institutions, including SENAVE, SENACSA, INTN, INAN, and DINAVISA, has been effective for the reduction of time to trade. Recommendation 2. Maintain flexibility and responsiveness of the project. Again, this recommendation is to continue to focus on one of the strengths of the project to date. The project has been successful despite considerable challenges outside of its control, and that success is in part due to its ability to adapt. The flexibility and responsiveness of the project to its context and the needs of stakeholders has been a highlight, helping keep the project relevant and effective more than simply completing activities as planned each year in a dynamic context. As indicated in the findings, based on discussions with T-FAST staff and KIIs, there are several factors that contribute to flexibility and responsiveness which T-FAST can work to maintain: • The annual workplan was not prescriptive, rather it identifies targeted results with general descriptions of actions leaving room for adaptation. • The project Steering Committee provides a knowledgeable sounding board for adaptations to evolving context and allows for quick approval of changes in plan. • T-FAST staff are highly responsive to needs as they arise. • While it had not been used during the period covered by this evaluation, the Activity 5 “on￾demand mechanism” brings added flexibility to address new needs as they arise. LINC LLC Page 49 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Recommendation 3. Support coordination by public institutions with the appropriate mandate. T-FAST is providing specific support and institutional strengthening to several individual public institutions. Project stakeholders generally agree that the selected public institutions are the highest priority for the goals of the TFA, of the NTFC, and of T-FAST. However, for several of the areas being supported, there are institutions with an existing mandate who could be more engaged in accompanying the consultancy from T-FAST. That engagement would serve as a capacity strengthening exercise for the partner to improve the support they provide on the same or similar topics, and it would allow them to take a lead role in coordinating across institutions. Two specific examples come from the fieldwork: • T-FAST is supporting laboratories with studies, training, and equipment. Many of the public institutions that the project supports have their own laboratories and work with private laboratories as well. Paraguay also has a Central Laboratory in the Ministry of Public Health (MSPyBS) which has a representative on the NTFC but has not been engaged in any direct support from the project. Given the Central Lab’s mandate of supporting and coordinating across other laboratories in the country, there is an opportunity to engage them in the various consultancy activities with institution or product specific laboratories. • T-FAST is also supporting institutions on processes related to the clearance of agricultural products and shipments for export or import. Similar to the Central Lab mandate related to laboratories, the Ministry of Agriculture (MAG) has a mandate related to regulation, policy, and support associated with agricultural goods. There is again an opportunity to engage MAG in the T-FAST consultancies that are relevant to them. Recommendation 4. Consider adding time to the project. Given that the pandemic caused some project delays in Year 1 and slowed implementation of some activities, the project and its beneficiaries could benefit from an extension. In addition to providing additional time for activities that were delayed or slowed, an extension will provide additional runway for sustainability efforts with the NTFC and could potentially allow the project to support any transitions necessary if the government changes after the next election. Recommendation 5. Review and strengthen the communications strategy and plan as appropriate. As noted, project and NTFC communications were highlighted as a strength by some stakeholders and as a weakness by others. T-FAST has already invested in improving and formalizing communications and communications planning, and a review of the strengths and opportunities would help ensure a more consistently positive perception of communications. Recommendation 6. Review and strengthen procurement procedures as appropriate. While responsiveness of T-FAST to stakeholder needs and requests is a highlight and strength, the feedback from some stakeholders on the slow speed of procuring some equipment or services by the project is worth further investigation. The specific procurement speed challenges may be standard procurement process constraints in terms of approvals or quotations, but if there is an opportunity to streamline those project processes it could benefit T-FAST and its stakeholders. LINC LLC Page 50 T-FAST Project Instituto Desarrollo Midterm Evaluation Report 5.2. Recommendations for NTFC Support Beyond being an important body for the facilitation of trade and a mechanism for achieving T-FAST project activities, the NTFC represents the best opportunity for sustainability of project activities and results. Recommendation 7. Further develop and strengthen the structure of the NTFC. The establishment of NTFC working groups has improved the performance and results of the overall committee. However, there are opportunities for continued improvement of the NTFC structure, including: • Despite the establishment of working groups to take on assigned responsibilities and remove them from the entire committee’s agenda, the NTFC meetings are still long and internal communications are a challenge. Increased autonomy for the working groups would be beneficial, but this still requires improved communications to keep the rest of the committee informed about working group activities. • The working groups have a lot of interest from NTFC members, but currently the groups are still quite large and many NTFC members are in more than one working group. Interest and active participation should be appreciated, but the NTFC and working groups would be more efficient with less overlap in membership across groups. • There will be moments when new working groups are needed or when existing working groups are no longer needed. This is a natural progression that should be facilitated by the NTFC leadership and can be raised by working group members themselves. T-FAST could establish a regular review where NTFC leadership and members can reflect on the structure of working groups and make decisions to adapt. • The NTFC is currently led by MRE with an individual Secretary for the committee. There would be value in establishing a coordinating body like a steering committee or a secretariat comprised of members of multiple institutions and including public and private sectors. That coordinating body would help to improve coordination across working groups and to maintain communications between working groups and the full NTFC in a way that eases the effort and time commitment of members. Recommendation 8. Facilitate regular review of the NTFC decree and legal framework. There is a tension between the interest in a more concrete legal framework for the NTFC from the legislative branch and the flexibility and authority associated with having a decree from the executive branch. The decree has also already been updated once, based on experience. There would be value to regularly reviewing the decree to propose revisions as appropriate, based on changing priorities, emerging constraints or opportunities, and NTFC needs. Recommendation 9. Increase private sector representation. The private sector has been very actively involved in the NTFC, the working groups, and other activities of the project. However, they have limited time and budget for participation in activities. It will be important to monitor their energy level and the balance of investment of time to value they receive for participation. One way to improve that balance could be to add new members to the NTFC from the LINC LLC Page 51 T-FAST Project Instituto Desarrollo Midterm Evaluation Report private sector which might both increase the value and decrease the necessary time commitment from any one individual or organization. Recommendation 10. Engage the government’s National Economic Cabinet (Equipo Economico Nacional). The National Economic Cabinet is a coordination body comprised of eight cabinet ministers. Five of the ministries represented are also members of the NTFC, but the topic of foreign trade facilitation has not been brought to them to date. Raising the topic would bring visibility to the work of the NTFC, could support a longer-term prioritization of foreign trade for the government, and would facilitate any reform that requires changing norms such as executive decrees and laws. This group’s support may also be necessary for the long-term financial sustainability of the NTFC. Recommendation 11. Review the mandate and scope of the NTFC for possible expansion. Per decree, the NTFC’s mandate is quite wide, however it has been primarily focused on meeting the requirements of the TFA. That is an understandable and important focus but may impede the NTFC from becoming a more sustainable entity with continued activities in the long term. A wider scope might include seeking additional funding for support from government or the private sector, or even from additional international donors when appropriate, and advocating or supporting non-TFA related foreign trade needs like infrastructure projects. Recommendation 12. Support NTFC with analysis of infrastructure needs. Related to the previous recommendation, it could be cost effective and beneficial for all stakeholders to have T-FAST support or conduct some exploratory analysis of the challenges related to infrastructure needed to expand and facilitate trade. This includes waterway infrastructure, bridges and buildings at border crossings, and large equipment for customs locations. The issue was top of mind for several stakeholders. Though it is outside of the current scope of the project and that of the NTFC, this would help the NTFC and its members identify upcoming challenges and shift focus to a longer-term future. LINC LLC Page 52 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Annexes LINC LLC Page 53 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Annex 1: Results Framework The T-FAST project Results Framework is included below, as presented in the project Performance Monitoring Plan (PMP) revised July 19, 2021. All boxes in green represent results that T-FAST expects to support through its activities. Figure 2: T-FAST FFPr Results Framework LINC LLC Page 54 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Annex 2: Indicator Table Table 19: Indicator Table with Target and Actual for Years 1 & 223 # Indicator Baseline Y1 Target Y1 Total Y2 Target Y2 Total LOP Target % Achievement of Y2 Target 1 1. Number of host government or community derived risk management plans formally proposed, adopted, implemented, or institutionalized with USDA assistance 0 0 0 0 0 8 N/A 2 2. Number of organizations with increased performance improvement with USDA assistance 0 0 0 1 0 9 0% 3 3. Value of new U.S. assistance commitments and new public and private sector investment leveraged by USDA to support food security and nutrition (USD) $0 $0 $0 $0 $378,520.66 $1,456,000 >100% 4 4.1 Number of laboratories with improved infrastructure to better support international protocols for food testing/safety, as a result of USDA assistance 0 0 0 0 0 6 N/A 23 Year 1 of T-FAST reflects the period from October 2019-September 2020, and Year 2 from October 2020-September 2021. LINC LLC Page 55 T-FAST Project Instituto Desarrollo Midterm Evaluation Report # Indicator Baseline Y1 Target Y1 Total Y2 Target Y2 Total LOP Target % Achievement of Y2 Target 5 5. Number of policies, regulations, and/or administrative procedures in each of the following stages of development as a result of USDA assistance 0 0 0 9 10 75 111% 6 6. Number of individuals who have received short￾term agricultural sector productivity or food security training as a result of USDA assistance 0 0 0 111 377 1,006 340% 7 7. Number of individuals participating in USDA food security programs 0 0 0 111 377 1,006 340% 8 8. Number of individuals benefiting indirectly from USDA-funded interventions 0 0 0 222 3,007 19,396 1,355% 9 Context: C 1.1 Value of agricultural imports $1,027,162,000 $1,027,162,000 $965,731,000 $1,030,243,486 $1,025,477,000 $1,036,434,219 99% 10 Context: C 1.2 Volume of agricultural imports 1,257,811 1,257,811 1,415,017 1,261,584 1,570,839 1,269,165 125% 11 Context: C 2.1 Value of agricultural exports $5,171,399,000 $5,171,399,000 $5,435,211,000 $5,186,913,197 $7,498,174,000 $5,218,081,358 145% 12 Context: C 2.2 Volume of agricultural exports 13,354,043 13,354,043 13,506,568 13,394,105 13,936,030 13,474,590 104% 13 Context: C 3 Percent of shipments assigned to the green channel for imports24 14% 10% 10% 12% 10% 16% 83% 24 Context indicator C 3 “Percent of shipments assigned to the green channel for imports” was added to the PMP after the baseline study in response to recommendations from the baseline. It has only been measured once to date and will next be measured in Year 3. LINC LLC Page 56 T-FAST Project Instituto Desarrollo Midterm Evaluation Report # Indicator Baseline Y1 Target Y1 Total Y2 Target Y2 Total LOP Target % Achievement of Y2 Target 14 1.1 Percent of stakeholders who perceive the NTFC is enacting their agenda to strengthen trade facilitation in Paraguay 70% 70% 70% 75% 92% 85% 123% 15 2.1 Average release time for agricultural goods25 463 hours 463 hrs 463 hrs 463 hrs 463 hrs 324 hrs N/A 16 2.3 Number of individuals in the agricultural trade system who have applied techniques or technologies that were provided with USDA assistance 0 0 0 0 0 970 N/A 17 3.1 Number of private and public sector comments regarding trade reform and TFA implementation received by gov’t through the T–FAST mechanism 0 0 0 0 0 57 N/A 18 Number of World Trade Organization Trade Facilitation Agreement provisions supported with USG assistance (EG.2.1-1) 0 0 0 2 2 15 100% 25 Custom indicator 2.1 “Average release time for agricultural goods” has only been measured at baseline and will next be measured in Year 3. LINC LLC Page 57 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Annex 3: Data Collection Instruments The evaluation team used two primary data collection instruments: an online survey and a semi￾structured interview guide for use with key informants. The final versions (in English) are included here. Draft Online Survey About the T-FAST Project The “Trade-Facilitating Agricultural Systems and Technology” (T-FAST) project is funded by the United States Department of Agriculture (USDA) in Paraguay. The T-FAST project aims to simplify, modernize and harmonize the processes of export, import, and transit of agricultural products in Paraguay, taking actions such as capacity building for the National Trade Facilitation Committee (NTFC) and providing support to commercial infrastructure. This online survey is part of the Mid-term Evaluation Study of the T-FAST project that seeks to improve trade facilitation during the years 2019-2023. Any information you provide by responding to this survey will be kept secure and confidential. All reports of results will be considered confidential and will not attribute any responses to individuals or their organizations. If you have any questions about the use or confidentiality of your responses, please feel free to contact rfromer@linclocal.org or jmolinasvega@desarrollo.edu.py Online Survey Objectives On behalf of the organizations responsible for the independent evaluation of the T-FAST project - LINC (USA) and Instituto Desarrollo (Paraguay) - we are conducting this survey with the consent of the NTFC. The T-FAST program and its independent evaluation are being funded by the United States Department of Agriculture (USDA). We value your opinion as a key stakeholder for the success of the project. The results of this survey and of the Mid-term Evaluation will be used to improve the project and the support provided to its stakeholders. Therefore, we ask your collaboration so that the T-FAST project can systematize the lessons learned, continue with good practices, and make the necessary adjustments. Your participation in providing this feedback is a sign of your commitment to the goals of the T-FAST project. We understand that as a member of a beneficiary organization, the success of the program will directly benefit your organization. The results of the survey will be used to guide measures to improve the responsiveness of T-FAST to strengthen trade facilitation in Paraguay. Your responses will be kept strictly confidential. All the questions below are about inputs that will allow us to evaluate the implementation of T-FAST from September 2019 to the present. It will allow us to identify suggestions to strengthen what is working well, and to take corrective actions where there are opportunities for improvement. The survey should take approximately 20 minutes to complete. I. Demographics and key characteristics 1. Name: 2. Gender: LINC LLC Page 58 T-FAST Project Instituto Desarrollo Midterm Evaluation Report 3. Organization: 4. Occupation/role: 5. Years of experience in your field: 6. Area of professional training: 7. Level of education: II. Knowledge about T-FAST 8. When did you learn about the T-FAST project: month/year 9. How did you learn about the T-FAST project? (Mark all that apply) a. Formal presentation of the project by the project team b. Read the official project document c. Social media, website d. Media e. Informal conversations about the project with colleagues f. Other: (Specify)_____________________ 10. Have either you or your institution received direct support from the T-FAST project? (Mark all that apply) a. Training b. ICT equipment or software c. Lab support d. Consultancy on key trade related process e. Other: (Specify)_____________________ III. Relevance 11. Please rate in a scale from (1) extremely poor to (7) excellent, to what extent T-FAST expected results at the end of the project is related to the priority issues and challenges of your organization. (1) Extremely Poor, (2) Very Poor, (3) Poor, (4) Neutral, (5) Good, (6) Very Good, (7) Excellent, (0) Do not Know 12. Please cite three pairs of: T-FAST expected results AND a priority challenge of your organization. a. (expected result)____________// (priority challenge of your organization)_______________ b. (expected result)____________// (priority challenge of your organization)_______________ c. (expected result)____________// (priority challenge of your organization)_______________ 13. Please rate in a scale from (1) extremely poor to (7) excellent, to what extent T-FAST activities already concluded addressed the priority issues and challenges of your organization. (1) Extremely Poor, (2) Very Poor, (3) Poor, (4) Neutral, (5) Good, (6) Very Good, (7) Excellent, (0) Do not Know 14. Please cite three pairs of: T-FAST activity already concluded AND a priority challenge of your organization. a. (activity concluded)__________// (priority challenge of your organization)_______________ LINC LLC Page 59 T-FAST Project Instituto Desarrollo Midterm Evaluation Report b. (activity concluded)__________// (priority challenge of your organization)_______________ c. (activity concluded)__________// (priority challenge of your organization)_______________ 15. Please rate in a scale from (1) extremely poor to (7) excellent, to what extent T-FAST activities already concluded addressed the goals and objectives of the World Trade Organization Trade Facilitation Agreement. (1) Extremely Poor, (2) Very Poor, (3) Poor, (4) Neutral, (5) Good, (6) Very Good, (7) Excellent, (0) Do not Know 16. Please rate in a scale from (1) extremely poor to (7) excellent, to what extent T-FAST activities complement, support, and/or build on other projects helping Paraguay implement the Trade Facilitation Agreement (e.g., projects by UNCTAD, Interamerican Development Bank, World Bank, etc.). (1) Extremely Poor, (2) Very Poor, (3) Poor, (4) Neutral, (5) Good, (6) Very Good, (7) Excellent, (0) Do not Know IV. Effectiveness 17. Please rate in a scale from (1) extremely poor to (7) excellent, to what extent these specific T￾FAST activities already concluded will enable to expand trade of agricultural products. (Note, if you are not aware of any of the following T-FAST activities or its contribution to trade facilitation, please answer “0-Do not Know” for that activity.) (1) Extremely Poor, (2) Very Poor, (3) Poor, (4) Neutral, (5) Good, (6) Very Good, (7) Excellent, (0) Do not Know T-FAST activity already implemented 1- Extremely Poor 2 3 4 5 6 7- Excellent 0-Do not Know a. Designing, validating, and procuring a digital platform to support NTFC members´ coordination, publication and availability of relevant information, and providing opportunity for the public to comment. b. Jointly developed action plans with NTFC members (e.g., SENAVE, INTN, MSPyBS-DINAVISA) c. Training courses for NTFC members d. Support to NTFC working groups e. Development and implementation of ICT software and collaborative process optimizations with selected institutions (e.g., SENAVE, MIC-VUE, SENACSA). LINC LLC Page 60 T-FAST Project Instituto Desarrollo Midterm Evaluation Report f. Coordinating activities with 13 binational chambers of commerce, and the largest business associations of the country. g. Training to laboratory professionals. h. T-FAST facilitated electronic payments and installed ICT server hardware on which digitalized processes are being programmed and implemented. i. T-FAST developed the first NTFC monitoring plan and a communications plan that addresses the needs of the NTFC’s 45+ members. j. T-FAST supported the process to achieve complete adoption of the Authorized Economic Operator (AEO) program at regulatory institutions. 18. Please rate in a scale from (1) extremely poor to (7) excellent, the overall level of effectiveness of T-FAST at reaching women beneficiaries, especially in capacity building activities. (1) Extremely Poor, (2) Very Poor, (3) Poor, (4) Neutral, (5) Good, (6) Very Good, (7) Excellent, (0) Do not Know 19. List 3 initiatives that could be particularly effective in reaching women beneficiaries that T-FAST should consider. i.________________________________ ii.________________________________ iii.________________________________ 20. Please rate in a scale from (1) extremely poor to (7) excellent, the overall level of effectiveness of T-FAST in contributing to a reduction of average time of trade, so far. (1) Extremely Poor, (2) Very Poor, (3) Poor, (4) Neutral, (5) Good, (6) Very Good, (7) Excellent, (0) Do not Know 21. List 3 T-FAST activities that were or could be particularly effective in contributing to a reduction of average time of trade. i.________________________________ ii.________________________________ iii.________________________________ LINC LLC Page 61 T-FAST Project Instituto Desarrollo Midterm Evaluation Report V. Sustainability 22. Please rate in a scale from (1) extremely poor to (7) excellent, to what extent the T-FAST project key results so far will be sustained in the long-term either by the NTFC members or through other means. (1) Extremely Poor, (2) Very Poor, (3) Poor, (4) Neutral, (5) Good, (6) Very Good, (7) Excellent, (0) Do not Know 23. List 3 main factors or conditions that indicate the NTFC members will sustainably maintain key results and activities initiated with support of the T-FAST project. i.________________________________ ii.________________________________ iii.________________________________ VI. Challenges 24. Please rate in a scale from (1) extremely poor to (7) excellent, to what extent external forces (i.e., not controlled by the T-FAST project) negatively impacted project implementation, so far. (1) Extremely Poor, (2) Very Poor, (3) Poor, (4) Neutral, (5) Good, (6) Very Good, (7) Excellent, (0) Do not Know 25. List 2 external forces that may have negatively impacted project implementation, so far. i.________________________________ ii.________________________________ 26. Please rate in a scale from (1) extremely poor to (7) excellent, to what extent external forces (i.e., not controlled by the T-FAST project) positively impacted project implementation, so far. (1) Extremely Poor, (2) Very Poor, (3) Poor, (4) Neutral, (5) Good, (6) Very Good, (7) Excellent, (0) Do not Know 27. List 2 external forces that may have positively impacted project implementation, so far. i.________________________________ ii.________________________________ iii.________________________________ LINC LLC Page 62 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Key Informant Interviews: Semi-Structured Interview Guide Instructions to Interviewers This interview guide has been developed to facilitate semi-structured interviews with representatives of institutional members of the National Trade Facilitation Committee (NTFC), and where appropriate, with other stakeholders to the T-FAST project such as international cooperation agencies. All interviews will be conducted by the Evaluation Team and are anticipated to take 30 to 60 minutes. The semi-structured interview is a conversational format, enabling the interviewer to direct lines of questioning to areas in which the respondent has particular expertise or insight. Respondents are anticipated to come from a variety of backgrounds and to have a variety of experiences relevant to T￾FAST and/or this evaluation. Therefore, some categories of discussion will be emphasized over others. Introduction for Key Informants About the T-FAST Project The “Trade-Facilitating Agricultural Systems and Technology” (T-FAST) project is funded by the United States Department of Agriculture (USDA) in Paraguay. The T-FAST project aims to simplify, modernize and harmonize the processes of export, import, and transit of agricultural products in Paraguay, taking actions such as capacity building for the National Trade Facilitation Committee (NTFC) and providing support to commercial infrastructure. This in-depth interview with key stakeholders is part of the midterm evaluation study of the T-FAST project that seeks to improve trade facilitation during the years 2019-2023. Any information you provide in this interview will be kept secure and confidential. All reports of results will be considered confidential and will not attribute any responses to individuals or their organizations. Key Informant Interview Objectives On behalf of the organizations responsible for the independent evaluation of the T-FAST project - LINC (USA) and Instituto Desarrollo (Paraguay) - we are conducting several interviews with key NTFC stakeholders with the consent of the NTFC. As mentioned, the T-FAST program and its independent evaluation are being funded by the United States Department of Agriculture (USDA). We value your opinion as a key stakeholder for the success of the T-FAST project. The results of the midterm evaluation will be used to improve the project and the support provided to its stakeholders. Therefore, we ask your collaboration so that the T-FAST project can systematize the lessons learned, continue with good practices, and make the necessary adjustments. Your participation in providing this feedback is a sign of your commitment to the goals of the T-FAST project. We understand that as a member of a participating organization, the success of the program will directly benefit your organization. The results of this interview will be used to guide measures to improve the responsiveness of T-FAST to strengthen trade facilitation in Paraguay. Your responses will be kept strictly confidential. All the questions we will ask are about inputs that will allow us to evaluate the implementation of T￾FAST from September 2019 to the present. It will allow us to identify suggestions to strengthen what is working well, and to take corrective actions where there are opportunities for improvement. The interview should take approximately 45 minutes to complete. Relevance 1. Please tell us briefly about yourself, your experience in the trade or trade facilitation area. Based on your experience, please identify the top three priorities to make trade more efficient in LINC LLC Page 63 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Paraguay. Do those priorities align with the goals and objectives of the Trade Facilitation Agreement (TFA)? a. Note to enumerator, if priorities do not align with TFA, continue to probe this in question 2 below 2. Do you think the T-FAST project addressed the top three priorities to make trade more efficient in Paraguay you just mentioned? a. Note to enumerator, if the respondent asks / needs more information, some example activities completed by T-FAST include: digital platform to support NTFC coordination and information sharing; development of joint action plans for NTFC members; training courses for NTFC members; support to 3 NTFC working groups; development of ICT solutions for select institutions (e.g., SENAVE, MIC-VUE, SENACSA); Training for laboratory professionals; development of NTFC monitoring plan; supporting regulatory institutions to adopt Authorized Economic Operator program. b. Some example expected results include: 30% reduction in foreign trade release time for agricultural goods; increased NTFC capacity and 85% of stakeholders who perceive the NTFC has enacted its agenda; agricultural import/export processes modernized at Government institutions and agencies; private sector engaged in trade reform and TFA implementation (e.g., providing input to government); laboratories with improved infrastructure to meet international food safety protocols. Effectiveness/Impact 3. What are the T-FAST Project’s strengths and weaknesses in the first two years of implementation? 4. Does the project create an inclusive environment for women to actively participate? If so, can you give examples of what you consider a good practice? What else can be done to improve women inclusion in the implementation of the T-FAST project? 5. Do you think the T-FAST project has the potential to significantly contribute to reducing the average time of trade in Paraguay by the end of 2023 with respect to the prevailing level in 2019? 6. What specific T-FAST activities were most effective in contributing to a reduction of average time of trade? Efficiency 7. Do you think the T-FAST project has focused on cost-effective activities during this first phase of implementation? Cost effective activities refers to activities that are expected to provide significant value in terms of trade facilitation with respect to the cost to deliver those activities. That is, they have a high value/cost ratio. Sustainability 8. From what you have observed so far, do you think local organizations represented in the NTFC will continue to carry on: (i) key initiatives started and (ii) maintaining the key results obtained by the T-FAST project so far? a. Note to enumerator, see above for example activities and results if prompt is needed/helpful 9. What could T-FAST do that would strengthen the sustainability of project activities and results? Challenges 10. What are the main external factors (where T-FAST staff have no control) that may have seriously affected the T-FAST project, either positively or negatively? Closing Questions 11. Is there any additional comment or reflection you would like to share with us? LINC LLC Page 64 T-FAST Project Instituto Desarrollo Midterm Evaluation Report 12. Is there anyone who is not a member of the NTFC who you suggest we speak with about the relevance, effectiveness, sustainability, or challenges of the T-FAST project? LINC LLC Page 65 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Annex 4: Midterm Evaluation Scope of Work PURPOSE AND SCOPE In compliance with the United States Department of Agriculture (USDA’s) monitoring and evaluation policy, IESC’s midterm evaluation will “critically and objectively review and take stock of the project’s implementing experience and environment, assess whether targeted beneficiaries are receiving services as expected, assess to what extent the project is on track to achieve its stated goals and objectives, review the results frameworks and assumptions, document initial lessons learned, and discuss necessary modifications or mid-course corrections that may be necessary to effectively and efficiently meet the stated goals and objectives.” The evaluation will examine both administrative and programmatic aspects of IESC’s Trade-Facilitating Agricultural Systems and Technology (T-FAST) project in Paraguay. Independence of the evaluation function from program implementation and management is a core principle of USDA evaluation. Independence helps to ensure both credible and objective evaluations. USDA-supported evaluations should be conducted by people who are not involved in the implementation and management of the project, and the evaluation process must be free from political influence and organizational pressure. For external evaluations, all evaluation team members will provide a signed statement attesting to a lack of conflict of interest or disclosing any real or potential conflicts of interest. Monitoring and evaluation activities should appropriately balance the desired creation of evidence with the protection of human subjects, including safeguarding the dignity, rights, safety, and privacy of participants. Evaluators are responsible for applying ethical principles in all stages of the evaluation, and for raising and clarifying ethical matters with stakeholders during the evaluation. AUDIENCE AND KEY STAKEHOLDERS The midterm evaluation is primarily designed to inform IESC (as the project’s implementer) about performance to date, in order to identify priorities and strategies for the remainder of the project, thus improving overall project performance and outcomes. The evaluation is also designed to provide USDA with material for congressional reporting and other internal purposes, and to provide an assessment of the reliability and accuracy of IESC’s reported data. Beyond making the midterm evaluation data available to USDA for their utilization, IESC will also share relevant results of the midterm to stakeholders in the public and private sector, and IESC’s partners on this project, Kansas State University, and the Center for International Private Enterprise (CIPE). KEY EVALUATION QUESTIONS The questions below will shape the midterm evaluation. The evaluation team may modify these questions or add additional questions to gather data related to the project indicators listed in the PMP, as reported by IESC. The evaluation team will review and revise these questions prior to the evaluation to improve clarity. The T-FAST Midterm Evaluation will answer the following questions related to the USDA standard evaluation criteria of relevance, effectiveness, efficiency, sustainability, and impact: • Relevance: To what extent have T–FAST expected results and activities to date addressed the core issues of target beneficiaries? Is the project relevant in addressing goals and objectives of the Trade Facilitation Agreement (TFA), particularly in assisting Paraguay with its Category C LINC LLC Page 66 T-FAST Project Instituto Desarrollo Midterm Evaluation Report notifications? Is the project coherent with other projects that support the TFA? Are the projects mutually supporting or duplicating efforts and if so, how? • Effectiveness: How effectively is the project reaching its targets; both timewise and per the work plan? Is T–FAST on track to achieve the specific targets and results established? Which activity or combination of activities has/have the highest potential to achieve T–FAST’s higher￾level result (expanded trade of agricultural products)? How effective is T–FAST at reaching women beneficiaries (particularly via capacity building activities) and what more could be done to improve women’s engagement / leadership? Which activities were most effective in contributing to a reduction of average time of trade? • Efficiency: To what extent are staffing, management, and oversight costs suitable, given the number and scope of activities carried out and the outputs obtained? • Sustainability: What factors contribute to the sustainability of project results and how has T– FAST focused on these to date? To what extent is progress being made towards project sustainability/graduation, and what could be done to strengthen the sustainability of project activities and results? • Impact: What are the immediate-, medium-, and long-term effects, both intended and unintended, positive and negative, of the project to date? • Challenges: o What are the external forces that negatively and positively impacted the project implementation and how did the project attempt to overcome, mitigate or capitalize its impact? o If external forces impacted the project implementation, what were the effects of this impact on different stakeholders (project staff, government counterparts, private sector stakeholders)? o What are the key challenges in collecting data and information and how did the project overcome or mitigate? o Have recommendations from the baseline evaluation been incorporated into the project and if not, why not? METHODOLOGY The stages of the mid-term evaluation include: 1. Desk Research 2. Prepare field research tools 3. Field research 4. Data analysis and prepare evaluation report The evaluation team will develop and operate according to a pre-defined work plan and will employ a variety of qualitative and quantitative methods, which will include review of secondary data from international organizations and government of Paraguay sources. In addition to existing secondary data, the evaluation team will employ quantitative surveys, focused group discussions, direct observation, and key informant interviews of key stakeholders including USDA representatives, T-FAST project team, key government officials, and representatives of relevant private organizations. The tools for collection of key data will consist of closed and open-ended instruments. The quantitative sample size of beneficiaries will be designed to reflect the project population of beneficiaries for each of the activities and will ensure a 95 percent confidence level with a 5 percent margin of error. LINC LLC Page 67 T-FAST Project Instituto Desarrollo Midterm Evaluation Report IESC will confirm details around survey design, interview questions, and sampling in conjunction with the contractor. IESC will also provide project documents to inform a preliminary desk review, e.g., program work plans, and semi-annual reports. The contractor will assess the situation and include any COVID-19 considerations in its data collection methodology as needed. Due to the likelihood that COVID-19 will still be prevalent during the midterm Evaluation period, all in￾person interactions and data collection with pre-identified beneficiaries will follow Paraguay’s public health protocols to protect the beneficiaries and the evaluation team. Virtual data collection methods successfully tested and used by the T-FAST program will be utilized when possible, including phone calls or video conference interviews, surveys conducted by emails, and online surveys using various platforms. T-FAST project staff will assist with site visit coordination once the external evaluator determines the sample of beneficiaries to visit, including providing contact data of beneficiaries and sensitizing beneficiaries to the evaluation (telling them that they may be contacted by evaluators, that IESC and USDA support the evaluation, and to be open and candid with evaluators. The evaluators mainly through their local consultants will reach all targeted collaborating organizations and a minimum stratified sample size calculated by the total anticipated beneficiary population. This is a performance evaluation and is intended to verify contributions and plausible links between impact and T-FAST activities through contribution analysis rather than scientific attribution through an experimental or quasi-experimental evaluation design. The concept of plausible association does not mean X input equal Y output. Rather, it means a case can be made that X input, has materially affected Y output. Contribution analysis will involve 1) gathering existing evidence on the various results and the links/assumptions that lead to these results; and 2) assembling and assessing the contribution story and the challenges to it. The challenges or weaknesses in the story point to where additional data or information is needed. The contribution analysis methodology is a useful approach when working in complex areas where assessment of sole attribution is difficult, such as in the space of agriculture and food safety in the context of local, regional, and international trade. The essential value of contribution analysis is that it offers an approach designed to reduce uncertainty about the contribution that T-FAST is making to the observed results through an increased understanding of why the observed results have occurred (or not) and the roles played by T-FAST and other internal and external factors. The conclusion from a contribution analysis is not definitive proof, but rather provides evidence and a line of reasoning from which T-FAST can draw a plausible conclusion that, at some level of confidence, the project has made an important contribution to the documented results. The evaluation contractor will apply the contribution analysis methodology using existing quantitative data from the program’s M&E system, qualitative information from interviews, and other contextual factors that help validate or clarify the contribution story. Any missing ‘piece’ of the story is where additional data or information is needed, which the evaluation contractor will collect. Quantitative data will be analyzed using the appropriate descriptive statistics rather than inferential statistics given there are no control or comparison groups. LINC LLC Page 68 T-FAST Project Instituto Desarrollo Midterm Evaluation Report EVALUATION MANAGEMENT AND COORDINATION The evaluations will be supported by the IESC home-office MEL director, who leads IESC’s dedicated evaluation unit based in Washington, DC in collaboration with the T-FAST Project Director and T–FAST MEL director, in conjunction with the IESC home-office program team. T–FAST project staff will assist the evaluator by providing logistical support during the baseline study and midterm and final evaluations. Roles and responsibilities for IESC staff and partners are outlined below. IESC Staff Role Project Director Providing necessary project documents. Introducing the team to stakeholders (government, private sector and NTFC) Getting government approvals for government site visits T-FAST MEL Director Logistical support to evaluation team. Identification of the required stakeholders. Setting up required stakeholder meetings and site visits. Home Office MEL Director Ensuring independence of the evaluator. Ensuring agreed methodologies are followed. Evaluation quality assurance. Home office T￾FAST project Team Providing project information. Liaison between TFAST project team and evaluator. Submission of evaluation reports. Evaluation contract management. ETHICAL GUIDELINES Every member of Contractor’s evaluation team must adhere to ethical guidelines as outlined in the American Evaluation Association’s Guiding Principles for Evaluators. A summary of these guidelines is provided below. • Systematic Inquiry: Evaluators conduct systematic, data-based inquiries. • Competence: The evaluation team possesses the education, abilities, skills, and experience appropriate to undertake the tasks proposed in the evaluation. Evaluators practice within the limits of their professional training and competence and decline to conduct evaluations that fall substantially outside those limits. The evaluation team collectively demonstrates cultural competence. • Integrity/Honesty: Evaluators display honesty and integrity in their own behavior and attempt to ensure the honesty and integrity of the entire evaluation process. • Respect for People: Evaluators respect the security, dignity and self-worth of respondents, project participants, clients, and other evaluation stakeholders. Evaluators regard informed consent for participation in evaluation and inform participants and clients about the scope and limits of confidentiality. • Responsibilities for General and Public Welfare: Evaluators articulate and consider the diversity LINC LLC Page 69 T-FAST Project Instituto Desarrollo Midterm Evaluation Report of general and public interests and values that may be related to the evaluation. • Evaluation activities must not endanger the evaluation team, stakeholders, and public health. CONTRACTOR TASKS The Contractor will undertake the following tasks: 1. Work Plan a. Participate in kick off coordination meetings with IESC and any IESC invitees, such as but not limited to representatives of USDA; b. Develop a mid-term evaluation work plan describing an understanding of the project based on desk review and the kickoff meeting; methodology; planned quality control measures; communications protocol; and a final timeline. 2. Pre-Assignment a. Provide IESC with draft data collection tools for review and comment; b. Develop final versions of the data collection tools incorporating IESC feedback; c. Develop an in-country travel schedule including, but not limited to, local travel dates and locations. d. Desk review of relevant program documents. 3. Conduct Primary Data Collection a. Travel to Paraguay to conduct primary data collection according to the agreed upon travel schedule. b. Review indicator results data and supporting documentation and verify calculations of key indicators results. 4. Post-Assignment a. Create clean and final versions of all quantitative datasets and qualitative transcripts in an agreed upon format; b. Develop a two- to three-page brief describing the evaluation design, key findings, other relevant considerations, and including graphics and tables, in layperson language to inform final evaluation stakeholders; c. Capture 15 – 20 high-quality photographs documenting data collection process;26 d. Give presentation of brief to IESC and any IESC invitee such as, but not limited to, USDA on the evaluation findings and materials; e. Develop a trip report following international travel to Paraguay. 5. Final Evaluation Report 26 Photographs are intended for IESC deliverable verification purposes only. Photographs of individuals that can reasonably be identified should only be used in the public-facing evaluation report if the participant was informed that their picture will be used in the report that will be publicly available, and consents to its use. LINC LLC Page 70 T-FAST Project Instituto Desarrollo Midterm Evaluation Report a. Create a draft of the T-FAST Midterm Evaluation report in English; as an editable Word document; and addressing all evaluation objectives and questions; b. Create an electronic final English version of the T-FAST Midterm Evaluation report in PDF and Word for internal IESC use. There are no set page limits; however, a midterm evaluation report is expected to be around 50-80 pages (not including annexes). Annexes within this version may contain proprietary information owned by third parties; information that could put individual safety at risk; and personally identifiable information. The report should include, but not limited to the following: i. List of acronyms/abbreviations; ii. Table of contents; iii. Executive summary; iv. Background, which should include a snapshot of the import/export landscape, e.g. largest ports, most-traded commodities by volume, etc. Background should consider overall trade picture since the baseline study and so provide context for time trade estimates, e.g. was there one or more major anomaly which disrupted trade flows during the time period. Background should also be inclusive of how project activities changed or adapted because of the COVID-19 pandemic and any other external contextual factors. Background may also include how the pandemic impacted market dynamics for different value chains. v. Detailed evaluation methodology; vi. Midterm evaluation findings: vii. Lessons Learned; viii. Recommendations for IESC for the remainder of the program; ix. Annexed Indicator and Results Table x. Annexed IESC response to findings xi. Annexed scope of work; xii. Annexed data collection instruments; and, xiii. Annexed photographs from meetings, or if work is performed remotely, screen shots ofvirtual meetings. Note: The document should include infographics and text boxes that include important data points and capture the reader’s attention. c. Create an electronic final English version of the T-FAST Midterm Evaluation report in PDF and Word public distribution without any proprietary information owned by third parties; information that could put individual safety at risk; and personally identifiable information in the report itself or annexes. The report should include, but is not limited to, the same elements as the final report for IESC internal use. Final versions of evaluation reports ready for publication should be accessible to persons with disabilities. For guidance on creating documents accessible to persons LINC LLC Page 71 T-FAST Project Instituto Desarrollo Midterm Evaluation Report with disabilities, please seethe following resources: https://www.section508.gov/create/documents https://www.section508.gov/create/pdfs LINC LLC Page 72 T-FAST Project Instituto Desarrollo Midterm Evaluation Report DELIVERABLES AND EXPECTED DUE DATES Deliverable Name Due Date 1. Work Plan • One (1) kick off meeting with contractor and IESC teams During 1st week after signature of the contract modification. (January 2022) • One (1) draft evaluation work plan that describes the following: o Understanding of the project based on desk review of beneficiary lists and all relevant program documents including, but not limited to, the baseline evaluation program reports, and program workplans; and kick-off meeting; o Mid term evaluation methodology, including detailed sampling plan, field work plan, and any limitations of the proposed approach; o Description of planned quality control measures; o Communication protocol with interview subjects related to purpose of interview, the project, and consent for participation and/or inclusion of subject in photograph captured during the study; and, o Final timeline. Within 2 weeks after IESC delivers beneficiary lists and all relevant program documents. • One (1) final evaluation work plan incorporating comments from IESC team. 5 business days after receiving IESC feedback 2. Pre-Assignment • One (1) folder draft data collection tools for IESC team comment. 2 weeks after submission of draft work plan. • One (1) folder electronic copies of all clean and final versions of data collection tools. 1 week after receiving IESC feedback • One (1) in-country travel schedule including, but not limited to, local travel dates and locations and planned meeting descriptions with potential participants. Two weeks prior to travel and no later than March 15, 2022 3. Post-Assignment • One (1) two- to three-page preliminary brief describing the evaluation design, initial and broad key findings, other relevant considerations, and including some graphics and tables as needed, in layperson language to inform final evaluation stakeholders. 1 week after return from in-country data collection • One (1) trip report following international travel to Paraguay. Contractor must submit a detailed trip report following international travel to Paraguay for in-country data collection efforts. Trip report must include the following attachments: o Proof of compliance with “Fly America” regulations; o Full receipts or other proof of lodging; o Proof of meals and incidental expenses rates using current U.S. State Department per diem rates for Paraguay. 1 week after return from in-country data collection LINC LLC Page 73 T-FAST Project Instituto Desarrollo Midterm Evaluation Report • One (1) virtual presentation to IESC and any IESC guest on the preliminary and broad evaluation findings and materials. 2 weeks after return from in-country data collection 4. Final Evaluation Report • One (1) draft of the T-FAST MidTerm Evaluation report in English; as an editable Word document; and addressing all evaluation objectives and questions, for IESC team and USDA comment. • One (1) Clean and final version of all quantitative datasets and qualitative data in an agreed upon format as appropriate, presented as an annex to the draft report. • Fifteen (15) to twenty (20) high-quality photographs documenting data collection process. 3 weeks after return from in-country data collection • One (1) electronic final English version of the T-FAST Midterm Evaluation report in PDF and Word for internal IESC use. The report should include but is not limited to the elements described under Contractor Tasks, Item 5. Note: The report might go through a series of reviews by USDA. The contractor should be prepared to respond to USDA’s comments, for potentially multiple rounds of review and comment, before IESC approves the report as final. 2 weeks after receiving IESC and USDA feedback on the draft • One (1) electronic final English version of the T-FAST Midterm Evaluation report in PDF and Word for public distribution without proprietary information owned by third parties; information that could put individual safety at risk; and personally identifiable information. The report should include but is not limited to the elements described under Contractor Tasks, Item 5. Note: The report might go through a series of reviews by USDA. The contractor should be prepared to respond to USDA’s comments, potentially multiple rounds of review and comment, before IESC approves the report as final. 2 weeks after receiving IESC and USDA feedback on the draft • One (1) virtual presentation to IESC and USDA Foreign Agricultural Service (FAS) of the midterm evaluation findings and recommendations. Within 2 weeks after notification from IESC that USDA approved the final version of the midterm evaluation. LINC LLC Page 74 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Annex 5: Status of Actions Planned by Activity and Sub-Activity The following table provides a list of the Year 2 planned T-FAST activities with start and end dates for each, and includes the evaluation team’s assessment of completion status based on semi-annual reports. Completed activities are those completed as planned for the Year 2. Activities in progress are those initiated but not concluded. Activities not initiated are those that are still in the planning process or not reporting any progress. The level of progress indicated in the “Status” column is based on the criteria of the evaluation team and focuses specifically on the progress reported in Year 2, given that in Year 1 no activity was reported as completed. Further, if the T-FAST semi-annual reports for Year 2 did not provide an update on a planned activity, the evaluation team considered this not progressed beyond the status from the previous report (if any). This midterm evaluation presents an overview of the level of completion of planned activities as of the close of Year 2 of the project for the purpose of gaining additional perspective on the effectiveness and efficiency of T-FAST. Table 20: Status of Actions Planned by Activity and Sub-Activity (Year 2) Activity, Sub-Activity, Action Start Date End Date Status Activity 1: Capacity building for the National Trade Facilitation Committee (NTFC) 1.1. Establish and Coordinate Steering Committee Generate a monthly T-FAST bulletin 10/1/2020 9/30/2021 Completed Conduct meetings as needed with the Steering Committee 10/1/2020 9/30/2021 Completed 1.2. Conduct Initial Assessment of the Trade Facilitation Environment for Food and Agricultural Goods Procure the diagnostic services locally 4/1/2021 5/30/2021 Not Initiated Conduct air transportation import/export process analysis 6/1/2021 9/30/2021 Not Initiated Conducted a Paraguay Agriculture Laboratory Capacity Assessment (continuation) 10/1/2020 11/30/2020 Completed Establish 12-month action plans 10/1/2020 12/31/2020 In Progress Implement 12-month action plans based on assessments to include trainings, lab equipment, ICT equipment and software development. 2/2/2021 9/30/2021 Not Initiated 1.3. NTFC Working Groups Re-organization Conduct virtual one-on-one and group interviews with all NTFC members, in coordination with UNCTD 8/1/2020 10/30/2020 Completed Develop a work group reorganization proposal and validation, in coordination with UNCTD 11/1/2020 12/30/2020 Completed Contribute to the discussions related to NTFC work group reorganization, working group mandates, roles, and guidelines in collaboration with UNCTAD. 12/1/2020 3/30/2021 In Progress Propose internal working rules for each work group, guidance based on previous experiences and best practices. 1/1/2021 2/28/2021 Completed Present regional examples that will work on TFA Article 5.3 – lab testing to determine lab learning trip destination 3/2/2021 3/31/2021 Not Initiated LINC LLC Page 75 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Activity, Sub-Activity, Action Start Date End Date Status Design a four to eight member learning trip to witness international lab practices 4/1/2021 4/30/2021 Not Initiated Conduct Lab Learning Trip 6/1/2021 6/30/2021 In Progress Complete the development of action plans for related to Laboratory Testing- Article 5.3 7/1/2021 8/30/2021 Not Initiated Present regional examples that will work on TFA Article 7 – Time Release and Expedited Shipping to determine learning trip destination 4/1/2021 4/30/2021 Not Initiated Design a four to eight member learning trip to witness international lab practices 5/2/2021 5/31/2021 In Progress Conduct Expedited Shipping Learning Trip 7/1/2021 7/30/2021 Not Initiated Complete the development of action plans for related to Time Release and Expedited Shipping - Article 7 8/1/2021 8/30/2021 Not Initiated Present regional examples that will work on TFA Article 8 – Border Integration to determine learning trip destination 5/2/2021 5/30/2021 Not Initiated Design a four to eight member learning trip to witness international lab practices 4/1/2021 4/30/2021 Not Initiated Conduct Border Integration Learning Trip 8/1/2021 8/30/2021 Not Initiated Complete the development of action plans for related to Border Integratión - Article 8 9/1/2021 9/30/2021 Not Initiated 1.4. Build Core Capacity of the NTFC Establish Government to Government MOU LoC with NTFC (MOFA) 10/1/2020 10/30/2020 Completed Develop a trainings calendar for NTFC members 2/2/2021 3/30/2021 Completed Implement the trainings calendar will begin after the validation according to the agreed calendar 4/1/2021 9/30/2021 Completed Facilitate discussions with NTFC members and staff on NTFC operational and structural priorities and staff management capacity 2/2/2021 2/28/2021 Completed Present international best practices, including regional NTFC examples to determine learning trip destination 2/1/2021 2/28/2021 Not Initiated Design a four to eight member learning trip to witness international lab practices 4/1/2021 4/30/2021 Not Initiated Conduct NTFC Learning Trip 5/1/2021 5/31/2021 In Progress Complete the development of action plans 6/1/2021 6/30/2021 Not Initiated Hiring an integrated Communications Consultant at NTFC 11/2/2020 11/30/2020 Completed Develop and validate of the NTFC Communications Plan by NTFC leadership 11/1/2020 12/30/2020 Completed Update and implement the NTFC Communications Plan 1/3/2021 9/30/2021 Completed Elaborate comunication reports 1/30/2021 9/30/2021 In Progress Hiring an Integrated M&E Specialist at NTFC 11/1/2020 11/30/2020 Completed Procurement of a tracking tool software 11/1/2020 11/30/2020 In Progress Conduct a half-day vision and mission virtual session facilitated by the T-FAST and UNCTAD 11/1/2020 12/30/2020 Not Initiated LINC LLC Page 76 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Activity, Sub-Activity, Action Start Date End Date Status Capture relevant milestones, timelines, and targets that relate to the indicators integrated into a MEL Plan 1/1/2021 2/28/2021 Completed Update specific indicators, collect data, analyze results, and develop a system and control panel for use by NTFC members 2/2/2021 9/30/2021 Completed Elaborate M&E Reports ######## 9/30/2021 Completed Develop regular meetings with Donor Coordination 10/1/2020 9/30/2021 Not Initiated Activity 2: Capacity building of government institutions 2.1. Support NTFC related to Publication/Opportunity to Comment Develop and validate of the Action Plan related to Public Comments and Reception of Policy Proposals 10/1/2020 2/28/2021 Completed Support policy development that allows government institutions to receive policy proposals 2/28/2021 8/30/2021 Completed Implement action plans related to Public Comments and Reception of Policy Proposals 2/1/2021 9/30/2021 Completed Follow-up training and support 11/1/2020 9/30/2021 Completed 2.2. Support NTFC related to Laboratory Testing Develop assessment that will determine the initial round of trainings, technical recommendations and suggest technological upgrades 10/1/2020 12/30/2020 Completed Assist in technical data collection with a local lab consultant 10/1/2020 11/30/2020 Completed Present findings and initial recommended actions from the first group of lab assessments to the Steering Committee 12/1/2020 12/30/2020 Completed Present findings and initial recommended actions from the second group of lab assessments to the Steering Committee 3/1/2021 3/31/2021 In Progress Implement the components of the 12-month action plans that are non-ICT related (not computer systems and networks) for the first group of labs 2/1/2021 9/30/2021 Completed Implement the components of the 12-month action plans that are non-ICT related (not computer systems and networks) for the second group 4/1/2021 9/30/2021 In Progress Follow-up training and technical assistance 7/1/2021 9/30/2021 Not Initiated Develop ICT action plans for laboratory assistance that have been validated by the lab authorities 11/2/2020 6/30/2021 Completed Implement 12-month action plans at laboratories 2/1/2021 9/30/2021 In Progress Develop and validate Terms of Reference for 12-month action plans at laboratories 2/15/2021 7/30/2021 In Progress Develop and validate the action plan to connect private laboratories with VUE-VUI so that test results can be reported directly to the authorities 7/1/2021 9/30/2021 In Progress Follow-up training and technical assistance 7/1/2021 9/30/2021 Not Initiated LINC LLC Page 77 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Activity, Sub-Activity, Action Start Date End Date Status 2.3. Support NTFC related to Time Release and Expedited Shipping Leveraging MITIC’s ICT Capacity to Support Trade Facilitation Systems 10/1/2020 9/30/2021 Completed Collaborate with the Ministry of the Interior to support the new electronic ID roll out 10/1/2020 10/31/2020 Completed Conduct individual virtual presentations to all organizations hesitant to implement e-signatures 11/1/2020 3/30/2021 Not Initiated Develop and validatate of the Action Plan 1/2/2021 9/30/2021 Not Initiated Implement E-Signature Plan 4/30/2021 9/30/2021 Not Initiated Coordinate with the WB to ensure its commitment to break down time release information for agricultural and non-agricultural products 11/3/2020 11/27/2021 In Progress Support INAN with data digitization and automate export and importer registration processes as well as verification. 10/1/2020 8/31/2021 Completed Develop ICT for the incorporation of SIGRA/INAN into the VUE 10/1/2020 3/29/2021 Completed Develop and validate ToR for international signature contract for INTN white paper production 10/2/2020 11/30/2020 In Progress Hiring for an international firm to analyze the regulatory changes needed for the institutional strengthening of INTN in a white paper 12/1/2020 2/28/2021 In Progress Conduct the study and draft the INTN white paper 2/1/2021 6/30/2021 Completed Socialize the INTN white paper to public-private 7/1/2021 7/30/2021 Completed Develop and validate a action plan to completely digitalize INTN’s service management platform 9/1/2020 11/30/2020 Not Initiated Implement the action plan 12/1/2020 9/30/2021 Not Initiated Develop a communications plan for INTN 10/1/2020 3/30/2021 Completed Implement trainings calendar for INTN detailed in the action plan 12/1/2020 9/30/2021 Completed Gap analysis assessment of stakeholder's capacity to accept electronic payments and deliver electronic receipts 10/1/2020 10/30/2020 Completed Develop and validate a Action Plan for improved payment delivery process 11/1/2020 4/30/2021 Completed Implement Electronic Payments Expansion Plan 5/2/2021 9/30/2021 Completed Develop the terms of reference the international specialist 9/1/2020 10/30/2020 In Progress Hiring an international specialist 12/2/2020 2/2/2021 Not Initiated Conduct the International conceptual conference 3/1/2021 3/31/2021 Not Initiated Develop workshops on electronic certification for SENACSA officials with international specialists 4/1/2021 5/31/2021 Not Initiated Develop and validate an Electronic Certification Implementation Plan for SENACSA 6/1/2021 6/30/2021 Not Initiated LINC LLC Page 78 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Activity, Sub-Activity, Action Start Date End Date Status Develop "fichas técnicas" for the processes and equipment necessary for the Electronic Certification Implementation Plan for SENACSA 7/1/2021 7/30/2021 Not Initiated Implementation of the Electronic certification and Modernization Plan 7/2/2021 9/30/2021 Not Initiated Assess the Backend systems at SENAVE 10/1/2020 10/30/2020 Completed Develop a Backend Management Systems Action Plan for SENAVE that correlates with the SENAVE Process Reorganization and e-Phyto Implementation Plan 11/1/2020 12/30/2020 Completed Develop and validate with SENAVE the Terms of References to procure support for SENAVE’s and the VUE’s in-house programmers 3/2/2021 3/30/2021 Not Initiated Implement the Backend Management Systems Action Plan 4/2/2021 9/30/2021 Completed Assess SIGOR’s ICT systems and the partnering VUE and VUI systems related to the issuing of zoosanitary certificates 9/1/2020 11/30/2020 Not Initiated Develop a Backend Systems and Digital Interfaces Action Plan linked with SENACSA's inhouse "Management System for Regional Offices" (SIGOR) 12/1/2020 2/1/2021 Not Initiated Develop and validate with SENACSA the Terms of References to procure support for SENACSA’s and the VUE’s in-house programmers 2/2/2021 3/30/2021 Not Initiated Implement Backend Systems and Digital Interfaces Action Plan at SENACSA 4/1/2021 9/30/2021 Not Initiated Develop backend management systems Action Plan for DINAVISA 11/1/2020 2/1/2021 Completed Develop and validate Terms of Refence for ICT hardware and supporting software development for “DNVS” software implementation 3/2/2021 4/30/2021 Completed Conduct trainings and Kaizen style workshops related to DNVS software implementation and process re￾engineering 5/3/2021 9/30/2021 In Progress Validate and deliver updated procedural manuals for DINAVISA's processes related to agrichemical and other agricultural products. 9/1/2021 9/30/2021 In Progress Assess the Hardware and software of SENAVE , in collaboration with the VUE, COSAVE and the Ministry of Information and Communication Technology 9/1/2020 11/30/2020 In Progress Develop and validate by SENAVE and VUE the SENAVE Process Reorganization and e-Phyto Implementation Plan 12/1/2020 3/30/2021 Completed Develop and validate "Fichas Técnicas" for software development related to ePhyto implementation at SENAVE 4/1/2021 4/30/2021 Not Initiated Implement SENAVE Process Reorganization and e-Phyto Implementation Plan 5/1/2021 9/30/2021 Not Initiated LINC LLC Page 79 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Activity, Sub-Activity, Action Start Date End Date Status Develop an AEO framework implementation plan for institutions that request 10/1/2020 9/30/2021 Completed Conduct the initial presentation of the AEO program accompanied by USAID representatives and Customs AEO Coordinator 11/1/2020 11/30/2020 Completed 2.4. Support NTFC related to Border Cooperation Explore harmonization of opening hours at SENACSA and SENAVE border crossings 10/1/2020 6/30/2021 Completed Develop the Terms of Reference for this legal and technical assistance in collaboration with SENAVE, MoFA and CIPE 10/1/2020 12/30/2020 In Progress Procurement of legal and technical assistance for SENAVE will be conducted 1/2/2021 2/28/2021 Completed Provide Legal and Technical Assistance to SENAVE related to regulatory reform of the Integrated Control Areas 3/1/2021 9/30/2021 Not Initiated Engage regional Phytosanitary and Zoosanitary technical staff and authorities to broaden acceptance of electronic certifications 3/1/2021 3/30/2021 In Progress Establish LoCs at the project level with CIP, CAPECO, CATERPPA and AMCHAM 6/1/2020 11/30/2020 In Progress Other activities Establish LoCs at the project level with MIC, INTN and Customs 2/3/2020 5/29/2020 In Progress Establish LoCs at the project level with SENACSA, MITIC, MSPBS for INAN and DINAVISA 2/3/2020 5/29/2020 Completed Activity 3: Capacity building of trade associations 3.1. Build Capacity of Private Sector to Engage in Trade Facilitation Reform Assess training needs at associations to work with their members to comment and propose reforms related to trade facilitation processes 10/1/2020 3/30/2021 Completed Develop the training action plan 4/2/2021 4/30/2021 Completed Implement specific training action plan for each private stakeholder 5/3/2021 9/30/2021 In Progress Develop an AEO promotion action plan in collaboration with Custom’s AEO Coordinator to effectively promote the AEO certification 3/3/2021 3/30/2021 Completed Implement action plan 5/1/2021 9/30/2021 In Progress Develop communication materials 3/3/2021 4/30/2021 Not Initiated Develop and implement the action plan 4/3/2021 9/30/2021 In Progress Activity 4: Provision of market and trade infrastructure 4.1. Develop Infrastructure to Implement NTFC Priorities related to Publication/Opportunity to Comment LINC LLC Page 80 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Activity, Sub-Activity, Action Start Date End Date Status Purchase of software and licenses to support an ICT system that allows government agencies to receive policy proposals from the public 5/3/2021 9/30/2021 Completed 4.2. Develop Infrastructure to Implement NTFC Informal Producers to Formal Market Purchase of laboratory equipment related to food and agricultural testing, as well as agrochemicals, aligning to EU/WB and IDB equipment purchase plans 3/1/2021 6/30/2021 Completed 4.3. Develop Infrastructure to Implement NTFC Priorities related to Time Release and Expedited Shipping Procure the necessary ICT infrastructure and software development at INAN 9/1/2020 11/30/2020 In Progress Procure ICT equipment and software development related to e-signatures in key public institutions with buy-in 4/1/2021 7/30/2021 In Progress Procure ICT equipment and software development to expand and improve electronic payments in key public institutions 5/2/2021 8/30/2021 Completed Procure ICT equipment and software development to support the process re-engineering and digitalization as well as the implementation of the “DNVS” Sanitary Registration Software at DINAVISA 5/2/2021 8/30/2021 Completed Procure the necessary ICT infrastructure and software development to support INTN’s service management platform 1/2/2021 6/30/2021 Completed Procure ICT equipment and software development necessary to expand the AEO program in public institutions with buy-in 6/1/2021 9/30/2021 Completed Procure an online training platform and subcontract the software development and/or adaption necessary 11/1/2020 11/30/2020 Completed Procure the necessary ICT infrastructure and software development based on the "SENAVE Process Reorganization and e-Phyto Implementation Plan" 5/1/2021 9/30/2021 Completed Procure the necessary ICT infrastructure and software development related to e-Cert based on the ICT assessment of SENACSA’s SIGOR systems and partnering VUE and VUI systems 8/1/2021 9/30/2021 Completed LINC LLC Page 81 T-FAST Project Instituto Desarrollo Midterm Evaluation Report Annex 6: Articles of Trade Facilitation Agreement Supported by T-FAST The table below provides information on the articles of the TFA that T-FAST supports the NTFC related to Paraguay’s Category C Notifications. Table 21: Articles of WTO TFA that T-FAST Supports Article Time Period Article 1 Publication and Availability of Information 1.1. Publication 1.2. Information available through internet 1.3. Enquiry points 1.4.Notification September 2021 Article 2 Opportunity to Comment, Information Before Entry into Force 2.1. Comments and information before entry into force 2.2. Consultations September 2020 Article 5.3. Test procedures September 2023 Article 7.6. Average release times September 2021 Article 7.7. Authorized operators September 2020 Article 7.8. Expedited shipments September 2022 Article 8 Border Agency Cooperation September 2024