Evaluation of the Transforming the Assessment and Inspection of Food Businesses (TAIB) in Egypt Project Midterm Evaluation 11/5/2021 DISCLAIMER: This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Evaluation of the Transforming the Assessment and Inspection of Food Businesses (TAIB) in Egypt Project Agreement Number: FCC-263-2018/003-00 Funding Year: Fiscal Year 2018 Project Duration: September 30, 2018 – June 30, 2024 Implemented by: Land O’Lakes Venture37 Evaluation Authored by: Clifford Zinnes, Ali Protik, Ridhi Sahai, and Jessica Wallach Acknowledgement This report would not have been possible without the efforts and contributions from a number of people. First, we would like to thank our colleagues at Land O’Lakes Venture37 for providing valuable information regarding the TAIB project that were key in our planning, data collection, and analysis. We are especially grateful for the inputs and comments we received from Meredith Saggers, Alexander Samel, Khaled Abdel Monem, and Sadie Paschke at various stages of this report. We are grateful for the superb qualitative data collection efforts by Zeinab Khadr from Cairo University and Mohamed Hassan Hosni from American University at Cairo. Finally, we would like to thank Sewit Tesfaselassie for her diligence in formatting the report. Table of Contents Acknowledgement......................................................................................................................................... i Table of Contents.......................................................................................................................................... ii List of Acronyms........................................................................................................................................... iv Executive Summary....................................................................................................................................... v Project Background and Purpose.............................................................................................................. v Evaluation Questions, Design, Methods and Limitations......................................................................... v Findings: Trade Analysis........................................................................................................................... vi Findings: Performance Evaluation .......................................................................................................... vii Recommendations.................................................................................................................................. vii 1. Introduction ..........................................................................................................................................1 1.1. Background: Country Context.......................................................................................................1 1.2. Food-Inspection Practices Prior to the TAIB Project.....................................................................3 1.3. Food Safety in Egypt......................................................................................................................3 1.4. The Egypt TAIB Project..................................................................................................................4 1.5. Theory of Change..........................................................................................................................5 1.6. Overview of Evaluation Methods and Questions.........................................................................6 2. Midline Analysis of Trade......................................................................................................................8 2.1. Impacts on Exports and Imports...................................................................................................8 2.1.1 Outcome Indicators and Data Sources ........................................................................................8 2.1.2 Methodological Approach............................................................................................................9 2.1.3 Findings: Prediction Accuracy of the Counterfactual.................................................................12 2.1.4 Findings: Impact of TAIB on Exports and Imports......................................................................16 2.2. Movement toward increasing trade ...........................................................................................20 2.3. Mitigating COVID-19 bias from trade-impact estimates ............................................................21 3. Performance Evaluation......................................................................................................................22 3.1. Methodology...............................................................................................................................22 3.1.1 Data Sources for the Performance Evaluation...........................................................................22 3.1.2 Data Collection and Analysis......................................................................................................23 3.2. Performance Evaluation Findings ...............................................................................................24 3.2.1 Alignment with Key Stakeholders..............................................................................................24 iii 3.2.2 Progress and Impacts of TAIB activities and Effectiveness in Reaching Targets........................26 3.2.3 Evaluation of TAIB efficiency......................................................................................................35 3.2.4 Key Success Factors and Challenges for the TAIB project..........................................................36 3.2.5 Sustainability of TAIB Project Activities.....................................................................................39 4. Conclusions and Recommendations...................................................................................................40 4.1 Trade Analysis Findings.....................................................................................................................40 4.2 Performance Evaluation Findings .....................................................................................................41 4.3 Recommendations............................................................................................................................42 Annex I. TAIB Performance Indicators and Targets for FY2021 and Life of the Project .............................44 Annex II. List of Treated and Untreated Categories....................................................................................46 Annex III. Supplement to Midline Trade Analysis.......................................................................................50 Annex to 2.1.2: Econometric estimation of ITSPO framework impact estimates..................................50 Annex to 2.1.3: Prediction accuracy of counterfactual for country-level matching...............................52 Annex to 2.2: Movement toward increasing trade................................................................................54 Annex to 2.3: Mitigating COVID-19 bias from trade-impact estimates.................................................56 Annex IV. Data Collection Tools..................................................................................................................60 NFSA KII Questionnaire - Midline............................................................................................................60 TAIB Staff KII Questionnaire - Midline ....................................................................................................62 Chamber of Food Industries KII Questionnaire - Midline .......................................................................65 3rd Party Certification Companies KII Questionnaire - Midline...............................................................67 Food Businesses KII Questionnaire – Midline.........................................................................................69 USDA: KII Questionnaire – Midline .........................................................................................................71 FGD Guide for NFSA Inspectors - Midline...............................................................................................73 Annex V. COVID-19 Safety Protocols..........................................................................................................76 iv List of Acronyms Acronym Full Term CFI Chamber of Food Industry DID Difference-in-differences model FDA US Food and Drug Administration FE Final Evaluation FGD Focus group discussion GDP Gross domestic product GOE Government of Egypt HS Harmonized Commodity Description and Coding Systems IFPTI International Food Protection Training Institute ITSPO Interrupted time-series analysis within potential outcomes KII Key informant interview MTE Midterm evaluation NFSA National Food Safety Authority NORC National Opinion Research Center SITCSITC Standard international trade classification SOP Standard Operating Procedure TAIB Transforming the Assessment and Inspection of Food Businesses in Egypt UNSD United Nations Statistics Division USDA The United States Department of Agriculture USG United States Government v Executive Summary Project Background and Purpose In Egypt, until as recently as 2016, oversight of food laws and regulations was spread across 17 agencies and 7 ministries. The responsibilities of each agency were overlapping, leading to inefficiencies and inconsistencies in enforcement. On January 2, 2017, the Parliament of Egypt passed law number 1/2017 establishing the National Food Safety Authority (NFSA) and on February 19, 2019, the government issued and published NFSA’s Executive Regulations, promulgating the agency’s authority. The law and executive regulations empower NFSA with the sole mandate to control and regulate food in Egypt, over￾seeing food safety from harvest to the Egyptian consumer. NFSA’s main responsibilities include food law and regulation harmonization, establishment of industry-specific protocols for regulations, food-industry inspection and approvals, risk assessments, crisis management, and consumer outreach. The United States Department of Agriculture (USDA) is supporting the development of NFSA through the Food for Progress Program: Transforming the Assessment and Inspection of Food Businesses in Egypt (Egypt TAIB), awarded in 2018 and implemented by Land O’ Lakes Venture37 (Venture37) and the International Food Protection Training Institute (IFPTI). The program was originally 5 years but has been extended to five-years and 9 months, with a budget of $8.5 million. Egypt TAIB is implemented through three distinct activities that is designed to: 1) enhance the organizational capacity of NFSA, including supporting the creation of administrative and regulatory policies and procedures; 2) develop a workforce training program for NFSA modeled after USDA and the US Food and Drug Administration (FDA), leveraging existing training curricula offered throughout the US to food safety regulatory personnel; and 3) promote improved policy and regulatory frameworks, to align Egypt’s food safety standards with international best practices that promote increased trade. The TAIB project is expected to result in expanding the domestic and international trade of Egyptian food products. Evaluation Questions, Design, Methods and Limitations Venture37 has contracted NORC @ the University of Chicago to conduct the evaluation of the Egypt TAIB project. To evaluate the TAIB project, NORC employed a mixed-method approach with quasi-exper￾imental and non-experimental designs. In particular, we are using two separate evaluation methods: (1) Macro-level evaluation: quasi-experimental interrupted time-series analysis within a potential outcomes framework (ITSPTO) using trade data to examine impacts of TAIB on the values of exports and imports for the food subsectors that are subject to NFSA oversight. (2) Performance evaluation: assessments based on document reviews, reviews of project monitoring data and assessment tools, key informant interviews, and focus group discussions. We conduct a quasi-experimental impact evaluation to determine whether we can detect changes in the values of exports and imports that can be attributed to the TAIB project. The evaluation design requires comparing actual trade to its counterfactual, that is, whattrade would have been without the TAIB intervention. The approach involves comparing trade values of specific product categories subject to NFSA inspection – the intervention group – to counterfactual values synthesized from trade categories not regulated by NFSA but containing products with characteristics of differing degrees of similarity to those in the intervention group – vi the comparison group. The latter synthesis is carried out using a statistical matching technique. This approach is called the interrupted time-series analysis within a potential outcomes (ITSPO) framework. During the midline, we also conducted a non-experimental performance evaluation using document reviews, key informant interviews (KIIs) and focus-group discussions (FGDs). The goal of the performance evaluation is to provide a wholistic account of TAIB project’s progress on its activities to date, assesses its impacts, and how it has performed against its targets. We also examine the success factors to TAIB’s performance, key challenges that it faced, and the extent to which the TAIB activities are sustainable. Findings: Trade Analysis We analyze the impacts on TAIB on Egyptian exports and imports based on two outcome indicators1 : • Value (in millions of 2010 constant US Dollars) of exports; and • Values (in millions of 2010 constant US Dollars) of imports. The data on trade flows come from the United Nations International Trade Statistics Database or UN Comtrade Database (https://comtrade.un.org/data/), the largest depository of international trade data, where data on the trade values (in US dollars) and volumes (net weight) of various commodities (including food products) are available on a quarterly and annual basis. This database, going back as far as 1962, is maintained by the United Nations Statistics Division (UNSD), to whom over 170 countries report annual international trade statistics data detailed by partner countries and commodities/service categories. The findings from the trade analysis are summarized below: • TAIB has had an impact on exports in 2019 that were almost 12-percent higher than what they would have been had TAIB not been implemented. This effect translates into a US$ increase in exports of almost 293 million (constant 2010). • TAIB has had an impact on imports in 2019 that were ten-percent higher than what they would have been had TAIB not been implemented. This effect translates into a US$ increase in imports of almost one billion (constant 2010). Thus, TAIB had an attributable impact of US$ 1.25 billion on total trade of NFSA-regulated products in 2019 (10.4% higher than what they would have been had TAIB not been implemented). • TAIB’s has had an impact on exports in 2020 that were almost 8-percent higher than what they would have been had TAIB not been implemented and the COVID-19 pandemic not occurred. This effect translates into a US$ increase in exports of almost 185 million (constant 2010). • TAIB’s impact on imports in 2020 is uncertain. Despite the evaluation team’s best effort, we could not calculate a credible estimate for TAIB’s impact on imports in 2020. The issue may be that some of the comparison imports experienced infrequent trade flows over the analysis period, resulting in significant volatility. One of the biggest challenges facing the present impact evaluation is that the onset of the pandemic took place between the start-up of TAIB and (for now) the midline. This requires that we statistically dis￾entangle the COVID-19 effects on Egypt’s trade flows from those of TAIB for detecting actual impacts on 1At baseline, we used trade volumes as additional outcomes. However, our exploration of the available data for the volume of trade indicated that the data is of poor quality and any analysis conducted with trade volume as the outcome indicator will not be reliable. As such, we recommended and agreed with USDA to use trade values as the only outcome indicator for the midline and endline analysis. vii trade. While we developed analytic tests to disentangle the TAIB effects from the COVID effects, key data elements from 2020 and 2021 required to carry out the tests were not publicly available at this time. While we have more confidence in the impact estimates for 2019, we recommend caution in using and interpreting the impacts estimates for 2020. Findings: Performance Evaluation TAIB has made significant progress in almost all activity areas against its targets for Activity 1, institutional capacity building. TAIB developed an assessment tool to track the progress in increasing NFSA institutional capacity that the evaluation team found very useful. By the end of project year 3, the score exceeded the Project Year 3 target and represents 85.7% of the target for the life of the project. One of the significant achievements in this activity was the development of inspection protocols. TAIB successfully developed trainings and adopted a train-the-trainer approach that was lauded by NFSA staff, inspectors, and other trainees. TAIB brought, for the first time, a suite of food safety training programs approved by the FDA to a foreign food competent authority leveraging the expertise of American food safety professionals to strengthen Egypt’s food inspectorate workforce. Although the COVID-19 pandemic was disruptive, TAIB was able to continue work on activities thanks to pre￾pandemic investments in digitization and further measures such as setting up institutional zoom accounts for NFSA staff and creating an online Learning Management Platform (LMS) to continue trainings digitally. TAIB has been able to train close to 1,500 people in many different areas over the life of the project. Although the project is on track to meet its year 3 target, it must accelerate the rate of training to meet the target of training 2,821 people by project’s end. TAIB’s support has been enormously impactful in the realm of policies and regulations. As of the first half of year 3, TAIB has achieved the passage of 12 policies, regulations, and administrative procedures, that surpassed the original target. The most significant regulations supported by TAIB was Decision No. 7 of 2020, which was the technical regulation for the risk-based design of food import control, and Decision No. 13 of the Year 2020, which removed Egypt’s zero tolerance on Ractopamine. The flexibility of the TAIB project has been a key factor in the success of the project despite the challenge of frequently pivoting strategies and priorities. Expanding the reach of the project to the Chamber of Food Industries (CFI), shortening the duration of plans submitted to NFSA, and forging strong working relationships with mid-level NFSA staff have been effective strategies to ensure progress. There are substantial internal and external challenges that may impede the progress of the project and/or impact its sustainability. There remains a lack of cultural awareness around food safety in the broader Egyptian community that hampers both the inspection process and the general demand for safe products. Recommendations Continue to work with the entire system of stakeholders such as CFI to lay the groundwork for implementation of new regulations and policies. This was an effective strategy for TAIB to continue improving food safety in Egypt despite external challenges outside of their control. This strategy is also important to create a conducive environment for new regulations to be implemented; the CFI is a key stakeholder that can facilitate communication between NFSA and the food industry. viii Increase the engagement of private sector stakeholders in the regulation development process with a system that allows them to comment on drafts and communicate more directly with NFSA. Improving feedback processes between NFSA and the food industry would help align regulations and policies with the needs and incentives of private businesses, which in turn will make their implementation smoother and more effective. TAIB is already planning to develop standard operating procedures for new channels of communication, and this will be very helpful in facilitating businesses to register with NFSA and provide feedback. Improve inspection trainings by including more demonstrations of application of theory. Inspectors explained that it was difficult to bridge theory and practice, and that the virtual format of trainingstook away opportunities for individual guidance. NFSA staff requested more videotaped trainings so their trainers can continue using them as resources past the end of the project. Scale up consumer awareness activities around the importance of food safety, especially in agricultural regions where inspectors frequently work. Inspectors described confusion from business owners that did not understand why they were being asked to make improvements. This was especially an issue in more rural areas. TAIB should work with NFSA to better communicate with food businesses and raise awareness around food safety in the Egyptian community in general. Continue to seek out and build relationships with middle management at NFSA. TAIB has implemented a strategy of building relationships and establishing technical working groups with middle management to spread out responsibilities. This has been a successful approach and should be continued to avoid delays and to keep work flowing as much as possible. 1 1.Introduction 1.1. Background: Country Context Egypt has the largest population in the Arab world, 106,437,241 as of July 2021, and is a central power in the North Africa and Middle East region. It is also the third most populous country in Africa, behind Nigeria and Ethiopia.2 Despite multiple major political and economic crises since the turn of the century, from the global food price crisis of 2007/2008 to the Arab Spring political crisis in 2011, Egypt is growing economically and seeing an expansion of its middle class. Egypt’s Gross Domestic Product (GDP) per capita has risen from $7,744.7 (constant 2011 International $) in 2000 to $11,951.4 in 2020.3 Egypt has the advantage of a young and productive population; the median age in Egypt is 24.1 years, and the total labor force is 24.1 million people as of 2020 with an unemployment rate of 7.86% (as of 2019).4 Egypt’s population is also steadily urbanizing; in 2021 the urban population represented 42.9% of the total population and is predicted to grow at an annual rate of 1.9% between 2020 to 2025 (ibid). Both imports and exports of food are central to Egypt’s economy and the food and nutrition security of its citizens. Egypt relies on imports for over 50% of its food and agricultural product needs5 and is import dependent on certain staple goods like wheat. It is the world’s largest wheat importer with imports for the current 2020/21 marketing year estimated at 13.5 million tonnes.6 With a population of more than 100 million that is growing at a rate of 2% per year, Egypt is the US’ largest food export market in Africa and the Middle East.7 The top exporters of food products to Egypt in 2019 were Brazil, Thailand, the United States, and France.8 The United States is the second largest agricultural supplier to Egypt with 14% market share as of 2020; in 2020 the US exported $1.9 billion USD in agricultural products to Egypt.9 Looking ahead, strong economic growth and an expanding middle class will continue to increase demand for imported agricultural goods in Egypt. While Egypt imports much of its food, agriculture is also a key sector in the Egyptian economy. High levels of solar radiation, the Nile River, and counter-seasonality of production provide comparative advantages for Egypt’s agricultural sector. Egyptian producers export food goods primarily to other countries in the North Africa and Middle East region, the United States and Russia. US total imports of agricultural products from Egypt totaled $220 million in 2020, an increase of 22% from 2019.10 As of 2 CIA World Factbook, Egypt https://www.cia.gov/the-world-factbook/countries/egypt/ 3 FAO Statistics: https://www.fao.org/faostat/en/#home 4 CIA World Factbook, Egypt https://www.cia.gov/the-world-factbook/countries/egypt/ 5 USDA Foreign Agricultural Service, US Department of Agriculture, Egypt 2019 Export Highlights https://www.fas.usda.gov/egypt-2019-export-highlights 6 Food and Agriculture Organization of the United Nations Global Information and Early Warning System (GIEWS) https://www.fao.org/giews/countrybrief/country.jsp?code=EGY&lang=en 7 TAIB Policy Brief: Effect of Egypt’s Modernized Food Import Control Strategy on U.S. Exporters & Producers by Sydney Maanibe and Alex Samel, Land O’Lakes Venture37 (Link). 8 World Bank World Integrated Trade Solution Statistics (WITS), Food Products Imports and Exports, Egypt, Arab Rep. 2019 https://wits.worldbank.org/CountryProfile/en/Country/EGY/Year/2019/TradeFlow/Export/ 9 USDA Foreign Agricultural Service Egypt 2020 Export Highlights, https://www.fas.usda.gov/egypt-2020-export￾highlights 10 American Chamber of Commerce in Egypt Webinar: “US-Egypt-Partnership, and the outlook for 2021”, https://dailynewsegypt.com/2021/02/23/egypts-agricultural-exports-to-us-rise-22-to-220m-in-2020/ 2 2020, the agriculture sector contributed 11.3% of Egypt’s gross domestic product, and accounted for 28% of all jobs, and over 55% of employment in Upper Egypt.11 Main agricultural products produced include sugar cane, sugar beet, wheat, maize, tomatoes, rice, potatoes, oranges, onions, and milk.12 Egypt’s young, large, and growing population provides a large and growing market for convenience and quality differentiated retail foods sold through the country’s approximately 119,000 grocery retailers, including formal and informal businesses. Grocery retailers grew in number by 4.3% over the past five years (2015-2020). Traditional grocery retailers still represent 96.6% of total grocery retailers’ outlets in Egypt, with sales estimated at $25.1 billion, about 74% of total sector sales. (ibid) It was also anticipated at the inception of the project that the number of food-processing and manufacturing companies in Egypt is likely to grow beyond the current 7,000 as a result of the country’s growing population with a taste for processed foods, as well as increasing demand from the 339 million consumers in the Middle-East region that these companies supply.13 The COVID-19 pandemic was tremendously harmful for Egypt’s economy. A 2020 report from the International Food Policy Research Institute estimated pandemic-related losses in Egypt’s agri-food system at about $611 million USD (EGP 9.6 billion).14 However, most of these losses are in food services sectors, especially those related to tourism15 The government responded to COVID-19 by suspending all flights and shutting down hotels, restaurants, and cafes in March 2020 and the Egyptian Center for Economic Studies estimated losses from the tourism sector at one billion US dollars per month.16 However, the agriculture sector has been the most resilient component of Egypt’s food system during the pandemic with estimated increases in output at about $191 million USD (3 billion EGP). (ibid). This is due in large part to increased exports and some substitution of domestic production for imports. Despite this substitution, imports of staple foods remained steady during the pandemic; between July 2020 and June 2021, wheat imports grew by 1.57% from the 2019/2020 marketing year, and in the same time period corn imports grew by 4.76% from the 2019/2020 marketing year.17 In terms of exports, total agricultural exports actually increased during the pandemic; according to a report from the Central Plant Quarantine Administration, total agricultural exports between January and July 2021 totaled 4.0179 million tons, a 15% increase from the total volume exported during the same period of 2020.18 This is 11 US Agency for International Development Egypt Fact Sheet for Agriculture, https://www.usaid.gov/egypt/agriculture-and-food-security 12 CIA World Factbook, Egypt https://www.cia.gov/the-world-factbook/countries/egypt/ 13 USDA Foreign Agricultural Service (2017). GAIN Retail Foods 2017 Annual: https://gain.fas.usda.gov/Recent%20GAIN%20Publications/Retail%20Foods_Cairo_Egypt_12-21-2017.pdf 14 Breisinger, Clemens; Raouf, Mariam; Wiebelt, Manfred; Kamaly, Ahmed; and Karara, Mouchera. 2020. Impact of COVID-19 on the Egyptian economy: Economic sectors, jobs, and households. MENA Policy Note 6. Washington, DC: International Food Policy Research Institute (IFPRI). https://doi.org/10.2499/p15738coll2.133764 15 Salem IE, Elkhwesky Z, Ramkissoon H. A content analysis for government’s and hotels’ response to COVID-19 pandemic in Egypt. Tourism and Hospitality Research. April 2021. doi:10.1177/14673584211002614 16 CGTN (2020) Egyptian tourism bets on domestic tourists to reduce COVID-19 impacts. Available at: https://news.cgtn.com/news/2020-05-07/Egyptian-tourism-bets-on-domestic-tourists-to-reduce-COVID-19- impacts-Qi8FyE6TW8/index.html 17 Egypt: Despite the COVID-19 Pandemic, Egypt’s Wheat and Corn Imports Hold Steady – Grain and Feed Update 2020, https://www.fas.usda.gov/data/egypt-despite-covid-19-pandemic-egypt-s-wheat-and-corn-imports-hold￾steady-grain-and-feed 18 https://www.freshplaza.com/article/9336384/egypt-agri-exports-up-to-4-million-tons-from-january-to-june/ 3 due to pandemic impacts in competing countries that export to the Arab and East Asia regions, namely India and Pakistan.19 1.2. Food-Inspection Practices Prior to the TAIB Project Prior to the establishment of NFSA in 2017, oversight of food safety laws and regulations was spread across 17 agencies and 7 ministries in Egypt. The responsibilities of each agency were overlapping, leading to inefficiencies and inconsistencies in enforcement. Additionally, Egypt’s technical regulations around food safety were outdated, complex, and generally not based in current science.20 A mandatory 100% sampling rate at the border was a barrier to trade, especially for importers (ibid). While large food companies had the resources and expertise to comply with international best practices, small and informal businesses did not always implement standardized food safety systems. Some food industry businesses were visited multiple times by inspectors from different agencies and given contradictory findings, and some regulations were not enforced. The distribution of responsibilities across different agencies sometimes limited investment in modernizing practices, reduced incentives for private sector compliance and contributed to a poorly informed industry and consumer base. 1.3. Food Safety in Egypt Poor food safety can have a significant negative human and financial impact on an economy. This effect can be insidious and difficult to measure, as the health consequences of food safety pathogens can range from mild to life-threatening and are highly variable given the diversity of micro-organisms that are harmful to humans. However, numerous recent food safety studies in Egypt point to persistent problems with such pathogens, often caused by preventable hygiene issues. A 2021 study published in the Advances in Animal and Veterinary Sciences journal tested 100 samples of local and imported beef meat from El- Moneeb, El- Waraq and El- Basateen in Cairo and from Swiss canal, Aswan and Giza governorates and found that they were contaminated with several serovars of E. coli and Staph. aureus., as well as Salmonella in varying percentages.21 The authors concluded that this may be due to neglected sanitary measures adopted during slaughtering, dressing, evisceration, storage, transportation, and distribution of cattle carcasses. They recommended the application of Hazard Analysis and Critical Control Points (HACCP), a methodology that has been adopted by NFSA and one that the TAIB project has already trained some inspectors to use. Similarly, a 2021 study of shawarma from fast food restaurants in Dakahlia Governorate, Egypt found that histamine levels in most samples exceeded international acceptable limits, although they differed significantly based on the hygienic level of the restaurants.22 These food safety issues were not limited 19 Korean Institute for Economic Policy Paper: Nashwa Mostafa Ali Mohamed, Egypt’s Trade During COVID-19 Crisis: An Assessment of Responses and Implications, https://www.kiep.go.kr/galleryExtraDownload.es?bid=0026&list_no=9311&seq=7 20 TAIB Policy Brief: Effect of Egypt’s Modernized Food Import Control Strategy on U.S. Exporters & Producers by Sydney Maanibe and Alex Samel, Land O’Lakes Venture37 (Link). 21 Sayed HK, Tolba KS, Sobhy HM, Hekal SHA (2021). Assuring the safety of local and imported beef meat from different slaughterhouses in egypt. Adv. Anim. Vet. Sci. 9(9): 1472-1482. DOI | http://dx.doi.org/10.17582/journal.aavs/2021/9.9.1472.1482 22 Khalid Ibrahim Sallam, Samar Elaraby Mohammed Morgan, Mohamed Zakaria Sayed-Ahmed, Saad Saeed Alqahtani, Samir Mohammed Abd-Elghany, Health hazard from exposure to histamine produced in ready-to-eat 4 to meat; a 2020 study of fluid cream, Laban Rayeb, pasteurized milk, and ultra-high temperature milk produced in El Fayoum province, Egypt found high microbial counts of TBC, E.coli and S. aureus in the fluid cream that present a health hazard to consumers.23 The establishment of the NFSA and subsequent improvements in the enforcement of food safety regulations, as well as updated and improved regulations themselves, has the potential to address these problems and enhance public health outcomes for Egyptian consumers. 1.4. The Egypt TAIB Project Following nine years of collaboration among government ministries, parliamentarians, and the private sector, on January 2, 2017 the Parliament of Egypt passed law number 1/2017 establishing the National Food Safety Authority (NFSA). On February 19, 2019, the government issued and published NFSA’s Executive Regulations, promulgating the agency’s authority. The law and executive regulations empowers NFSA with the sole mandate to control and regulate food in Egypt, overseeing food safety from harvest to the Egyptian consumer. While transitioning the responsibilities from multiple governmental agencies presents short-term challenges with regard to the allocation of human and financial resources, in the long￾run, as the sole regulator of food within Egypt, NFSA should bring improved oversight, food safety, and increased trade to the country. NFSA’s main responsibilities include food law and regulation harmonization, establishment of industry-specific protocols for regulations, food-industry inspection and approvals, risk assessments, crisis management and consumer outreach. The United States Department of Agriculture (USDA) is supporting the development of NFSA through the Food for Progress Program: Transforming the Assessment and Inspection of Food Businesses in Egypt (Egypt TAIB), awarded in 2018 and implemented by Land O’ Lakes Venture37 (Venture37) and the International Food Protection Training Institute (IFPTI). The program was originally 5 years but has been extended to five-years and 9 months, with a budget of $8.5 million. Egypt TAIB’s implementation is organized into three categories of activity: 1) organizational capacity building for NFSA to allow it to grow sustainably and effectively to fulfill its mandate as the sole entity responsible for food safety in Egypt; 2) training programs for inspectors and other NFSA staff that include training of trainers and leverage existing curricula from the US Food and Drug Administration (FDA) and the Codex Alimentarius, a collection of food safety international standards, guidelines and codes of practice based in current science (from here on, CODEX); and 3) support for technical regulations and other regulatory measures to align them with international standards and improve clarity, transparency, and ease of implementation. Additionally, TAIB has worked with other stakeholders in the food safety space such as the Chamber of Food Industries to improve the implementation environment for new regulations, registration requirements and inspection protocols, and has produced numerous tools and publications (such as re-designing NFSA’s website, or publishing a brochure for consumers) to disseminate information about these changes and improve consumer shawarma widely consumed in Egypt, Journal of Food Composition and Analysis,Volume 97, 2021,103794, ISSN 0889-1575, https://doi.org/10.1016/j.jfca.2020.103794. 23 Zeinhom, Mohamed M. A. and Meshref, Arafa Meshref Soliman and Akl, Mohamed A. R. and Abdel-Rahman, Aya A. M., Bacteriological quality and safety of four fluid dairy products sold in El Fayoum governorate, Egypt, Future of Food: Journal on Food, Agriculture & Society,2020, doi:10.17170/kobra-202011192209 5 awareness of food safety. TAIB is expected to expanded domestic, Middle-East regional, and international trade. Specifically, Egypt TAIB implements the following activities: ACTIVITY 1. CAPACITY BUILDING: GOVERNMENT INSTITUTIONS. Egypt TAIB is building the capacity of NFSA to regulate and enforce food safety in Egypt. Egypt TAIB provides technical assistance to NFSA that includes developing inspection protocols, standard operting procedures (SOPs) to harmonize NFSA’s workflow, a code of ethics, job descriptions for needed positions, and trainings in administration, soft skills, and management for staff, as well as numerous other organizational capacity building activities in response to the evolving needs of the Authority. ACTIVITY 2. TRAINING: FOOD SAFETY AND MANAGEMENT. Egypt TAIB has built a competency-based curriculum framework tailored to NFSA and is using the framework to train a core cadre of senior-level inspectors. TAIB will also use this framework to support NFSA to transition towards a risk-based approach to inspection and support the development of food safety training courses to be offered at local universities to prepare individuals for entry into the food safety regulatory profession. ACTIVITY 3. CAPACITY BUILDING: PROMOTE IMPROVED POLICY AND REGULATORY FRAMEWORKS. Egypt TAIB supports NFSA to modernize and harmonize its food safety legislation and technical regulations with international standards and to promote compliance from the private sector. The central reform on this front has been transitioning NFSA away from 100% consignment to a risk-based sampling approach. TAIB has drafted many technical regulations that are in various stages of passage, including 12 that have been fully passed and are being implemented. 1.5. Theory of Change The expected result is an increase in domestic and international trade of Egyptian food products. The theory of change shown in Figure 1, developed in consultation with TAIB staff at baseline by the evaluation team, describes the pathways hypothesized to lead to increased exports/imports and domestic sales of safer food products. These are the changes expected on the supply side following the implementation of the project. As a trained food-protection regulatory workforce conducts regular inspections, thereby enforcing new, more-stringent food-safety standards according to an inspection policy and supported by technical regulations, we anticipate businesses to change their perceptions of the likelihood of enforcement of the food-safety regulations and fines for non-compliance. It may also lead to their overall perceptions of the importance of food safety in general. Thus, the affected businesses are expected to increase their awareness and knowledge of food-safety practices as well as compliance, changing the perceptions of profitability of raising the safety of their food products, in the short run. This in turn may stimulate investments in food safety practices and equipment in the medium or longer term. Consequently, these investments should improve the safety of food produced and/or supplied through the food value chain for domestic consumption and exports. Similarly, these investments should also improve the safety of food products supplied to the domestic market through imports. 6 At the same time, the TAIB project will directly improve consumer awareness of the importance of food safety through publications and improvements to NFSA’s website. Thus, demand for safer food products will be positively affected as a result of the project. Improved demand of consumers for food safety will further incentivize businesses to improve the food safety of their products. Eventually, the changes in both the supply and the demand sides will complement and reinforce each other to increase and sustain the sales of safer food products within and outside of Egypt. Figure 1: Theory of Change – TAIB Project 1.6. Overview of Evaluation Methods and Questions Venture37 has contracted NORC @ the University of Chicago to conduct the baseline, midterm evaluation (MTE), and final evaluation (FE) of the Egypt TAIB project. To evaluate the TAIB project, we employ a mixed-method approach with quasi-experimental and non-experimental designs. In particular, we are using two separate evaluation methods: (1) Macro-level evaluation: quasi-experimental interrupted time-series analysis within a potential outcomes framework (ITSPTO) using trade data to examine impacts of TAIB on the values of exports and imports for the food subsectors that are subject to NFSA oversight; (2) Performance evaluation: assessments based on document reviews, reviews of project monitoring data and assessment tools, key informant interviews, and focus group discussions. This evaluation will answer evaluation questions related to the relevance, impact, effectiveness, efficiency, and sustainability of the project. In Table 1, we list the key evaluation questions and the evaluation method we are using for each of them. 7 Table 1: Key Evaluation Questions Criteria Evaluation Questions Evaluation Method Relevance • How well aligned are the Egypt TAIB activities with the goals, objec￾tives and strategies of USDA, USG, and the Egyptian Government and the needs of NFSA and the broader Egyptian community? PE Impact • How has the project affected macro-level imports, exports of food products? Macro • How has TAIB support affected progress of laws, policies, and regulations?? • How has capacity building by Egypt TAIB affected the organizational capacity of NFSA? • How has training of NFSA staff affected their knowledge and ability to carry out their work? • How has the private sector been impacted by NFSA regulations? • What policies or actions have been effective in enforcing food safety standards to ensure public health and nutrition security? What role can various actors, including government, exporters, processors and the private sector play to enforce food safety standards to ensure public health and nutrition security? PE Effectiveness • To what extent are the program targets and outcomes likely to be achieved by project end, specifically expanded trade and improved capacity of NFSA? • What key successes should be replicated, or key improvements made to the implementation to maximize the results? • What internal and external factors have influenced the ability of the project to meet expected results and targets? • How has COVID-19 impacted project implementation? • Has the project improved distribution of funding across project activities to maximize impact? PE and Macro Efficiency • Were program activities carried out in a timely manner and with effective use of resources? • How well has the project been managed and MEL data used to make programmatic decisions? PE Sustainability • Which project benefits are likely to be sustained or not past the project lifespan and why? PE Notes: PE = Performance evaluation based on document reviews, key informant interviews, and focus-group discussions; Macro = Quasi-experimental approach using trade data. The next two chapters discuss the methodology and findings of each evaluation method. Chapter 2 explains the methodology for the macro-level trade analysis and presents the findings. Chapter 3 discusses the performance evaluation methodology and findings. And finally, Chapter 4 summarizes the conclusions from the macro-level trade analysis and the performance evaluation and provides some recommendations. Supplementary materials and technical details of the trade analysis is presented in the Annex. 8 2.Midline Analysis of Trade The current chapter presents empirical findings from our analysis of trade data using econometric approaches to measure changes in Egyptian trade. The chapter is divided into three sections. In the first section, we present findings from our analysis using a quasi-experimental impact evaluation to examine changes in the values of exports and imports that can be attributed to the TAIB project. In the second section, we present empirical strategies developed to examine intermediate indicators that suggest movements towards trade. In the final section, we discuss the implications of COVID-19 for the analysis of trade and econometric strategies we develop to mitigate biases in our empirical findings of impacts on exports and imports. 2.1. Impacts on Exports and Imports We present the empirical findings from our analysis of trade data on exports and imports in this section. First, we describe the two outcome indicators that we use to examine impacts of the TAIB project on exports and imports along with the data sources to measure these outcomes and for the analysis. We them provide details of our quasi-experimental evaluation design and the methods used for the econometric analysis of impacts, followed by the empirical findings. 2.1.1 Outcome Indicators and Data Sources The two main outcomes of interest are:24 • Value (in millions of 2010 constant US Dollars) of exports; and • Values (in millions of 2010 constant US Dollars) of imports. To measure the outcomes on targeted trade flows we create variables of total value for imports and exports at the two-digit Harmonized System (HS) code25 for the particular food-product subsectors subject by government mandate to NFSA regulatory oversight. We refer to this group of HS codes as the treatment group. For each treatment HS code, we also create a corresponding analogous observation with the counterfactual total value constructed from other importing and exporting subsectors not currently subjected to NSFA oversight. We refer to these two groups of constructed counterfactuals as the “synthetic” comparison groups, with one associated with treated imports and the other for treated exports.26 The data on trade flows come from the United Nations International Trade Statistics Database or UN Comtrade Database (https://comtrade.un.org/data/), the largest depository of international trade data, where data on the trade values and volumes (net weight) of various commodities (including food 24At baseline, we used trade volumes as additional outcomes. However, our exploration of the available data for the volume of trade indicated that the data is of poor quality and any analysis conducted with trade volume as the outcome indicator will not be reliable. As such, we recommended and agreed with USDA to use trade values as the only outcome indicator for the midline and endline analysis. 25 The Harmonized System is a global product classification system administrated by the World Customs Organization. Trade commodities are classified using HS codes to denote the type of product that is being imported or exported. The HS assigns specific six-digit codes for varying classifications and commodities. You can use the first 2 digits for the general type of commodity. 26 The reader should note that the synthetic comparison group contains the ultimate counterfactuals; it is constructed using comparison subsectors of untreated HS-code transactions. 9 products) are available on a quarterly and annual basis. This database, going back as far as 1962, is maintained by the United Nations Statistics Division (UNSD), to whom over 170 countries report annual international trade statistics data detailed by partner countries and commodities/service categories. The commodities/services are disaggregated into up to six levels of subcategories and classified using six￾digit codes. The UN Comtrade Database provides data on trade values in US dollars. As part of the baseline analysis, we extracted detailed annual data on exports and imports of food products for the five available years (2014-2018) of pre-treatment data for Egypt as a reporting country, disaggregated at the level of two-digit categories. For the MTE, we supplemented this data with two years of in-treatment (2019-2020) data. For the purpose of our analysis, we consider all food subcategories as treated unless specifically indicated by Venture37 staff as non-treated (e.g., live animals, live fish, straw and fodder, etc.). Like at baseline, we compute the trade values of various commodities in terms of 2010 constant values by adjusting for inflation. The consumer price index data for 2010 and the years 2014-2020 are used to adjust the trade values for all years in terms of the 2010 values. 2.1.2 Methodological Approach The quasi-experimental evaluation design requires comparing actual trade to its counterfactual, that is, what trade would have been without the TAIB intervention. The approach involves comparing trade values of specific product categories subject to NFSA inspection – the intervention group – to counterfactual values synthesized from trade categories not regulated by NFSA but containing products with characteristics of differing degrees of similarity to those in the intervention group – the comparison group. The latter synthesis is carried out using a statistical matching technique. The method essentially utilizes the pre-intervention values of the comparison group to construct a “synthetic” counterfactual for each intervention-group product category. For the midline impact evaluation we use the actual midline values of the comparison group of trade categories to compute the midline values of a synthetic counterfactual for each intervention-group product category. We then compare the midline value of these (now synthetic) counterfactuals to the actual observed values of the corresponding trade categories subject to NFSA regulation. Any difference is considered an impact of TAIB.27 This approach is called the interrupted time-series analysis within a potential outcomes (ITSPO) framework. We implement the ITSPO approach in two steps: (i) use statistical matching to create a counterfactual group of traded commodities not subject to NFSA that are statistically similar as a group to the group of commodity categories subject to NFSA oversight; and (ii) estimate an attribution equation to detect whether there is a statistical difference between the counterfactual and intervention groups after the start-up of NFSA – the impact. The first step was carried out during baseline and repeated using updated Comtrade data to take advantage of any revisions the UNSD had made over the intervening two years. The MTE involves the second step of estimating impacts two years after the beginning of the TAIB intervention. We provide details of each of the two steps below. However, first we define the treatment variable for the purpose of the trade analysis. 27 The goal of matching is to remove any observable baseline systematic differences between the two groups being matched so that any post-baseline observed systematic differences can be attributed to treatment, in this case TAIB. 10 Treatment Variable. We defined the treatment variable as equal to 1, if the import or an export was within a two-digit HS category associated with the products under the NFSA mandate, and 0 otherwise. Creating a counterfactual. We started the matching process by identifying a set of commodity categories that are not subject to NFSA oversight but have some proximate characteristics that suggest they might, when taken as a group, roughly track the commodity categories that are subject to NFSA oversight.28 This turned out to be quite problematic because it led to a tradeoff: it required splitting two-digit HS categories into their four-digit components and at that level of disaggregation the time series were rather sparse and noisy. Fortunately, a practical alternative was identified. It comprised selecting all two-digit HSs not targeted by the TAIB intervention as the input sectors for synthesizing comparison observations with entropy matching—a statistical matching approach of Hainmueller (2009). The treatment HS codes (as identified at baseline) and input comparison HS categories for the matching are listed in Annex II. We use statistical matching for each treatment-group HS code to create its “synthetic” comparison group.29 These groups, once constructed, should be both statistically similar (“balanced”) on characteristics of interest as well as not be subject to NFSA oversight (they must be “untreated”).30 We conduct robustness tests to check that results are not sensitive to the time-series period for the sample or the unit of matching (HS vs country).31 Likewise, we checked for statistical balance (to ensure good matches) – comparing the means of each year of the treatment HS to that of the synthetic comparison observation. (In fact, as per Hainmueller (2009), synthetic matching leads to near-perfect matches.). 32 The idea is that once NFSA begins its program of regulatory compliance, the synthesized comparison group will continue to mirror what the treated trade categories would have experienced had NFSA not existed. That way, in the attribution step we can compare that counterfactual experience to that which the treatment group actually exhibited. Any statistical differences would be “impacts” that the analysis would attribute to the existence of NFSA. Figure 2 conceptually illustrates the calculation of impact. 28 The proximate characteristics include similar degrees of processing, similarity in markets and use of market infrastructure, cold-chain requirements, input sourcing, energy intensiveness, value-added per kilo, as well as similar capital and labor requirements. 29 We conduct matching separately for each treated HS code so the sample contains 5 treatment observations against approximately 76 comparison HS codes (each also with five observations). Matching was limited to the top 20 trading partners to ensure data availability for the complete sample and evaluation period. Once matched there is one synthetic comparison observation for each treatment observation. To ensure the statistical matching process was not influenced by the volume of trade of a particular HS category, each two-digit HS trade-flow was first deflated by its mean value over the period, 2014-2018. (Those deflators were also applied to the HS’s 2019 and 2020 values.) 30 By statistically balanced we mean that the means and standard deviations of characteristics thought to influence outcomes are statistically same for both groups. 31 In addition to conducting the matching and subsequent analysis at the two-digit HS level we also conducted it at the country trading-partner level (i.e., for each direction of trade, matching a country’s treated transactions to its own comparison transactions across HS codes and years). While these results were not as precise, they generated similar estimates. This suggested that the results we present, below, for the HS-analysis are indeed robust to unit of matching. 32 Essentially, the robustness tests comprised examining whether the impacts detected depended on the sample used (e.g., with/without 2014 and with/without the outlier HS codes). 11 Figure 2: Illustration of how the ITSPO approach can capture the TAIB-intervention effects Note: The upward-pointing kink (“curvature”) in the outcome trajectory between t1 and t3 is simply illustrative. In reality, we may find it to be upward, downward, or nonexistent (linear), each bearing its own policy implications Likewise, the dips and their relative size are purely illustrative. Their actual sizes will be econometrically estimated at endline. Detecting an impact. Having established the comparison group in the first step through statistical matching, the second step in the ITSPO framework is to estimate an attribution equation to detect whether there is a statistical difference between the counterfactual and intervention groups after the start-up of NFSA, or in other words the impact. We estimate the impacts of TAIB in two different ways: • Unconditional impact estimates (without control variables). The unconditional impact estimates are computed as the difference between the counterfactual (the synthetic comparison) outcomes and the actual treated-group outcomes in 2019 and in 2020.33 We refer to these as estimates as unconditional since the calculation does not involve any control (conditioning) for other post-baseline influences that might have asymmetrically affected the treated and the comparison products. Since in the first step of the ITSPO framework, the counterfactual group was constructed by removing any systematic differences between it and the treatment group, it represents an estimate of what would have happened without TAIB. This implies that the only difference between the treated group and the synthetic comparison group is the fact that the former is exposed to TAIB and the NFSA mandate and that the latter is not. Thus, where the actual trade value is larger than the counterfactual, we can potentially attribute that difference as an “impact” of TAIB. 33 The astute reader may wonder why no difference-in-differences estimator was calculated. This is because at baseline each treatment category and its synthetic comparison (counterfactual) had essentially the same value, so no double-differencing was required. = Treated products = Possible treated trajec￾tory under Covid-19 = Comparison products = Possible counterfactual trajectory under Covid-19 Gap at t = Δ𝑋𝑋𝑡𝑡,𝑠𝑠 T − Δ𝑋𝑋𝑡𝑡,𝑠𝑠 C Δ𝑋𝑋𝑡𝑡 T, change in treated products at time t 3 Δ𝑋𝑋𝑡𝑡,𝑠𝑠 C , change in synthetic comparison group at time t3 without Covid￾19 End of evalua￾tion period, t 3 𝑋𝑋𝑡𝑡,𝑠𝑠 C 𝑋𝑋𝑡𝑡,𝑠𝑠 T Outcome Start of TAIB, t1 Start of histori- Midline, t2 cal period, t 0 12 • Conditional impact estimate (with control variables).34 A more sophisticated approach to estimating impacts would be to employ an econometric approach to control for various sources that might influence intertemporal variation in the trade values (export or import) specific to a sector or at the macro level.35 At the macro level, examples of such influencing factors can be exchange-rate, and interest-rate fluctuations, and political and policy shocks (e.g., strikes, changes in government). The maintained assumption under the unconditional estimates, above, is that these would affect all sectors – and, therefore, treatment and comparison groups – roughly equally. In that sense, the conditional impact estimates will not be different from the unconditional impact estimates. However, the precision of the estimates would likely improve as a results of taking the influence of these intertemporal variations explicitly.36 If, however, the maintained assumption is unsound (Covid-19 being an example) then the conditional (econometric) approach would likely remove the ensuing bias. For this reason – and as documented in Annex III – we estimated a series of econometric models with alternative specifications. Unfortunately, we were not able to utilize these economic models yet for two reasons . First, the main potential source of asymmetric influence – Covid-19 – only occurred in 2020, the last year of the midline period, giving us just one observation per HS category with which to statistically disentangle TAIB from Covid-19 contributions. Second, Covid-19 greatly slowed down statistical reporting and access to data for 2020 (and for 2019, as well), hobbling efforts to draw on other data sources. As a result, we were not able to generate estimates of sufficient precision to warrant reporting, so the impact findings presented, below, are based on the unconditional approach.37 2.1.3 Findings: Prediction Accuracy of the Counterfactual As in the baseline stage, once the matching was done using the revised Comtrade data, we tested the prediction accuracy of our approach by measuring the in-sample prediction error and the quality of the predicted counterfactual. Testing in-sample prediction error. Conceptually, if the synthetically matched comparison group predicted the actual trade values observed in the pre-intervention sample period (2014-2018) with minimal errors, then we could be more confident that they would also predict the no-intervention trade flows during the post-intervention period that we can then use to compare the actual treatment-period trade values to estimate impacts. We compute the mean in-sample absolute prediction errors by two￾digit trade category over time. These statistics allowed us to determine whether certain trade categories are responsible for the significant prediction error, should we observe any. 34 See Annex III for the technical details. 35 Since the construction of the counterfactual group removed sources of systematic differences between it and the treatment group over the sample period (2014-2018 in our case), asymmetric influences on the two groups from these macro factors would come from sources that either did not vary during the sample period, only emerged post-baseline (e.g., Covid-19), or were not taken into account. 36 In other words, the standard errors of the impact estimates would be smaller. 37 Fortunately, by endline we will both have two additional years of data as well as access to some of the additional covariates sought to overcome the aforementioned impediments. 13 The results of these tests were uniformly excellent with very low forecast errors over the sample period for exports (see Figure 3) and imports (see Figure 4) by HS category. Let us describe how to interpret these graphics as we examine their findings. Figure 3 and Figure 4 present the within-sample forecast errors for exports and imports, respectively, for each of the 18 treatment HS trade categories.38 These graphs indicate the degree to which matching reproduces the actual historical trade flows of the baseline period. Each value displayed represents the forecast error for the respective year. The forecast error is calculated as: Forecast error = (Synthetic comparison trade value – Actual trade value) / (Actual trade value) We see that the synthetic counterfactuals are generally within one percent of the associated actual treatment value for each HS code, both for exports (Figure 3) and imports (Figure 4). This reinforces our confidence in the analysis approach. Figure 3: Within-sample (2014-2018) forecast error for exports, by HS code Note: The forecast error is the percent-difference between the actual and the synthetic comparison created through the statistical matching process, i.e., (Synthetic comparison – Actual)/Actual. Matching was conducted on the trade flow deflated by the within-HS, across-year mean. 38While the graphs appear to show 22 categories, in fact several intervening categories are missing because they are not subject to NSFA regulation. Also note that for two-digit codes 02 to 09 the leading zero is dropped from the value’s label. 14 Figure 4: Within-sample (2014-2018) forecast error for imports, by HS code Note: The forecast error is the percent-difference between the actual and the synthetic comparison created through the statistical matching process, i.e., (Synthetic comparison – Actual)/Actual. Matching was conducted on the trade flow deflated by the within-HS, across-year mean. Testing the quality of the predicted counterfactual. Another useful test of the quality of the baseline synthetic-control counterfactuals is to consider how good a one-year-out forecast would be over the pre-treatment period. We therefore check how good the approaches would be to forecast 2018 had we used the approaches in 2017 using just our data for 2014-2017. Toward this end, we re-compute the synthetic weights by repeating the matching process using just four years (2014 to 2017) of data and then used the new weights to predict the import values for 2018. As shown in Figure 5 for exports and Figure 6 for imports, in both cases the results showed that the matching approaches generated robust counterfactuals of one-year-out trade. No forecast error is greater than two-fifths of one percent. 15 Figure 5: Forecast error using 2014-17 data to predict 2018 NFSA-regulated exports (by HS code) Source: Authors’ calculations based on UN Comtrade Database. Notes: Forecast error is the percentage difference between the actual total of partner trade with Egypt for 2018 and HS category regulated by the NFSA and the associated values constructed through entropy matching restricted to 2014-17 data on comparison HS categories exempt from NFSA oversight. It is measured as the mean in-sample forecast error. Figure 6: Forecast error using 2014-17 data to predict 2018 NFSA-regulated imports (by HS code) Source: Authors’ calculations based on UN Comtrade Database. Notes: Forecast error is the percentage difference between the actual total of partner trade with Egypt for 2018 and HS category regulated by the NFSA and the associated values constructed through entropy matching restricted to 2014-17 data on comparison HS categories exempt from NFSA oversight. It is measured as the mean in-sample forecast error. 16 2.1.4 Findings: Impact of TAIB on Exports and Imports The impacts of TAIB for 2019 and 2020 by HS code, as estimated using the ITSPO approach, are presented in Figure 7 for exports and Figure 8 for imports. As described above, a negative value indicates a positive impact of TAIB on trade and a positive value indicates a negative impact of TAIB on trade. We indicate estimates for 2019 by blue circles and for 2020 by red diamonds. The mostly negative values of the circles in Figure 7 and Figure 8 suggest that in 2019 most HS categories experienced a greater amount of trade than their matched historical performance would suggest – i.e., an indication of impacts on specific HS-level exports (Figure 7) and imports (Figure 8). On the other hand, the opposite story appears to be the case for 2020 as illustrated by the diamond-shaped values in the figures: actual exports and imports across the HS categories are all lower than what their counterfactual values (i.e., matched historical performance) would suggest. In other words, at first blush impacts would appear negative (making trade worse). However, this may be because, unlike 2019, the presumed positive impact of TAIB was overwhelmed by the presumed negative effects of COVID-19. The apparent asymmetric effect of Covid-19 on NSFA-regulated products and the other products used to construct the synthetic comparison may be due to the matching only picking up systematic sources of variation operating during the pre-treatment sample period and extreme adverse health influences would not have been one of those. At endline when more midline data become available we will be able to utilize econometric analysis to disentangle the TAIB effect from the Covid-19 effect).39 Figure 7: Unconditional impact estimates for 2019 and 2020 on exports, by HS code Note: The forecast error is the percent-difference between the actual and the synthetic comparison created through the statistical matching process, i.e., (Synthetic comparison – Actual)/Actual. Hence, a negative (positive) value implies a positive (negative) impact. Matching was conducted on the trade flow deflated by the within-HS, across-year mean. 39 This more sophisticated approach utilizes a model specification and conditioning covariates based on an interrupted time series analysis within a potential outcomes (ITSPO) framework and estimates the impact drawing on a sample for the combined pre-intervention and intervention period, 2014-2020. The details of these models are described in Annex III. 17 Figure 8: Unconditional impact estimates for 2019 and 2020 on imports, by HS code Note: The forecast error is the percent-difference between the actual and the synthetic comparison created through the statistical matching process, i.e., (Synthetic comparison – Actual)/Actual. Hence, a negative (positive) value implies a positive (negative) impact. Matching was conducted on the trade flow deflated by the within-HS, across-year mean. While the above two graphs present the impacts in terms of percentage change, they can also be presented in US$ terms, as shown in Figure 9 for exports and Figure 10 for imports. Figure 9: Comparison of actual-to-(unconditional)-counterfactual exports, 2014-2020, by HS code (in 2010 constant US$) Notes: For 2014 through 2018, the model’s counterfactuals (synthetic comparisons) are so precise that they appear in the graph as single points (though in color the discriminating reader may detect both of them). Values are for the Egypt’s top-twenty trade partners (see Footnote 29) Also, for 2020, we omit the counterfactual from the graph; it is roughly 3.0 x e+10 so scaling the vertical axis to include this point would have obscured variation in the values for the other years. 18 Figure 10: Comparison of actual-to-(unconditional)-counterfactual imports, 2014-2020, by HS code (in 2010 constant US$) Notes: For 2014 through 2018, the model’s counterfactuals (synthetic comparisons) are so precise that they appear in the graph as single points (though in color the discriminating reader may detect both of them). Values are for the Egypt’s top-twenty trade partners (see Footnote 29). We summarize the impact findings using the unconditional impact estimates under the ITSPO framework, below, and in Table 2: • TAIB has had an impact on exports in 2019. When compared to the corresponding total value of counterfactual exports in 2019, we find that the actual total value of Egypt’s NFSA-regulated exports in 2019 were almost 12-percent higher than what they would have been had TAIB not been implemented. In other words, TAIB appears to have had an attributable positive impact on NFSA-regulated exports in 2019. This effect translates into a US$ increase in exports of almost 293 million US$ (constant 2010).40 • TAIB’s has had an impact on exports in 2020. When compared to the corresponding total value of counterfactual level of exports in 2020, we find that the actual total value of Egypt’s NFSA￾regulated exports in 2020 were almost 8-percent higher than what they would have been had TAIB not been implemented and the COVID-19 pandemic not occurred. In other words, until we conduct the procedures to mitigate COVID-19 bias (see below) we cannot say whether TAIB has had an attributable positive impact on NFSA-regulated exports in 2020. • TAIB has had an impact on imports in 2019. When compared to the counterfactual level of imports in 2019, we find that the actual total value of Egypt’s NFSA-regulated imports in 2019 were ten-percent higher than what they would have been had TAIB not been implemented. In other words, TAIB appears to have had an attributable positive impact on NFSA-regulated imports in 2019. This effect translates into a US$ increase in imports of almost one billion US$ (constant 2010). 40 For the top-twenty trading partners. 19 • TAIB’s impact on imports in 2020 is uncertain. Comparing the corresponding total value of counterfactual imports in 2020 to that of the actual total value of Egypt’s NFSA-regulated imports in 2020 we were confronted with a rather large, suspect result.41 The team worked extremely hard to identify the source of the (presumed) error, but so far to no avail. As such, we have reached out to Comtrade, the source of the trade data for some clarifications. At the same time, we have been digging into each step of the complex entropy-matching￾assisted forecast model and believe the issue may be that some of the comparison-HS-code imports experienced infrequent trade flows over the analysis period, resulting in significant volatility. In the meantime, we have not reported the 2020 imputed import impact.42 Regardless, until we conduct the procedures to mitigate COVID-19 bias (see below) we will not be able say whether TAIB has had an attributable positive impact on NFSA-regulated imports in 2020. Table 2: Total impact of TAIB on Egypt's exports and imports, 2014-2020 (20 countries) Total impact (2010 constant USD)(a) Exports Imports Total(d) Year Value (mln) Share of total(b) Value (mln) Share of total(b) Value (Bln) Share of total trade 2019 293 11.7% 960 10.0% 1.25 10.4% 2020 185 7.8% [TBD](c) [TBD](c) [TBD](c) [TBD](c) Source: Authors’ calculations based on Comtrade data downloaded June 2021. Notes: (a) For Egypt’s top-twenty trading partners for each direction of trade. (b) Total impact summed over all impacted treatment categories divided by total value of the respective direction of trade for those categories, then multiplied by 100. (c) Under revision. These values appeared suspiciously large, so we are reaching out to Comtrade to determine if there are reporting errors as well as examining the role of a small number of infrequent comparison-HS trade flows. (d) Exports plus imports. In conclusion, while it is relatively clear that in 2019 TAIB had an attributable impact of US$ 1.25 billion on total trade of NFSA-regulated products (10.4% higher than what they would have been had TAIB not been implemented), the figures for 2020 must remain uncertain until endline when more data become available, permitting the econometric separation of Covid-19 and TAIB effects (and until the current temporarily unresolved technical difficulties are addressed). 41 That said, it is worth noting that The Federal Reserve Bank of St. Louis estimated the COVID-19 impact for these countries as the difference between the actual gross domestic product growth rate in 2020 and the IMF forecast for it made in October 2019. Juan M Sanchez (2021) “COVID-19’s Economic Impact around the World”, Web Bulletin, Federal Reserve Bank of St. Louis, August 11. Since for low-income countries, GDP growth in 2020 was whopping 5.2-percentage-points lower than expected (Sources: IMF World Economic Outlook Reports (April 2021 and October 2019), Penn World Table/version 10.0, and author’s calculations), this suggests that the impact of Covid-19 for Egypt in 2020 was approximately 5 percent of GDP. 42 The investigation has been made all the more challenging since the impact on imports depends on the counterfactual forecast for 2020 and that requires not just modeling the effect on imports had TAIB not been implemented but also modeling the effect on imports had the COVID-19 pandemic not occurred. 20 2.2. Movement toward increasing trade Since often large impacts on trade can be a lagging variable, the evaluation team planned to also measure some indicators of movement towards increasing trade in the TAIB evaluation. At the outset of the midline evaluation, NORC developed a formal econometric strategy to analyze several indicators that would suggest movement towards trade by examining movement in intermediate outcomes that the theory of change postulates must first occur. For example, a strong increase in trade of agricultural products due to TAIB would require that firms either increased their production while maintaining constant their export share, increased their export share without increasing production, or a combination of the two.43 Changes in production might also be accompanied with an increase in investment. The econometric strategy involves examining two intermediate outcomes—employment and number of firms exporting (importing)—to analyze movements towards trade. We expect an increase in employment in the sectors that should benefit from the reforms supported by TAIB, at least if such movement is the result of an increase in agricultural production relative to employment levels in other sectors competing for the same labor. We would also expect sectors that benefit from the reforms supported by TAIB to manifest movement toward trade by experiencing an increase in the number of firms involved in exporting and importing relative to what other sectors experience. We provide an overview of the econometric strategy below; technical details are provided in Annex III. We developed two analytic strategies. The first involves comparing the percentage change in employ￾ment (or number of firms) between the twelve months of 2019 and those of July 2020 through June 2021 (aka 2021*) in TAIB-targeted sectors to those of non-TAIB-targeted sectors, statistically accounting for the extent of the fall in employment (or the fall in number of firms) due to COVID-19 from 2019 to 2020 and changes in an international price index for each traded sector from 2019 to 2021*. The hypothesis is that the percentage change in employment (or number of firms) in the TAIB-targeted sectors are higher. The second strategy is similar to the matching strategy employed for the trade analysis explained, above, to examine the extent to which employment (or number of firms) in agricultural subsectors exhibits a greater increase (or smaller decrease) than in other sectors. This entails using non-agricultural sectors as the source of comparison sectors for the synthetic matching. Once matched, we would generate forecasts for 2020 and 2021* based on pre-intervention data. The difference between the 2020 forecast and actual employment in 2020 provides an estimate of the extent to which employment (or number of firms) fell due to COVID-19 in 2020. Likewise, the respective 2021* forecasts provide estimates of the extent to which sector employment (or number of firms) in 2021* recovered from COVID-19, so the differences between them and actual employment (or number of firms) in 2021* provides an estimate of the increase in employment (or number of firms) due to TAIB. Unfortunately, these analyses all require data for 2020 and 2021 on sector-level domestic labor employment, value of production, and number of firms. Based on our exhaustive search of publicly available sources, these data for are currently not yet available. Although we observed some impacts of TAIB on trade at midline, NORC proposes to conduct these analyses at endline if strong evidence of impacts on exports and imports is not observed (and data from public sources become available), since they would still serve to document movement toward trade, whether at midline or endline. We provide details about the data availability in Annex III. 43Of course, if the world price of Egypt’s agricultural exports increases then the value of Egypt’s exports could go up without any change in the quantity or share of production exported. 21 2.3. Mitigating COVID-19 bias from trade-impact estimates One of the biggest challenges facing the present impact evaluation is that the onset of the pandemic took place between the start-up of TAIB and (for now) the midline. This requires that we statistically dis￾entangle the COVID-19 effects on Egypt’s trade flows from those of TAIB, both for inferring movement toward increased trade as well as for detecting actual impacts on trade. In the case of the trade flows, themselves, we base the impact evaluation on the ITPSO framework (detailed above) using data that we have first synthetically matched. This approach entails two critical assumptions. The goal of the synthetic matching approach is to create a comparison group at baseline that is statistically similar to the treatment group in the observed characteristics employed in the matching process. Hence, our first assumption is that the right characteristics (covariates) were included in the matching so that no systematic influences were ignored (i.e., assuming that no “unobservables” were omitted). This ensure that the only systematic difference in these two groups is whether their trade categories will be subject to the NFSA mandate or not. Thus, any systematic differences in the trade values (imports or exports) between the two groups after the start of the intervention should be solely attributable to the TAIB intervention. Furthermore, the ITSPO method makes the assumption that the effects of external shocks such as COVID-19 are expected to be system wide with symmetric impacts on the treatment and synthetic comparison groups, on average. In short, these two assumptions imply that COVID-19 effects should drop out of any doubly differenced time-series analysis. That said, it is possible that the COVID-19 pandemic may have had differential effects on the treatment and the synthetic comparison groups’ average trade values. This could occur if the effect of COVID-19 operates through channels (variables) not captured by those used for matching but effect a firm’s ability to trade. This is quite possible since such influences may not have contributed to variation in the short historical time-series used in the matching. As part of our midline evaluation we therefore developed – and intended to carry out – a series of analytic tests and adjustments to purge, if necessary, COVID-19 impacts from TAIB impacts, which were essentially perfectly correlated. The first analytical test is to use a sector’s percentage change in a given direction of trade over the period 2020-2021 as a proxy indicator of COVID-19’s effect. This indicator can then be included explicitly as a control variable in an attribution equation. The other analytic strategy is to use a two-sided means￾test to examine if the pre-COVID-19 average labor intensity of production across the treatment sectors and synthetic comparison sectors is statistically equal. If the test reveals that they are not, then we would include the differential labor-intensity along with the sector’s percentage change in trade as control variables in the attribution equation. Unfortunately, these analyses require data on sector-level domestic labor employment for 2020 and 2021, which, based on our exhaustive search of publicly available sources, are currently not yet available. It also requires data for sectors’ percentage change in trade for 2021, a year still in progress so data is not yet available. Practically speaking, then, these analyses could only be carried out in late-2021 or first-quarter 2022 – too late for the present midline report. Instead, given that the empirical approach have now been developed, NORC proposes to conduct these analyses at endline since they would still serve to increase the credibility of the estimated TAIB impacts by mitigating potential bias from COVID-19. We provide more details about the empirical strategy and data availability in Annex III. 22 3.Performance Evaluation This chapter presents a wholistic account of TAIB project’s progress on its activities to date, assesses its impacts, and how it has performed against its targets. We also examine the success factors to TAIB’s performance, key challenges that it faced, and the extent to which the TAIB activities are sustainable. We begin by describing the methods for the performance evaluation, including the data sources used, data collection methods, and analytic approach. We then present the findings of the performance evaluation in detail. 3.1. Methodology During the midline, we conducted a non-experimental performance evaluation using document reviews, key informant interviews (KIIs) and focus-group discussions (FGDs). The goal of the performance evaluation was to broadly answer the research questions, listed in Table 1, related to the relevance, impact, effectiveness, efficiency, and sustainability of the project as a whole. Below we describe the data sources used in the performance evaluation and the data collection and analysis process. 3.1.1 Data Sources for the Performance Evaluation Table 3 summarizes qualitative data collection activities and includes information on the number of interviews conducted for each data collection activity and respondent type. At midline, we conducted 16 interviews total, including 2 focus groups with NFSA inspectors. Table 3: Summary of Qualitative Data Collection Activities Data Collection Activity Respondent Type Number of Interviews KIIs NFSA Staff 3 TAIB staff 5 USDA 1 Business Chambers 1 Food Businesses 2 3rd Party Certification Companies 2 FGDs NFSA Inspectors 2 groups of 8-10 inspectors This performance evaluation also used TAIB program documents to assess the progress of the project: • TAIB Semi-Annual Reports covering the periods: April 1-September 30, 2019; October 1, 2019 to March 30, 2020; April 1-September 30, 2020; and October 1, 2020-March 31, 2021. • The NFSA Organizational Capacity Scorecards (Organizational Assessment Tool) for Years 1, 2 and 3. • NFSA regulatory documents, including the published regulations of Decision No. 3 of 2020, Decision No. 6 of 2020, Decision No. 7 of 2020, and Decision No. 13 of 2020 as well as other 23 draft regulations such as the Hotel Licensing Technical Regulation and the Free Zones Technical Regulation. • Brochures, industry guides, and other public facing documents such as: “FAQ for Food Import Licensing”; “NFSA Manual for Industry on COVID-19”; “Consumer Food Safety Brochure for Web”; and “Food Handler’s Guide in Packhouse”. • Articles about TAIB activities, including “Supporting Science, Increasing Trade: Science-based Approaches Remove Trade Barriers for Beef Liver Exports to Egypt”. • NFSA administrative and internal policy documents such as their Code of Conduct, Food Complaints Handling Procedure, Technical Competencies Manual, and Food Risk List. • The report, “NFSA’s Modernization of Egypt’s Food Import Control System”. • Monitoring data on administrative policies and procedures, trainings, and policy, law and regulation progress. • TAIB Evaluation Plan. • TAIB 5-year Work Plan. • NFSA Needs Assessment and Narrative. • TAIB Plan of Operation and Activities. 3.1.2 Data Collection and Analysis NORC collected primary qualitative data to supplement the documents provided by TAIB. The primary data consisted of 14 key informant interviews with TAIB staff, NFSA staff, a representative from the Chamber of Food Industries, staff from food businesses and 3rd party certification companies, and a representative from USDA. Additionally, two focus group discussions were conducted with NFSA inspectors to gather their perspectives. These midline KIIs and FGDs provide information on the progress of implementation, challenges and successes since baseline, and recommendations for the project moving forward. The NORC evaluation team contracted Dr. Zeinab Khadr and Mr. Mohamed Hassan Hosni, experienced social scientists from American University in Cairo, to coordinate and conduct the KIIs and FGDs. Interviews were conducted between August 16th and September 20th, 2021 and were conducted remotely given continued concerns around COVID-19; in addition, respondents across categories were acclimated to working remotely, including inspectors who had previously participated in virtual trainings. Interviews with TAIB staff averaged about one hour, ranging from 40 minutes to 85 minutes. All other interviews took between 30 and 60 minutes, with the exception of one KII with NFSA staff that went for one and half hours, and the focus groups with inspectors which also took approximately one and half hours. Interview protocols can be found in Annex IV and our COVID-19 Safety Protocol for the data collection field work can be found in Annex V. All interviews were recorded. Interviews were conducted in Arabic or English, depending on the preference of the respondent, and the team at American University in Cairo transcribed and translated them. NORC’s qualitative specialist then coded all transcripts using a deductive coding scheme in Dedoose qualitative analysis software to produce relevant excerpts by theme and evaluation question. These excerpts were analyzed to find congruencies and divergences among respondents and to provide information to answer the evaluation questions. 24 The categories of respondents were expanded from baseline to include NFSA inspectors, NFSA staff, food businesses, 3rd party certification companies, and USDA in addition to the previous categories of TAIB staff and Chamber of Food Industries representatives. It was not possible to interview representatives of other Ministries that work on food safety in Egypt. This qualitative data was collected for the primary purpose of validating and contextualizing the information on TAIB’s progress from the program documentation listed above. As such, the interview questions and sampling design were developed in order to maximally complement the analysis of progress especially from the Organizational Capacity Scorecards, monitoring data, and Semi-Annual reports. 3.2. Performance Evaluation Findings 3.2.1 Alignment with Key Stakeholders We begin our presentation with a detailed look on the alignment of the TAIB activities with key stakeholders of the project—the US Government (USG) and the USDA, the Egyptian Government and NFSA, and the broader Egyptian community. Alignment with USDA and USG As a food for progress program from the USDA, TAIB supports USDA’s overarching objective of increasing trade both for the target countries generally and between the US and those countries. Through building the capacity of NFSA to ensure safer imports and exports and harmonizing food safety policies and regulations with international standards, TAIB is designed to facilitate trade and also promote policies that are in line with USG regulatory frameworks. A key example of aligned priorities between USDA/USG and TAIB is the reform of Egypt’s policy on Ractopamine. Before the establishment of NFSA and the implementation of TAIB, Egypt had aligned its regulatory framework more closely with the European Union and adopted their policy of zero-tolerance for Ractopamine (a veterinary drug used for beef). This zero-tolerance policy was not in line with the approved CODEX standard, or the WTO’s guidance that requires a deviation from an approved CODEX standard to be set based on consumption risk assessments. The zero-tolerance policy towards Ractopamine had significantly impacted US exports, reducing US meat exports to Egypt from peak to trough by almost $90 million since Egypt began enforcing the policy in 2018. 44 With TAIB’s support, NFSA conducted a consumption study to establish minimum acceptable levels of Ractopamine for consumption and implement it as a technical regulation. Since Egypt is the world’s largest importer of beef liver, it represents a highly important market for US producers, and this change was beneficial to US producers. USDA reports that rejections of consignments containing imported beef liver is down 90% since the 44 Source: USDA/Venture37: Supporting Science, Increasing Trade: Science-based Approaches Remove Trade Barriers for Beef Liver Exports to Egypt “TAIB was able to work directly with NFSA to help establish standards and regulations that are consistent with those that are based on science, number one. And they are consistent with international norms. In this case, the Codex Alimentarius, which is really the standard for food safety under the World Trade Organization, one of the pillars, if you will, […] that was really the direct match to what we are trying to do, as USDA to facilitate trade.” (USDA, KII) 25 adoption of Egypt’s new veterinary drug residue technical regulation.45 In that regard, TAIB’s activities are well aligned with the trade objectives of USDA and the USG, which was also echoed by the representative from USDA. Alignment with NFSA and the Egyptian Government The Egyptian Government and NFSA share the goals of streamlining food safety oversight processes to be more efficient as well as aligning regulations with international standards to increase trade, both domestically and with other countries. Respondents from TAIB highlighted that increased trade in food products is a priority for the President, Prime Minister, and Ministry of Trade in Egypt: specifically, reducing the cost of trade, encouraging food exports, generating foreign currencies, and increasing incomes for Egyptian farmers, pack houses, and processing firms. In addition, these stakeholders share the objective of ensuring food safety and nutrition security for Egyptian citizens. Making food safety oversight processes more streamlined and efficient necessitates reducing overlap between the duties of NFSA and other ministries. The TAIB project supports reducing this overlap in two ways: 1) by building the capacity of NFSA so that it can take on more oversight until it eventually fulfills its mandate of being the sole entity responsible for food safety in Egypt, and 2) supporting NFSA as it navigates relationships with ministries, particularly through the drafting of Memorandums of Understanding. TAIB has supported the drafting of Memorandum of Understandings between NFSA and the Customs Authority under the Ministry of Finance, the Cairo Chamber of Commerce, Egyptian Drug Authority, General Organization for Veterinary Services, Central Administration for Plant Quarantine and the Free Zones Authority. Additionally, TAIB has supported NFSA to develop the food import control strategy. This strategy moves away from 100% sampling, instead sets sampling rates based on assessments of risk, and is designed to reduce the time and cost of trade. Alignment with the Broader Egyptian Community TAIB’s activities are also well aligned with the needs of the broader Egyptian community. There are high prevalence rates of different enteric virus infections among the Egyptian population that are linked to food safety such as Hepatitis A and E viruses, human rotaviruses, human noroviruses, human astroviruses, and human adenovirus.46 The activities of the TAIB project will improve the enforcement of food safety regulations and increase consumer and industry awareness, which will in turn lead to less foodborne illnesses and better public health outcomes. During one KII, NFSA staff also mentioned that there is a “new national direction towards good nutrition for adolescents and young people”. TAIB project activities have included communications with the Egyptian public, such as preparing a newsletter on consumer protection and building NFSA’s website. TAIB also worked with NFSA to develop a brochure for consumers educating them on safe practices for purchasing and storing food. This brochure was disseminated electronically and TAIB provided 9,000 printed copies to NFSA for further dissemination. Several respondents mentioned that Egyptian consumers are becoming increasingly aware of the importance of food safety; one respondent from a 3rd party certification company stated, “The Egyptian market is currently experiencing major concern with this subject. It is not price only, no 45 USDA GAIN Report: Adopting A Risk Based Tolerance for Ractopamine in Beef Reduces Rejections by 90 Percent, July 28, 2021 (link). 46 Aboubakr H, Goyal S. Involvement of Egyptian Foods in Foodborne Viral Illnesses: The Burden on Public Health and Related Environmental Risk Factors: An Overview. Food Environ Virol. 2019 Dec;11(4):315-339. doi: 10.1007/s12560-019-09406-z. Epub 2019 Sep 27. PMID: 31560123. 26 there are some groups of the population who seek food safety.” In addition, several respondents brought up the issue of tourism and how food safety issues can prevent tourists from coming to Egypt, thus making food safety an important topic in sectors outside of the food system. 3.2.2 Progress and Impacts of TAIB activities and Effectiveness in Reaching Targets In this section, we discuss progress of TAIB activities for each of the three activities: Activity 1: Capacity Building: Government Institutions; Activity 2: Training: Food Safety and Management; and Activity 3: Capacity Building: Promote Improved Policy and Regulatory Frameworks. We examine the project’s effectiveness in reaching targets by reviewing TAIB project documents, including progress on various indicator measures set by the project, validating them based on the information from the KIIs and the FGDs. Activity 1: Capacity Building: Government Institutions At the onset of the project, IFPTI conducted a Needs Assessment of NFSA through a series of in-person interviews with key NFSA stakeholders to identify organizational capacity needs. The IFPTI needs assessment team divided these organizational needs into five dimensions: 1) Policies, Procedures, and Programs; 2) Training; 3) Outreach; 4) Integration (Change Management); and 5) Capacity and Systems. In total, the team identified 260 needs distributed among the five capacity-building dimensions. NFSA and the TAIB team planned to regularly revisit this NFSA organizational capacity assessment, using it as a roadmap to track NFSA activities and progress. As part of the midline evaluation, NORC reviewed TAIB program documents to assess the progress of TAIB on building the capacity of NFSA. During the initial needs assessment of NFSA, IFPTI developed a scorecard system to measure progress in the five dimensions – Policies, Procedures, Programs, Training, Outreach, Integration and Capacity and Systems, which NORC found to be an effective way to track progress on increasing NFSA institutional capacity to regulate food safety—an indicator of progress of TAIB under Activity 1. The scorecard uses a point system, 1 for each of the 260 measures, identified as needs in the IFPTI Needs Assessment document. TAIB tracks progress by first assigning a range of scores for each measure based on levels of completion; for example, those that were started are assigned a score of 0.2 and those that are almost complete are assigned 0.8, with completed activities assigned a 1.0 score. They then calculate the progress for each dimension as the percentage of total measures completed in that dimension. TAIB made progress in varying degrees on all dimensions identified in the needs assessment. Below is a description of progress against indicators to date. By the end of project year 3 this score exceeded TAIB’s Project Year 3 target and represents 85.7% of the target for the life of the project. TAIB has also made great progress as measured by 2 other indicators under Activity 1, as described below. • For the indicator, Number of administrative policies and procedures issued or adopted by NFSA, TAIB has supported the issuance or adoption of 20 policies and procedures at the end of year 3, or 83% of the target (24) for the life of the project. • Finally, the indicator, Value of new USG commitments and new public and private sector investment leveraged by USDA to support food security and nutrition, measures investment expenditures (including cost-shares with partners and joint procurements with NFSA) for public and private stakeholders. At the end of year 3, TAIB has leveraged $394,946 total in investments, or 70% of the target for the life of the project ($567,059). 27 The NORC Evaluation Team asked the KII respondents about the NFSA capacity building activities. According to respondents from TAIB, one of the primary successes in the domain of capacity building was the development of a code of ethics under the policies, procedures and programs sub-activity. Another key achievement of the project is the preparation of more than 50 job descriptions for administrative jobs in NFSA under the capacity and systems sub-activity. In addition, TAIB has designed and delivered a suite of training programs that focus on building the soft skills of NFSA’s existing staff for better management under the training sub-activity. Some of the contents of these trainings include leadership and management skills; effective communication skills; and a three-day stress management skills training. The content of these trainings is designed to help NFSA staff, especially middle management, implement daily tasks more effectively while building a productive working culture amongst colleagues and private sector stakeholders. These trainings were given to 12 staff members and at the request of NFSA leadership will be repeated as part of upcoming project activities, to be targeted at administrative staff, inspectors, and managers. In addition, there were numerous procedures developed by TAIB to facilitate NFSA processes such as procedures to receive applications for licenses, procedures to register certification companies, and the development and establishment of procedures for complaint committees to handle food safety complaints from citizens under the policies, procedures and programs and the capacity and systems sub￾activity. In addition, TAIB also prepared a list of frequently asked questions for the private sector to facilitate the food importer licensing process, that details the required application steps. This was one of a several publications that TAIB prepared to help private sector businesses navigate new processes and understand NFSA regulations, including a manual for industry on COVID-19 and infographic videos on NFSA’s website. Table 5 lists the administrative policies and procedures completed under activity 1 as of the end of year three. Table 5: Completed Administrative Policies and Procedures under Activity 1 as of the end of Year 3 No. Title of Administrative Policy or Procedure Purpose 1 Evaluation Forms The Inspectorate evaluation forms are used to evaluate NFSA inspectors in both senior and moderate roles. 2 Continuous Improvement Team Procedure The Continuous Improvement Team Procedure outlines the rules and responsibilities of the team that is responsible for using quality techniques to analyze and provide suggestions to top management. 3 Induction Procedure The Induction Procedure details the awareness and orientation program for new employees. 4 Document Control Procedure The Document Control procedure details how NFSA retains documents and standard forms. 5 Internal Audit Procedure The Internal Audit procedure details roles and responsibilities of the internal audit department. “NFSA is considered the first authority in Egypt to adopt its ethical code. This ethical code contains the ethics that every employee, whether among inspectors or other specialists or jobs, have to commit to.” (TAIB staff, KII) 28 No. Title of Administrative Policy or Procedure Purpose 6 Management Review Procedure The Management Review Procedure establishes systems, roles and responsibilities of planning, implementation, recording management and reviews of the quality management system at NFSA to insure sustainability and effectiveness in accordance with the quality policy and objectives. 7 Nonconformity Procedure The Nonconformity Procedure describes the standard of which NFSA can handle nonconformity related to the quality management system and other systems applied. 8 NFSA Corrective Actions Procedure The NFSA Corrective Actions Procedure explains how to investigate the root cause of non-conformity cases and to take the appropriate actions to erase these cases and ensure it doesn't occur again. 9 NFSA Organizational Context Procedure The NFSA Organizational Context Procedure explains the context of NFSA and outline the internal and external factors related to its purposes and directions. 10 Risk and Opportunities Procedure The Risk and Opportunities Procedure identifies, evaluates, and mitigates risks and identifies and enhances opportunities within NFSA activities and departments. 11 NFSA Training and Development Policy The NFSA Training and Development Policy outlines the detailed procedures of training needs analyses, training planning and training evaluation processes for NFSA in compliance with local law and CAOA decrees. 12 NFSA Performance Management Procedure The NFSA Performance Management Procedure outlines the procedure for planning, evaluating and managing staff performance at NFSA. 13 Technical Competencies Manual The Technical Competencies Manual serves as a guideline for determining the knowledge, skills, and / or abilities for NFSA employees. By using the technical competencies, NFSA management will support the efforts in the several human capital aspects such as learning and development, recruitment and selection, succession planning, and performance management. 14 NFSA Job Families NFSA Job Families classify NFSA jobs that are similar in the field/profession of the work and core accountabilities that based on function. 15 Food Complaints Handling Committee Procedure The Food Complaints Handling Committee Procedure explains processes from receiving to closing food complaints from different entities. 16 NFSA Publication and Style Guide The NFSA Publication and Style Guide illustrates NFSA official publication requirements including the use of official logos, e-mail signatures, internal communication templates and others. 17 NFSA Consumer Guidelines for Food Complaints The NFSA Consumer Guidelines for Food Complaints guide consumers on how to identify and report food complaints. 18 NFSA Food Handling License Standard Operating Procedure The NFSA Food Handling License Standard Operating Procedure explains NFSA processes to issue food handling licenses for restaurants. 19 NFSA Food Disposal Procedure The NFSA Food Disposal Procedure describes steps and procedures of food disposal at NFSA. 20 NFSA Health Certificate Standard Operating Procedure The NFSA Health Certificate Standard Operating Procedure describes NFSA steps to issue health certificates for exported foods. Source: Monitoring Data: Admin Policies and Procedures Registration Data as of 11/11/2021 29 In project year 2, TAIB and NFSA began the process of re-designing and updating NFSA’s existing website to facilitate some of these processes under the needs assessment’s capacity and systems dimension. This new website provides a multitude of services: disseminating information to the public and trading partners (under the outreach sub-activity); enabling businesses to register and access regulatory information, allowing consumers to file complaints, and publishing communications with the public during times of foodborne illness outbreaks. The registration process for businesses will allow them to be publicly listed on the website, creating a whitelist to facilitate partnerships with exporters. Respondents from NFSA staff highlighted this whitelist as a good strategy to encourage businesses to interact with NFSA and comply with food safety regulations. They noted that reduced fees and procedures on imports for whitelist establishments could serve as an extra incentive. However, when asked about the registration process with NFSA, a respondent from a 3rd party certification company complained of problems with communication. They asked for more avenues of communication, such as phone numbers or WhatsApp numbers. For the 3rd party certification companies, TAIB is planning to draft standard operating procedures to guide the inspection process. These are planned for the near future and should address some of these respondent concerns. Apart from the above, one of the most important areas of capacity building from TAIB was the development of inspection protocols to guide inspectors in the field under the policies, procedures and programs sub-activity. In project year 2, TAIB leveraged inspection protocols from the FDA and different US states’ Departments of Agriculture to launch the development of a comprehensive inspection manual. This manual intends to include step-by-step instructions, protocols and procedures for conducting inspections, report writing, sample taking, auditing inspectors, training inspectors, incident and crisis management, enforcement, and inspection record retention. Under this inspection manual framework, TAIB supported NFSA in developing standard operating procedures for inspection and inspection checklists for packhouses, retail establishments, manufactured food and mobile food carts. Activity 2: Training: Food Safety and Management The TAIB project has made great progress in training inspectors and NFSA staff. TAIB, NFSA, and IFPTI successfully adapted the FD-170 FDA training curricula to the Egyptian context and translated it to Arabic. The adapted Arabic language FD-170 program is now a mandatory training for all NFSA food safety inspectors. To facilitate the capacity building of trainers within NFSA TAIB and IFPTI adopted a training-of-trainer approach and led trainings on instructor skills including the FD-170 CSIT, which is a course designed to practice delivering the FD-170 as an instructor and not as a student. A total of 42 people have been trained in a training of trainers program by the end of year 3, which includes a skills development course. TAIB has also trained 1,021 people in government (e.g.: NFSA inspectors and staff) by the end of project year 3, or 90% of the target for the life of the project (1,133) in this category. The total number of people trained in different categories at the end of year 3 is 1,495, or 53% of the target for the life of the project (2,821)—this includes trainees from firms, producers, and civil society. “They applied a good strategy in training. At the beginning they train people on the basics of food safety which is FD170. They applied the training of trainers in which they trained 13 trainers who received highly efficient training with the aim for this group to train their colleagues. This was done in reality. They tested those trainers and selected the first group who participated in training the other NFSA employees with TAIB team.” (NFSA Staff, KII) 30 The NORC evaluation team explored how NFSA staff viewed the trainings and to what extent they benefitted from the trainings. A respondent from NFSA staff commended the success of the training of trainers approach, which helps ensure sustainability beyond the life of the project and called it “highly efficient”. In addition, they noted how the iterative nature of the trainings has allowed for continual improvement with their delivery. The inspectors who received the training of trainers expressed their appreciation for the training of trainers approach during the FGD discussions, noting how it provided them the confidence to train others. During the COVID-19 pandemic, the overall numbers of inspectors trained had to be reduced due to restrictions on the numbers of participants per training session (down from 30 to 12). However, TAIB staff were able to continue implementing this activity and hit their target numbers, using online learning tools. TAIB worked with IFPTI to modify FDA’s Good Manufacturing Practices training program (FD-190) for remote learning and successfully delivered the training course over eight days to a cadre of NFSA’s inspectors. This training was designed to be as interactive as possible. TAIB is also supporting the incorporation of an adapted FD-207 training (FDA’s Plan Review training course) into NFSA’s process for the inspection and approval of catering establishments. A Plan Review is a tool to ensure that food establishment layouts are designed to meet the requirements of safe food production. Importantly, TAIB has also facilitated the launch of an online learning management system with eight courses translated into Arabic, with 7 courses adapted from the FDA. These courses include: NFSA 190: Food GMPS, Application and Evidence Development (adapted from FDA); Sanitation Practices (FDA); Preventative Controls (FDA); Sampling (FDA); Food Allergens (FDA); FD-170 (FDA); Plumbing (FDA); and Advanced HACCP (CODEX). Table 6 lists the numbers of trainees per course in the online learning management system. Table 6: Numbers of Trainees per Course in the Online Learning Management System Course Description Number of Trainees Online: NFSA 190: Food GMPS, Application and Evidence Development 6 Online: NFSA Basic Food Safety (Packhouse) Training - V2_2020 106 Online-Advanced HACCP- NFSA-Q1-2021 44 Online-Food Allergens- NFSA-Q1-2021 131 Online-Plumbing in food establishments- NFSA-Q1-2021 54 Online-Preventive Controls- NFSA-Q1-2021 121 Online-Sampling- NFSA-Q1-2021 119 Online-Sanitation Practices - NFSA-Q1-2021 129 Source: TAIB Indicator Tables. “Before the training, I used to be very stressed and it is my nature not to talk too much and had an introvert personality. But after the training, I became more daring and applied my training on the 500 Pharmacists who are to be appointed by NFSA and they had to receive a month of training […] without the training of trainers it would have been hard for me to do it.” (Inspector, FGD) 31 The trainings that TAIB has provided have led to a shift in the culture of food inspections in Egypt. In the focus groups with inspectors, they described how there is now more flexibility with inspections; the perception of inspectors being government officers coming to punish businesses has been changing gradually and they are instead viewed more as partners on the same team as the businesses, working together to improve their operation. To that end, TAIB has not only greatly expanded NFSA’s library of technical trainings, but they have also rolled out a much appreciated ‘soft skills’ training to help inspectors deal with the social aspects of their work more easily. Despite success in developing training curricula and training inspectors, there have been some challenges faced by TAIB in improving the knowledge and ability of the NFSA staff. Respondents from TAIB explained that simply getting huge numbers of trainees to travel from their respective governates to one place on the first day of training was tough. TAIB has also had to tailor trainings to inspectors with extremely variable levels of background knowledge. TAIB has implemented two training systems that help to overcome the challenges with variability between inspectors. One is the inclusion of ‘soft skills’ trainings, mentioned above, to help inspectors manage relationships with each other, their clients, and their management. This has helped reduce tensions. The other system in place is the A,B,C system which ranks inspectors based on evaluations of their knowledge from the trainings. This seemed to be a good way to help integrate inspectors from other ministries into the general inspectorate work force while avoiding tensions around some having much more experience in food safety—in the focus group they clearly understood that their classification was not based on age or years of experience but on evaluations of their retention of training skills and knowledge. Despite this difficult balancing of training quality with the need to train large volumes of inspectors, plus the COVID-19 pandemic, TAIB is currently on track to meet its targets that were set for the indicators under Activity 2. Descriptions of progress against these indicators to date is described below. For more detail, please see Annex I. • Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance. The target number for this indicator for year 3 is 770, and at the end of year 3 TAIB has trained 881 people, or 114% of the target. The target number of people to be trained by the end of the project is 2,821. As of the end of year 3, TAIB has trained 1,495 people, or 53% of the target for the life of the project; so TAIB will need to accelerate the rate of training to achieve the target by project’s end. • Number of international training programs localized and translated. TAIB has localized and translated 10 programs as of the end of year 3, or 63% of the target (16) for the life of the project. • Number of unique businesses that pass National Food Safety Authority (NFSA) inspection. A total of 2,782 businesses have been approved as of the end of year 3, or 52% of the target number “In your mission, you meet different factory directors. Some of them might not accept that you give them some notes and the issue would be how to deal with those persons. Others might be more likely to engage in long argument so our training in communication skills gave us the needed skills to deal with different personalities.” (Inspector, FGD) 32 for the life of the project of 5,352; 2,492 of these businesses were approved during year 3, showing an accelerating pace of approvals as TAIB continues its efforts. Activity 3: Capacity Building: Promote Improved Policy and Regulatory Frameworks TAIB support has been enormously impactful in the realm of policies and regulations. Since the inception of the project, TAIB prepared policy frameworks and conducted international regulatory review for foundational areas of regulatory control including food exports and labelling. The most significant pivot in strategy that the project has made is in terms of its priorities among the three categories of activities (organizational capacity building, inspector trainings, and regulations). A TAIB respondent described this shift in strategy, which now prioritizes regulations first, then inspector trainings, then organizational capacity building, “as almost the reverse [of the original prioritization of activities].” (TAIB Staff, KII) NFSA staff also clearly stated in their interviews that regulations are their first priority, as evidenced in the quote below from an NFSA staff emphasizing the importance of developing bylaws. TAIB’s shift in strategy reflects not only the priorities of NFSA staff but also the priorities of USDA in facilitating trade flows. In addition, focusing on regulations is an efficient use of TAIB’s technical expertise, and will contribute to the sustainability of the project. Originally, TAIB set a target of 10 policies, regulations, and/or administrative procedures passed or adopted as a result of USDA assistance over the life of the project. In light of the pivot towards prioritizing policies and regulations, they set new targets that increase the number of passed or adopted policies to 20 (stage 5), and also sets targets for policies and regulations in other stages of development. Table 7 shows all the policies, laws, and regulations TAIB is supporting NFSA to pass and the stages of their development. Table 7: Policies and Regulations by Stage of Development Policy, Law, and Regulation Stages 1 2 3 4 5 Chairman's Decree: Reduction in Sampling Rates for Imported Consignments in Response to COVID-19 - Policy Health Certificate - Regulation Amendment of Decision No. 2 / 2020 – Regulation Hotel Establishment Licensing - Regulation Import Chapeau - Regulation Organic Agriculture Executive regulation - Regulation Preborder Notification Requirements - Regulation Pre-Requisite Program for Food Establishments - Regulation Registration of Third Parties for Issuing Conformity Certificates for Exported Consignments and for Imported Consignments- Regulation “Any good audit system has to rely on good bylaws that act as ruler against which you measure the right and wrong. So the first priority is to finish our bylaws […] this is one of the most important things TAIB did with NFSA. We establish committees and if we need some experience that is not available, we request TAIB help in this regard.” (NFSA Staff, KII) 33 Policy, Law, and Regulation Stages 1 2 3 4 5 Registration of Importers - Regulation Risk Categorization and Sampling for Imports - Regulation Veterinary Drug Residues - Regulation Micro, Small and Medium - Sizes Enterprises executive regulation - Regulation Customs Law Executive Regulation Code of Conduct Technical Draft - Regulation Registration of Food Safety Management Systems Certification Bodies - Regulation Mobile Food Carts Technical Requirements – Regulation Food Export Control - Policy Food Additives - Regulation Technical Requirements for Food Storage Establishments – Regulation Temporary Release Requirements for Imported Food Consignments – Regulation Food Sanction Law (Amending Law No. 1/2017 of the National Food Safety Authority – Law Traceability - Regulation Sanctions Law – Law Pesticides – Regulation Judicial Officers - Regulation Exporting Country Recognition Agreements - Regulation Administration of Imports and Exports - Regulation Food Central Laboratories - Regulation Amending decision 1/2019 for the Salt - Regulation Chemical Contaminants Food Transportation - Regulation Harmonized Food Law Guideline for CCAfrica Region - Policy Domestic Risk Categorization - Policy Agriculture Quarantine - Law Slaughterhouses - Regulation Packhouse Technical Regulation - Regulation Microbiological Contaminants - Regulation Licensing of Establishments in Free Zones – Regulation Food Expiration Dates - Regulation Food contact material - Regulation Fish and Fish Products - Regulation Bakeries establishments - Regulation Source: Monitoring Data: All Policy Law Regulation Progress as of 11/11/2021 34 Below is a description of TAIB’s progress against indicators for activity 3 to date. For more detail, please see Annex I. • As of the end of Year 3, TAIB has achieved the passage of 14 policies, regulations, and administrative procedures, or 140% of the original target and 70% of the new target. • Under activity 3, TAIB also measured progress with the indicator, Number of individuals benefiting indirectly from USDA-funded interventions, which covers awareness raising activities such as publishing guides, manuals, and frequently asked question documents for industry to improve the implementation environment for policies and regulations. The target for the life of the project is 55,500, and 44,252 individuals have indirectly benefited as of the end of year 3, or 80% progress against the target. The most significant regulation supported by TAIB was Decision No. 7 of 2020, which was the technical regulation for the risk-based design of food import control. This regulation removed NFSA’s 100% systematic inspection approach and transitioned Egypt to a risk-based approach. Then, in January 2021, Decision No. 6 of 2020: Importer Licensing Requirements entered into force, launching the first stage of NFSA’s new food import control system. In February 2021, the Ministry of Finance issued Decree No. 38/2021, establishing a pre-shipment registration system under a new single window. This single window system is slated to roll out in October of 2021. TAIB is supporting this rollout to ensure that the requirements under NFSA’s food import control system regulation and strategy are incorporated into the system. In addition, TAIB supported NFSA with the implementation of the new risk-based food import control system through developing the process for food import licensing and drafting NFSA’s food commodity risk list that categorizes foods based on their potential to harbor harmful pathogens and hazards. Aligning regulations with international standards is key for TAIB’s impacts on public health and nutrition security in Egypt. Egypt is an import-dependent country, so regulations that make imports more efficient and cheaper have significant implications for food security. The new import control strategy will be extremely important moving forward in facilitating trade and ultimately reducing prices of quality food items for Egyptian consumers. In addition, improved inspection processes will ensure that both imported food and domestically produced food is safer to eat. NFSA staff, in their KIIs, outlined ambitions to focus on the food safety for the domestic sector in the coming years. There are more technical regulations, executive regulations, a sanction law, and policies in the pipeline. Some of these that have been drafted but not yet issued include: a Prime Ministerial Decree that would organize the central food laboratories operating at Egyptian ports to be under the administration of “The most important [support from TAIB] is issuing the legislations and bylaws and these are the most important success stories between NFSA and TAIB. In particular, decree #7 for year 2020 which focuses on risk-based auditing. I consider it one of the most important and successful decrees for NFSA […] on the legislative side, TAIB was very successful.” (NFSA Staff, KII) “We enter a new activity every day, we started with exports and now we are entering supermarkets, restaurants and touristic sector and this would require an ongoing support from TAIB. […] Long term, I would say that the last thing we have are street vendors and the uncontrolled sector.” (NFSA Staff, KII) 35 NFSA; technical regulations for food storage; temporary release requirements for imported food consignments; a chemical contaminants technical regulation that would transition Egypt away from relying on European Union protocols and towards adopting CODEX guidance on risk based management; recognition agreements with foreign competent authorities; and guidance for food expiration dates. However, there remains some room to support NFSA’s communications with private businesses. During KII, private sector respondents complained that NFSA did not communicate with them transparently around the regulatory process. TAIB has been active in supporting the private sector businesses; it held workshops with industry representatives to help them understand the new regulatory requirements. In 2021, TAIB organized one workshop for 41 representatives from food industries to discuss the differences between the old and new food import control systems, explain the benefits of the risk-based approach, and outline the requirements for applying for food import licenses. Another workshop was attended by 26 participants from private sector associations, international development projects, and large-scale trade businesses to review the TAIB-supported Supplier Zero Traceability Manual and discuss approaches to train industry on new traceability requirements. TAIB has also directed efforts to fostering good relationships between NFSA and the Chamber of Food Industries, which will allow for the private sector to take on a more involved role in the development of regulations, policies, and strategies, as well as creating a good enabling environment for their implementation. The respondent from CFI highlighted the importance of a common understanding of the visions and priorities of NSFA. CFI members consider the Chamber as a formal representative of the whole sector. When some businesses are stuck in corrective procedures which might require some understanding and discussion between the companies and NFSA, the Chamber works as a moderator, and when NFSA wants to disseminate messages across the industry, the Chamber is an excellent platform for these messages. 3.2.3 Evaluation of TAIB efficiency A major feature of the TAIB project is its flexibility in the face of changing priorities from NFSA. Respondents from TAIB explained that NFSA is navigating a difficult political environment that can influence its priorities rapidly. However, while NFSA staff may sense the changes in policy direction or priorities from the Government, TAIB staff are not always privy to such information. The management of TAIB has been challenged with changing priorities outside of the control of the project that dictates its activities and timeline. However, TAIB staff have shown high levels of resourcefulness in figuring out ways of side￾stepping obstacles, overcoming challenges, and continuing to make progress against their mandates even if this progress does not come out of the activities originally envisioned. “We are perfectly aligned. […] TAIB, as part of its mission and responsibilities, is responsible for cooperating with NFSA and CFI at the same time to facilitate communicating any NFSA output to the food sector in Egypt in a clear way so a common understanding at the level of visions and priorities can be achieved.” (Chamber of Food Industries, KII) “I think the project adjusted very well, to the needs of NFSA […] I'm really very proud of my time with working with NFSA and TAIB project very, very helpful and very good and excellent staff, leadership.” (USDA, KII) 36 TAIB has also been making good use of MEL data in adjusting their approach in some areas. One way in which the TAIB project improved on its approach in response to MEL data was to expand their activities to work with other stakeholders in the food safety sector instead of focusing solely on NFSA. By working with stakeholders like the Chamber of Food Industries, TAIB can improve the regulatory environment and make new processes and technical regulations easier to implement. Finally, in response to NFSA indicating that its priority was to pass policies and regulations leveraging TAIB’s technical expertise, TAIB changed its original prioritization of the three activities to draft and pass more regulations, updating their target numbers and even re-allocating their budget. The original and the amended budget, as of September of 2021, allocated across the three activity areas are presented in Figure 11. Figure 11: Original and Amended Budget Allocation across the Three TAIB Activity Areas 3.2.4 Key Success Factors and Challenges for the TAIB project Key Success Factors One of the major successes of TAIB thus far has been the trust and partnership that they have fostered with NFSA. In particular, TAIB’s willingness to be responsive to NFSA’s priorities and objectives instead of rigidly keeping to their own agenda has been crucially important. A TAIB respondent explained that at first, they planned out the next six months or a year into the future, but soon realized that this was not agile enough to respond to NFSA’s changing priorities. So, they shortened the duration of their plans to 3 months or even one month. TAIB’s agility was recognized by the NFSA Staff who stated, “With every stage NFSA goes through, TAIB provides support which in most cases is different from its original plan.” (NFSA Staff, KII) 37 The flexibility of the TAIB project has been a key factor in the success of the project despite the challenge of frequently pivoting strategies and priorities and has earned them a role as a trusted partner of NFSA. This overarching strategy of being flexible and responsive to the needs of NFSA was also recognized by the respondent from USDA who commended the management of the project. Another aspect that was clear from the KIIs is that one of the most important value-adds that TAIB brings to the table is technical expertise. NFSA staff greatly appreciated being able to access experts through TAIB’s networks, especially for international experience in specific areas of food safety science. In terms of successes to be replicated, a prime example is TAIB’s efforts to develop alternative processes to work with middle management. A respondent from TAIB recommended that “TAIB needs to continue to work with the middle management in NFSA, this is a very important point.” (TAIB Staff, KII) In addition, continuing the collaboration with stakeholders in the private sector; TAIB activities with the Chamber of Food Industries will continue to be important for their efficacy. Challenges There were several internal and external factors that influenced TAIB’s ability to implement project activities as planned. Most of these factors will also continue to be challenges for TAIB as they progress towards the end of the project, which may require TAIB to work around them to maintain their pace and reach the targets by the end of the project. Overlapping Food Safety Activities between NFSA and Ministries. Among the external factors, there are continual conflicts between NFSA and other ministries that have historically been tasked with food safety oversight. While overlap with ministries is starting to decrease, this is a slow process as NFSA builds its capacity and takes on more and more oversight over different sectors. The work that TAIB is doing to train inspectors and build NFSA’s capacity is key to reducing overlap between them and other ministries. Broader Cultural Awareness around Food Safety. Yet another external factor that is important for the long-term success of the project is broader cultural awareness around food safety. Improving this awareness would not only help the Egyptian public but would also make inspections easier. NFSA staff in their KIIs expressed the need for more public campaigns around food safety. TAIB has already done some work on this front, but NFSA staff has more ideas about how public awareness materials can be updated and expanded moving forward. They called for an updated short “You know, we have been responsive to so many needs of [NFSA], it was just natural when they had another need […] they came to us. […] I think it's really important for future projects, to recognize that and to go into every conversation with your counterpart […] and be open to understanding their point of view and what their priorities are.” (TAIB Staff, KII) “We also have a challenge of changing the society and culture with regard to food safety. People are used to specific practices. To change these practices will take a long time […] This is one of the issues that we need to work on with TAIB. TAIB can support activities to deliver media messages to the public about food safety.” (NFSA Staff, KII) 38 movie on the NFSA as well as branching out into radio and television in coming phases of the project. This was echoed in the focus groups with inspectors, who complained that consumer culture around food safety is a barrier to their work as well since the people they interact with for inspections do not take their mission seriously. The COVID-19 pandemic. Finally, as in everywhere else in the world, the COVID-19 pandemic impacted implementation of some of the TAIB activities, especially those related to capacity building and training. At the same time, it also catalyzed processes that will enhance the sustainability of the project: namely, investments in digitization and acclimation to reduced sampling rates. In the beginning of the pandemic, TAIB implementation activities were slowed and for a brief period, halted before technologies to facilitate communication were established. A respondent from TAIB described this abrupt change in working environment: “Before COVID we used to exist inside NFSA all the time and on a continuous contact and discussion with NFSA policy makers whether on current issues or future ones, receive their support and notes all the time. But with COVID, NFSA was shut down for a while and started to change place and have more than one place. In addition, we had to work in turns and cycles and we worked from home.” (TAIB Staff, KII) The pandemic also diverted attention away from some of the planned organizational capacity building sub-activities under Activity 1, as TAIB staff were focused on passing the temporary measure to reduce sampling rates to 25% to protect the health of the inspectors and facilitate trade during the pandemic (Decision No. 3 of 2020 Reduced Sampling Due to COVID-19), converting trainings to digital formats, and ensuring that all NFSA staff had the necessary materials to protect themselves and stay safe. However, the temporary measure to reduce sampling to 25% aligned nicely with the switch to risk-based sampling; since NFSA is moving away from 100% inspection and sampling anyway, a TAIB respondent explained, this helped sensitize people to reduced inspection rates—dropping down to 25% allowed stakeholders to get used to a different system than 100% consignment. The COVID-19 pandemic posed an enormous challenge for TAIB’s training activities under activity 2, especially since food safety trainings often include important demonstrations of techniques, so it was difficult to adapt them to virtual or hybrid formats. The pandemic pushed TAIB to continue to invest in digital ways of conducting trainings by establishing an online learning platform with 8 internet-based courses, based on FDA and CODEX trainings and translated into Arabic. This was a continuation of investments in digitization that had begun before the pandemic, investments that greatly facilitated TAIB’s activities during the crisis. Before the pandemic, TAIB staff were able to host trainings in their offices with rooms that could hold up to 22 inspectors, but during the pandemic they had to enforce a maximum number of 12 people per room. However, they still hit their targets number of inspectors to train. Now that the virtual trainings are created and the technology is in place, these tools should help TAIB in its continued work in keeping up with demands for trained inspectors as NFSA expands rapidly. However, there could be hybrid solutions that would allow for more individual “The interaction is not as good as the face to face, in some cases, when I do not understand a specific thing, I can talk to the trainer during the break. On Zoom, a break means that the trainer is off and you cannot communicate with him.” (Inspector, FGD) 39 attention for trainees as well as practical, hands-on demonstration of techniques, because the inspectors in the focus group identified problems with virtual training that would create trade-offs in continuing to conduct them virtually. 3.2.5 Sustainability of TAIB Project Activities Project Benefits Likely to be Sustained Past the Project Lifespan Most of the activities of the TAIB project are specifically designed to outlive the project, such as developing training curricula, operating procedures and job descriptions, or regulations. One of the key elements of the TAIB training agenda has been to bring international experts to implement a train-the￾trainer approach and to mentor these NFSA trainers as they conduct the training themselves for other NFSA staff. The training curricula are being institutionalized by NFSA and they are now conducting some training by themselves. NFSA staff was particularly cognizant during the KII of the lasting impact of the regulations that TAIB supported, and greatly appreciate their role in bringing in experts and helping to align regulations with international standards. However, it is also the organizational capacity building products such as job descriptions and standard operating procedures that will continue to provide benefits for NFSA long into the future. Multiple respondents from TAIB pointed to these capacity building deliverables as central to the sustainability of the project. Finally, TAIB’s collaboration with the CFI is likely to have a longer-term effect on Egypt’s food sector. Over the course of the TAIB project, project leadership has increasingly recognized the importance of working with CFI and has begun implementing activities to foster their collaboration and assistance. Specifically, in the last year TAIB has worked on reviewing the Chamber’s organizational structure and job descriptions for their departments, and also provided trainings for 6 staff members of the Chamber of Food Industries that were designated to serve as future trainers. The training programs focused on NFSA’s Law, regulations, and other food related legislation that impacts food businesses in the private sector. Through these activities, TAIB has had a positive and sustainable impact on the implementation environment for new NFSA regulations and processes. Elements Unlikely to be Sustained Past the Project Lifespan Despite the focus of TAIB on sustainability, there are a few activities that seem unlikely to be sustained past the end of the project. The activities that are least sustainable are the provisions of equipment or funding. “There's a direct impact, because the standards that are in place affect trade facilitation in both ways. […] Through this project, for example, Egypt was able to study the risk assessment methodology standards, and […] establish limits or maximum residue limits that are based on Codex […] that's probably the most important I would say, takeaway from this project, is to allow that process to take place in Egypt. So that the standards are science based and consistent with international norms.” (USDA, KII)) 40 4.Conclusions and Recommendations At midline, we conducted a mixed-method evaluation of the TAIB project to examine its impact. We employed two different approaches, one involving a macro-level trade analysis to examine the impacts of TAIB on Egyptian exports and imports and the other a performance evaluation based on document reviews, key informant interviews, and focus-group discussions to examine the progress and impacts of TAIB activities, their effectiveness in reaching targets, and sustainability. The summary of the findings is presented below. 4.1 Trade Analysis Findings We conduct a quasi-experimental impact evaluation to determine whether we can detect changes in the values of exports and imports that can be attributed to the TAIB project. The evaluation design requires comparing actual trade to its counterfactual, that is, what trade would have been without the TAIB intervention. The approach involves comparing trade values of specific product categories subject to NFSA inspection – the intervention group – to counterfactual values synthesized from trade categories not regulated by NFSA but containing products with characteristics of differing degrees of similarity to those in the intervention group – the comparison group. The latter synthesis is carried out using a statistical matching technique. The findings from the trade analysis are: • TAIB has had an impact on exports in 2019 that were almost 12-percent higher than what they would have been had TAIB not been implemented. This effect translates into a US$ increase in exports of almost 293 million (constant 2010). • TAIB’s has had an impact on exports in 2020 that were almost 8-percent higher than what they would have been had TAIB not been implemented and the COVID-19 pandemic not occurred. This effect translates into a US$ increase in exports of almost 185 million (constant 2010). • TAIB has had an impact on imports in 2019 that were ten-percent higher than what they would have been had TAIB not been implemented. This effect translates into a US$ increase in imports of almost one billion (constant 2010). • TAIB’s impact on imports in 2020 is uncertain. Despite the evaluation team’s best effort, we could not calculate a credible estimate for TAIB’s impact on imports in 2020. The issue may be that some of the comparison imports experienced infrequent trade flows over the analysis period, resulting in significant volatility. One of the biggest challenges facing the present impact evaluation is that the onset of the pandemic took place between the start-up of TAIB and (for now) the midline. This requires that we statistically dis￾entangle the COVID-19 effects on Egypt’s trade flows from those of TAIB for detecting actual impacts on trade. While we developed analytic tests to disentangle the TAIB effects from the COVID effects, key data elements from 2020 and 2021 required to carry out the tests were not publicly available at this time. While we have more confidence in the impact estimates for 2019, we recommend caution in using and interpreting the impacts estimates for 2020. 41 4.2 Performance Evaluation Findings We conducted a non-experimental performance evaluation using document reviews, key informant interviews (KIIs) and focus-group discussions (FGDs). The goal of the performance evaluation was to provide a wholistic account of TAIB project’s progress on its activities to date, assesses its impacts, and how it has performed against its targets. We also examine the success factors to TAIB’s performance, key challenges that it faced, and the extent to which the TAIB activities are sustainable. This evaluation found that: TAIB has made significant progress in almost all activity areas against its targets for Activity 1, institutional capacity building. TAIB developed an assessment tool to track the progress in increasing NFSA institutional capacity that the evaluation team found very useful. By the end of project year 3, the score exceeded the Project Year 3 target and represents 85.7% of the target for the life of the project. One of the significant achievements in this activity was the development of inspection protocols. TAIB successfully developed trainings and adopted a train-the-trainer approach that was lauded by NFSA staff, inspectors, and other trainees. TAIB brought, for the first time, a suite of food safety training programs approved by the FDA to a foreign food competent authority leveraging the expertise of American food safety professionals to strengthen Egypt’s food inspectorate workforce. Although the COVID-19 pandemic was disruptive, TAIB was able to continue work on activities thanks to pre￾pandemic investments in digitization and further measures such as setting up institutional zoom accounts for NFSA staff and creating an online Learning Management Platform (LMS) to continue trainings digitally. TAIB has been able to train close to 1,500 people in many different areas over the life of the project. Although the project is on track to meet its year 3 target, it must accelerate the rate of training to meet the target of training 2,821 people by project’s end. TAIB’s support has been enormously impactful in the realm of policies and regulations. As of the first half of year 3, TAIB has achieved the passage of 12 policies, regulations, and administrative procedures, that surpassed the original target. The most significant regulations supported by TAIB was Decision No. 7 of 2020, which was the technical regulation for the risk-based design of food import control, and Decision No. 13 of the Year 2020, which removed Egypt’s zero tolerance on Ractopamine. But there are also tensions between NFSA and the private sector around lack of communications that may weaken the long-term transparency in development of Egypt’s regulatory framework. The flexibility of the TAIB project has been a key factor in the success of the project despite the challenge of frequently pivoting strategies and priorities. Expanding the reach of the project to the Chamber of Food Industries (CFI), shortening the duration of plans submitted to NFSA, and forging strong working relationships with mid-level NFSA staff have been effective strategies to ensure progress. There are substantial internal and external challenges that may impede the progress of the project and/or impact its sustainability. There remains a lack of cultural awareness around food safety in the broader Egyptian community that hampers both the inspection process and the general demand for safe products. 42 4.3 Recommendations Based on the findings of the performance evaluation, NORC proposes that the TAIB project consider implementing the recommendations below. Many of these recommendations call for continuing strategies that TAIB has already developed to circumnavigate obstacles to implementation. In these cases, NORC found that TAIB’s strategies have been effective in moving project activities forward despite persistent challenges. Other recommendations propose new activities that could enhance project sustainability and efficacy moving forward. Continue to work with the entire system of stakeholders such as the CFI to lay the groundwork for implementation of new regulations and policies. This was an effective strategy for TAIB to continue improving food safety in Egypt despite external challenges outside of their control. Not only did awareness raising and capacity building activities for other stakeholders allow TAIB to continue activities when facing delays, these activities are also crucially important to create a conducive environment for new regulations to be implemented. The Chamber of Food Industries is a key stakeholder that can facilitate communication between NFSA and the food industry and can help guide businesses through new processes being implemented. Increase the engagement of private sector stakeholders in the regulation development process with a system that allows them to comment on drafts. This was a recommendation that came up in interviews with both private sector respondents and TAIB staff. Improving feedback processes between NFSA and the food industry would help align regulations and policies with the needs and incentives of private businesses, which in turn will make their implementation smoother and more effective. Focus on improving avenues of communication between private sector companies and NFSA, in particular, with point people that can be contacted or phone services so that everything is not over email. Respondents from the private sector described challenges with communication, particularly during NFSA licensing processes. TAIB is already planning to develop standard operating procedures for new channels of communication, and this will be very helpful in facilitating businesses to register with NFSA. Specifically, respondents called for a point person who could manage communications so that responses to corrections and questions are faster and clearer. Make small improvements to the trainings such as including more demonstrations of application of theory and videotaping them. The switch to virtual trainings, while necessary for the pandemic and useful to maximize numbers of inspectors trained, entailed some tradeoffs. Inspectors in the focus group discussions explained that it was difficult to bridge theory and practice, and that the virtual format of training took away opportunities for individual guidance. Thus, they called for more demonstrations of applying theory, as well as more opportunities to ask questions in person. Additionally, NFSA staff requested more videotaped trainings so their trainers can continue using them as resources past the end of the project. Scale up consumer awareness activities around the importance of food safety, especially in agricultural regions where inspectors frequently work. Improving consumer awareness of food safety was a clear priority from the Government of Egypt and NFSA, not only for better public health outcomes, but also to facilitate smoother and easier inspections. Inspectors in focus group 43 discussions described confusion from business owners that did not understand why they were being asked to make improvements. This was especially an issue in more rural areas. Continue to seek out and build relationships with middle management at NFSA. TAIB has implemented a strategy of building relationships and establishing technical working groups with middle management to spread out these responsibilities. This has been a successful approach and should be continued to avoid delays and to keep work flowing as much as possible. 44 Annex I. TAIB Performance Indicators and Targets for FY2021 and Life of the Project Activity Performance Indicator Disaggregation Baseline Targets Y3 Achieved Y3 Targets LOP Achieved LOP % of LOP Achievement Activity 1 Number of organizations with increased performance improvement with USDA assistance Total 0 1 1 2 1 50% Government Agencies 0 1 1 1 1 100% Activity 3 Number of public-private partnerships formed as a result of USDA assistance Total 0 0 0 0 1 100% Other (Chamber of Food Industries) 0 0 0 1 1 100% Activity 1 Value of new USG commitments and new public and private sector investment leveraged by USDA to support food security and nutrition Total 0 $151,200 $46,865 $567,059 $394,946 70% Activity 3 Number of policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance Total 0 12 23 24 46 192% Stage 1 0 0 4 3 16 533% Stage 2 0 3 3 7 3 43% Stage3 0 0 8 5 11 220% Stage 4 0 2 2 9 6 67% Stage 5 0 7 6 20 10 50% Activity 2 Value of change in trade as a result of NFSA oversight Processed/Value added Products TOTAL TBD 1.0% 10.4% 3.0% 10.4% 347% Activity 2 Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance Total 0 770 881 2,821 1,495 53% Male 0 539 405 2,281 878 38% Female 0 231 476 990 617 62% New 0 700 804 2,821 1,495 53% 45 Activity Performance Indicator Disaggregation Baseline Targets Y3 Achieved Y3 Targets LOP Achieved LOP % of LOP Achievement Continuing 0 70 110 450 301 67% People in firms 0 520 31 2,122 410 19% People in Government 0 250 846 1,133 1,021 90% People in Civil Society 0 0 4 16 41 256% Activity 2 Number of individuals participating in USDA food security programs Total 0 770 881 2,821 1,495 53% Male 0 539 405 2,281 878 38% Female 0 231 476 990 617 62% 15-29 years old 0 308 570 1,290 687 53% 30+ years old 0 462 302 1,981 730 37% People in government 0 250 846 1,133 1,021 90% Proprietors of USDA￾assisted private sector firms 0 0 0 0 0 - People in Civil Society 0 0 4 16 41 256% Laborers 0 520 27 2,122 410 19% Activity 3 Number of individuals benefiting indirectly from USDA-funded interventions Total 0 16,000 33,502 55,500 44,252 80% Activity 2 Number of unique businesses that pass National Food Safety Authority (NFSA) inspection Total 180 700 2,492 5,352 2,782 52% Activity 2 Number of administrative policies and procedures issued or adopted by NFSA Total 0 5 7 24 20 83% Activity 2 Number of international training programs localized and translated Total 0 4 5 16 10 63% 46 Annex II. List of Treated and Untreated Categories HS￾Category Description Treated Categories 02 Meat and edible meat offal 03 Fish and crustaceans, mollusks and other aquatic invertebrates 04 Dairy produce; birds eggs; natural honey; edible products of animal origin, not elsewhere specified or included 05 Products of animal origin, not elsewhere specified 07 Edible vegetables and certain roots and tubers 08 Edible fruit and nuts; peel of citrus fruit or melons 09 Coffee, tea, mate and spices 10 Cereals 11 Products of the milling industry; malt; starches; inulin; wheat gluten 12 Oil seeds and oleaginous fruits; miscellaneous grains, seeds and fruit; industrial or medicinal plants; straw and fodder 15 Animal or vegetable fats and oils and their cleavage products; prepared edible fats; animal or vegetable waxes 16 Preparations of meat, of fish or of crustaceans, mollusks or other aquatic invertebrates 17 Sugars and sugar confectionery 18 Cocoa and cocoa preparations 19 Preparations of cereals, flour, starch or milk; pastrycooks' products 20 Preparations of vegetables, fruit or nuts or other parts of plants 21 Miscellaneous edible preparations 22 Beverages, spirits and vinegar Untreated Categories 1 Live animals 6 Live trees and other plants; bulbs, roots and the like; cut flowers and ornamental foliage 13 Lac; gums, resins and other vegetable saps and extracts 14 Vegetable plaiting materials; vegetable products not elsewhere specified or included 23 Residues and waste from the food industries; prepared animal fodder 24 Tobacco and manufactured tobacco substitutes 25 Salt; sulfur; earths and stone; plastering materials, lime and cement 26 Ores, slag and ash 47 HS￾Category Description 27 Mineral fuels, mineral oils and products of their distillation; bituminous substances; mineral waxes 28 Inorganic chemicals; organic or inorganic compounds of precious metals, of rare-earth metals, of radioactive elements or of isotopes 29 Organic chemicals 30 Pharmaceutical products 31 Fertilizers 32 Tanning or dyeing extracts; tannins and their derivatives; dyes, pigments and other colouring matter; paints and varnishes; putty and other mastics; inks 33 Essential oils and resinoids; perfumery, cosmetic or toilet preparations 34 Soap, organic surface-active agents , washing preparations, lubricating preparations, artificial waxes, prepared waxes, polishing or scouring preparations, candles and similar articles, modelling pastes, "dental waxes" and dental preparations with a basis of plaster 35 Albuminoidal substances; modified starches; glues; enzymes 36 Explosives; pyrotechnic products; matches; pyrophoric alloys; certain combustible preparations 37 Photographic or cinematographic goods 38 Miscellaneous chemical products 39 Plastics and articles thereof 40 Rubber and articles thereof 41 Raw hides and skins (other than fur skins) and leather 42 Articles of leather; saddlery and harness; travel goods, handbags and similar containers; articles of animal gut (other than silk-worm gut) 43 Fur skins and artificial fur; manufactures thereof 44 Wood and articles of wood; wood charcoal 45 Cork and articles of cork 46 Manufactures of straw, of esparto or of other plaiting materials; basketware and wickerwork 47 Pulp of wood or of other fibrous cellulose material; recovered (waste and scrap) paper or paperboard 48 Paper and paperboard; articles of paper pulp, of paper or of paperboard 49 Printed books, newspapers, pictures and other products of the printing industry; manuscripts, typescripts and plans. 50 Silk 51 Wool, fine or coarse animal hair; horsehair yarn and woven fabric 52 Cotton 53 Other vegetable textile fibers; paper yarn and woven fabric of paper yarn 48 HS￾Category Description 54 Man-made filaments 55 Man-made staple fibers 56 Wadding, felt and non-wovens; special yarns, twine, cordage, ropes and cables and articles thereof 57 Carpets and other textile floor coverings 58 Special woven fabrics; tufted textile fabrics; lace, tapestries; trimmings; embroidery 59 Impregnated, coated, covered or laminated textile fabrics; textile articles of a kind suitable for industrial use 60 Knitted or crocheted fabrics 61 Articles of apparel and clothing accessories, knitted or crocheted 62 Articles of apparel and clothing accessories, not knitted or crocheted 63 Other made up textile articles; sets; worn clothing and worn textile articl ... 64 Footwear, gaiters and the like; parts of such articles 65 Headgear and parts thereof 66 Umbrellas, sun umbrellas, walking sticks, seat sticks, whips, riding-crops 67 Prepared feathers and down and articles made of feathers or of down; artificial flowers; articles of human hair 68 Articles of stone, plaster, cement, asbestos, mica or similar materials 69 Ceramic products 70 Glass and glassware 71 Natural or cultured pearls, precious or semi-precious stones, precious metals, metals clad with precious metal and articles thereof; imitation, jewelry; coin 72 Iron and steel 73 Articles of iron or steel 74 Copper and articles thereof 75 Nickel and articles thereof 76 Aluminum and articles thereof 78 Lead and articles thereof 79 Zinc and articles thereof 80 Tin and articles thereof 81 Other base metals; cermets; articles thereof 82 Tools, implements, cutlery, spoons and forks, of base metal; parts thereof of base metal 83 Miscellaneous articles of base metal 84 Nuclear reactors, boilers, machinery and mechanical appliances; parts thereof 49 HS￾Category Description 85 Electrical machinery and equipment and parts thereof; sound recorders and reproducers, television image and sound recorders and reproducers, and parts and accessories of such articles 86 Railway or tramway locomotives, rolling-stock and parts thereat railway or tramway track fixtures 87 Vehicles other than railway or tramway rolling stock, and parts and accessories thereof 88 Aircraft, spacecraft, and parts thereof 89 Ships, boats and floating structures 90 Optical, photographic, cinematographic, measuring, checking, precision, medical or surgical instruments and apparatus; parts and accessories thereof 91 Clocks and watches and parts thereof 92 Musical instruments; parts and accessories of such articles 93 Arms and ammunition; parts and accessories thereof 94 Furniture; bedding, mattresses, mattress supports, cushions and similar stuffed furnishings; lamps and lighting fittings, not elsewhere specified or included; illuminated signs, illuminated name¬plates and the like; prefabricated buildings 95 Toys, games and sports requisites; parts and accessories thereof 96 Miscellaneous manufactured articles 97 Works of art, collectors' pieces and antiques 99 Commodities not specified according to kind 50 Annex III. Supplement to Midline Trade Analysis In order to keep the body of the text as easy to read as possible, the statistical descriptions of the methodologies used in Chapter 2 are presented in this annex. There are four sections in this annex: the first section (Annex to 2.1.2) provides technical details for the econometric estimation of the interrupted time series with potential outcomes (ITSPO) framework impact estimates; the second section (Annex to 2.1.3) presents findings on the prediction accuracy of the counterfactual when the unit of analysis is the country trading partner as opposed to HS trade categories; the third section provides (Annex to 2.2) technical details for the analysis of movement toward increasing trade; and the final section (Annex to 2.3) discusses technical details for the analysis of mitigation of COVID-19 bias from the trade impact estimates. Annex to 2.1.2: Econometric estimation of ITSPO framework impact estimates In Section 2.1.2, we presented the results of what we called the unconditional impact estimates. These were derived from a simple difference of the actual observed values of treatment and the corresponding values of the counterfactual as estimated from entropy matching. This is what is called a design-based quasi-experimental estimate since the matching is assumed to create two groups mimicking random assignment. There is no additional conditioning of these differences using covariates in an attribution model to account for sources of asymmetric influences on the treatment and comparison groups. This reduces the precision of the impact estimate and could even lead to omitted￾variable bias. As mentioned in 2.1.2, a more sophisticated approach would have been to utilize a model specification and conditioning covariates and would be estimated over the combined pre-intervention and intervention period, 2014-2020: 47 However, we were not able to perform this conditional approach due to dearth of data for 2020 and 2021. The equation for this model is: Here, the 𝛼𝛼𝑛𝑛 are coefficients to be estimated, when i equals 1 then 𝑋𝑋𝑡𝑡,𝑠𝑠,𝑖𝑖 denotes its actual outcome data and when i equals 0 then 𝑋𝑋𝑡𝑡,𝑠𝑠,𝑖𝑖 denotes the counterfactual outcome data (the synthetic comparison values derived from the matching) for the period 𝑡𝑡 = [2014,2015, … 2020], 𝑇𝑇𝑡𝑡 = 0 prior to TAIB (≤2018) and 1 once NFSA is running (≥2018), 𝐺𝐺𝑠𝑠,𝑖𝑖 is the experimental group (treatment or comparison) and equal to 1 if i is a treated observation and 0 otherwise, is a HS category fixed effect. 48 Last, the vector, captures various sources influencing intertemporal variation in the out￾47 Of course the down-side of the added sophistication is the introduction of additional assumptions in the form of the model specification. 48Note that there is no𝐺𝐺𝑡𝑡,𝑠𝑠𝑇𝑇𝑡𝑡 . {𝜉𝜉𝑠𝑠} 𝒁𝒁𝑡𝑡,𝑠𝑠 (s) 51 come indicators specific to a sector.49 At the macro level, there will be exchange-rate, and interest-rate fluctuations, and political and policy shocks (e.g., strikes, changes in government). The maintained assumption is that these would affect all sectors – and, therefore, treatment and comparison groups – roughly equally. Next there may be within-food-sector sources of variation, such as weather, exogenous wage and price shocks to food-sector inputs (domestic or imported), and, of course, regulatory (NFSA) influences. Finally, there will be firm-level factors, but, in the present case, these are limited to the number of firms and employment in each subsector since NORC currently does not have access to firms’ or NFSA’s data. Taking derivatives with respect to 𝐺𝐺𝑠𝑠,𝑖𝑖 and 𝑇𝑇𝑡𝑡, the gap between the treatment and comparison groups over the intervention period would be calculated as . 50 For the midline application of Equation (2-1), we have excluded , but retained the time trend and HS-code fixed effects. To convert the results of the above calculation of the US$ value of the trade impacts is based on the following methodology. Let be the US$ value of the TAIB impact for trade-flow F (F=exports or imports) in period t = 2019 (or 2020): Where is the forecast error for each treatment HS code, h from the statistical matching analysis and consists of the de-meaned trade flows, and its associated counterfactual (synthetic comparison), 51 and 𝐶𝐶ℎ,𝑡𝑡 𝐹𝐹 is the counterfactual (comparison) value of HS￾code h’s exports. 𝐶𝐶ℎ,𝑡𝑡 𝐹𝐹 is computed as where the sum is taken over all 76 non-treated HS-code categories and 𝑤𝑤ℎ,𝑘𝑘 𝐹𝐹 is the entropy-matching weight for the kth untreated HS component used to construct the synthetic comparison value of HS code h and 𝑉𝑉ℎ,𝑘𝑘,𝑡𝑡 𝐹𝐹 is the matching trade-flow-value covariate. Finally, to calculate the total impact – exports and imports combined – from TAIB. This is simply ℐ𝑡𝑡 ≡ ℐ𝑡𝑡 X + ℐ𝑡𝑡 M, where t would either be 19 (for 2019) or 20 (for 2020) and, as above, ℐ𝑡𝑡 𝐹𝐹 is the US$ value of the TAIB impact for trade-flow F (F=exports or imports). The total impact, 𝜄𝜄𝑡𝑡, in percentage terms is 49Note that this paragraph is not meant to list every possible potential covariate, but just to provide examples. Moreover, the ultimate choice of covariates will depend on experimentation during model estimation, in part based on what we are able to additionally collect in the Egyptian context for the MTE. 50At endline, when we have more years of data, if we add the term𝛼𝛼7𝑡𝑡𝑇𝑇𝑡𝑡𝐺𝐺𝑠𝑠,𝑖𝑖 to Equation (2-1) then the gap would be calculated as and would represent the estimated impact for the respective time period. This is a simple linear trend and is independent of the HS category. Alternatively one could instead add the expression 𝛼𝛼7𝑡𝑡1𝑇𝑇𝑡𝑡𝐺𝐺𝑠𝑠,𝑖𝑖 + 𝛼𝛼8𝑡𝑡2𝑇𝑇𝑡𝑡𝐺𝐺𝑠𝑠,𝑖𝑖 + 𝛼𝛼9𝑡𝑡3𝑇𝑇𝑡𝑡𝐺𝐺𝑠𝑠,𝑖𝑖 to Equation (3), where 𝑡𝑡𝑡𝑡 = 1 if Period t and 0 otherwise, and test the hypotheses forj,k ∈{1,2,3}, j≠k. 51 The mean used for de-meaning was computed from the associated in-sample (2014-2018) trade-flow values. 𝒁𝒁𝑡𝑡,𝑠𝑠 (s) ℐ𝑡𝑡 𝐹𝐹 52 , and as defined above. Finally, while synthetic matching as described, above, guarantees perfect balance between treatment and comparison groups, at endline we plan to follow Abadie and Gardeazabal (2003) and carry out “placebo testing” to determine whether the gaps (impacts) could have been the result of chance.52 At endline we will have 2 additional years of data that could be enough to perform the placebo testing. Annex to 2.1.3: Prediction accuracy of counterfactual for country-level matching In Section 2.1.3, we presented findings on the prediction accuracy of the counterfactual in terms of forecasting error with the unit being two-digit HS trade categories. In addition, we also examined for robustness the degree of forecast error there would be if the unit of analysis were the country trading partner, of which there were 76. The forecast errors from the application of the same matching used for the HS-code analysis are shown in Figure II.1 and Figure II.2. 52Abadie, A., and Gardeazabal, J. (2003), “The Economic Costs of Conflict: A Case Study of the Basque Country,” American Economic Review, 93 (1), 112–132. In essence, this test reruns the analysis for all the different HS codes in the sample and then calculates whether the change detected in the HS codes of interest could have simply been the result of chance. 𝜄𝜄𝑡𝑡 = 𝑉𝑉𝑡𝑡 X 𝑉𝑉𝑡𝑡 𝜄𝜄𝑡𝑡 X + 𝑉𝑉𝑡𝑡 M 𝑉𝑉𝑡𝑡 𝜄𝜄𝑡𝑡 M , (2-C) 𝑉𝑉𝑡𝑡 ≡ 𝑉𝑉𝑡𝑡 X + 𝑉𝑉𝑡𝑡 M 53 Figure II.1: Within-sample (2014-2018) forecast error for exports, by country code Figure II.2: Within-sample (2014-2018) forecast error for imports, by country code 54 Annex to 2.2: Movement toward increasing trade Though data are not yet available in time for the current midline, NORC proposes to conduct the following analyses at endline since it would still serve to provide a way to document movement toward trade in those sectors which at endline have not had sufficient time for a detectible trade impact to materialize. Movement in employment. Perhaps the most obvious indicator of movement toward trade is an increase in employment in the sectors that should benefit from the food safety reforms supported by TAIB, at least if such movement is the result of an increase in agricultural production relative to employment levels in other sectors competing for the same labor. The approach, below, can be applied separately to exports, imports, and the sum of the two. Consider the case of TAIB-sectors that export. We implement this maintained hypothesis using two alternative types of analysis. First, letting 𝐿𝐿𝑠𝑠 be the size of employment in sector s, we will compare the percentage change in employment, between the twelve months of 2019 and those of July 2020 through June 2021 (aka 2021*) in TAIB-targeted sectors to those of non-TAIB-targeted sectors. At a minimum, this comparison would control for the extent of the fall in employment due to COVID-19, from 2019 to 2020 and changes in an international price index for each traded sector from 2019 to 2021* (July 2020 through June 2021). For example, consider estimating the following model specification: where subscripts, 𝑡𝑡 and s stand for time period (first COVID-19 year = 0, midline = 1) and sector (product category); the 𝛼𝛼𝑘𝑘, 𝑘𝑘 = {0,1,2,3} are coefficients to be estimated; 𝐺𝐺𝑠𝑠 is the experimental group (TAIB￾targeted = 1 or comparison = 0); the vector, 𝒁𝒁𝑡𝑡,𝑠𝑠 captures other sources (e.g., the change in an interna￾tional price index for a sector, pre-COVID-19 average labor productivity, pre-COVID-19 export share of production, change in the real exchange rate) influencing intertemporal variation in the outcome indicators;53 and 𝜀𝜀𝑠𝑠 denotes independent and identically distributed standard errors. The hypothesis to test is whether the estimate of 𝛼𝛼2 is positive can be rejected at, say, a one-sided p-level of 0.05. As a second analytic approach we will use matching (as done in the trade-flows analysis) to examine the extent to which employment in agricultural subsectors exhibits a greater increase (or smaller decrease) than in other sectors. This entails using non-agricultural sectors as the source of comparison sectors for the synthetic matching. The matching should draw on several years of data, e.g., 2015 to 2019, and then gen￾erate forecasts for 2020 and 2021*. These forecasts for each sector predict the expected level of employ￾ment had there not been a pandemic. These forecasts are useful in two ways. First, the difference between the 2020 forecast and actual employment in 2020 provides an estimate of the extent to which employment fell due to COVID-19 in 2020. Second, the respective 2021* forecasts provide estimates of the 53 Pre-COVID-19 average labor productivity would be calculated as 𝐿𝐿𝑠𝑠/𝑌𝑌𝑠𝑠, where 𝑌𝑌𝑠𝑠 is the value of production of sector s, and pre-COVID-19 export share of production would be calculated as 𝑋𝑋𝑠𝑠/𝑌𝑌𝑠𝑠, where 𝑋𝑋𝑠𝑠 is the value of exports of sector s. 55 extent to which sector employment in 2021* recovered from COVID-19, so the differences between them and actual employment in 2021* provides an estimate of the increase in employment due to TAIB.54 Data on sector-level domestic labor employment and value of production would come from the Central Bank of Egypt’s (CBE) Monthly Bulletin and the CAPMAS Report of Internal Trade, respectively. However, we were not able to find it during our exhaustive search of publicly available sources.55 The value of exports would come from the UN Comtrade database. International price indicators could come from U.S. and E.U. statistical agencies and from the U.N. FAO. Movement in number of firms exporting. We would also expect sectors that benefit from the SPS reforms supported by TAIB to manifest movement toward trade by experiencing an increase in the number of firms involved in importing and exporting relative to what other sectors experience.56 The approach, below, can be applied separately to sector that export, import, or do both. Consider the case of TAIB￾sectors that export. We can use the same two analytic approaches that we propose for the case of employment as an early indicator of movement toward trade. First, letting 𝑁𝑁𝑠𝑠 be the number of establishments (firms) in sector s, we will compare the percentage change in the number of firms, between the twelve months of 2019 and those of July 2020 through June 2021 in TAIB-targeted sectors to those of non-TAIB-targeted sectors. At a minimum, this comparison would control for the extent of the fall in the number of firms due to COVID-19, from 2019 to 2020 and changes in an international price index for each traded sector between 2019 and 2020/1 (July 2020 through June 2021). For example, consider estimating the following model specification: where subscripts, 𝑡𝑡 and s stand for time period (first COVID-19 year = 0, midline = 1) and sector (product category); the 𝛽𝛽𝑘𝑘, 𝑘𝑘 = {0,1,2,3} are coefficients to be estimated; Gs is the experimental group (TAIB￾targeted = 1 or comparison = 0); the vector, 𝒁𝒁𝑡𝑡,𝑠𝑠 captures other sources (e.g., the change in an interna￾tional price index for a sector, the sector’s pre-COVID-19 share of establishments exporting, pre-COVID￾19 average labor productivity, pre-COVID-19 export share of production, change in the real exchange rate) influencing intertemporal variation in the outcome indicators; and 𝜈𝜈𝑠𝑠 denotes independent and identically distributed standard errors. The hypothesis to test is whether the estimate of 𝛽𝛽2 is positive can be rejected at, say, a one-sided p-level of 0.05. Likewise, we will also use matching (as done in the trade-flows analysis) to examine the extent to which the number of firms in agricultural subsectors exhibits a greater increase (or smaller decrease) than in other sectors. This entails using non-agricultural sectors as the source of comparison sectors for the syn￾54 In fact, the 2021* forecast would actually either (a) be annualized using recent historical data on the usual rela￾tionship between a year’s average employment and that reported for the first two quarters of a year or (b) running the synthetic matching on just the first six months of each year over the historical period. 55 Likely the pandemic slowed down its collection and or reporting. Perhaps, TAIB staff can ask these agencies whether the information was reported, but not yet released to the public. (Note, these data are sector totals, not firm specific.) 56 One could also use the approach laid out here and in the subsection on employment effects to look at the growth in lower-value products relative to higher-value products as well as the change in the share of lower-value exports. 56 thetic matching. The matching should draw on several years of data, e.g., 2015 to 2019, and then gener￾ate forecasts for 2020 and 2021*. These forecasts for each sector predict the expected number of firms had there not been a pandemic. These forecasts are useful in two ways. First, the difference between the 2020 forecast and the actual number of firms in 2020 provides an estimate of the extent to which the number of firms fell due to COVID-19 in 2020. Second, the respective 2021* forecasts provide estimates of the extent to which a sector’s number of firms in 2021* recovered from COVID-19, so the differences between them and the actual number of firms in 2021* provides an estimate of the increase in the number of firms due to TAIB. In addition to the sources listed at the end of this chapter, the number of firms by sector could be ascer￾tained from the Central Agency for Public Mobilization and Statistics’ (CAPMAS) Count of Establishments. These data are currently not available (please see the table at end of this Annex on data availability). Annex to 2.3: Mitigating COVID-19 bias from trade-impact estimates As mentioned in the main body of this report, to estimate the project’s 2020 impacts NORC endeavored to disentangle the effect of Covid-19 from that of TAIB. Lack of trade data for more than one year of Covid-19 and lack of success in acquiring sectoral data on Egypt’s industrial output and employment due to sporadic reporting in 2020 prevented NORC from statistically identifying the project and pandemic effects at a level of precision worthwhile reporting. A description of the models developed follows, below. Under the assumption that any systematic fall in exports and imports over the period 2019-2020, an , respectively, was due to COVID-19, then we may consider using a sector’s percentage change in a given direction of trade as a proxy indicator of COVID-19’s effect. This indicator can then be included as a control variable in an attribution equation. To test TAIB’s effect on the percentage change in exports over the period 2020-2021, the specification would be:57 where the first dual subscript is for time period (first-COVID-19 year = 0, midline = 1) and the subscript 𝑠𝑠 is for sector (product category);58 the 𝛾𝛾𝑘𝑘, 𝑘𝑘 = {0,1,2} are coefficients to be estimated; 𝐺𝐺𝑠𝑠 is the experi￾mental group (TAIB-targeted = 1 or synthetic comparison = 0); and 𝜁𝜁𝑠𝑠 denotes independent and identically distributed standard errors. We infer that TAIB has had an effect on trade if we can reject the hypothesis that the estimate of 𝛾𝛾1 is less than or equal to zero at, say, a one-sided p-level of 0.05. The test in Equation 2-4) uses the same export/import data we are using for the trade analysis. It is probable, however, that Covid-19 did not have a equivalent effect on the treatment and (synthetic) comparison groups. Therefore, once data become available, NORC proposes to expand the approach leading to Equation 2-4 by drawing on additional assumptions. For example, it is reasonable to posit that, at the highest level, COVID-19 affects trade via supply and demand effects. 57 See Footnote 54 on how we would deal with having on trade data for the first half of 2021. 58 Note that this is an ANCOVA-type regression so there is really only one time period. Hence, 𝐺𝐺𝑠𝑠 and 𝜁𝜁𝑠𝑠 have no time subscript. 57 The supply effect relates to the loss of production due to lower employment and reduced work-hours (e.g., due to lock-downs and illness). One would expect to observe a greater supply effect on those of Egypt’s exports and imports (i.e., trading-partner exports) whose production is more intensive in their use of indoor labor. In support of this conjecture, consider the following analysis of Covid-19 on the Egyptian economy from IFPRI (2020):59 Figure II.3: COVID-19 final estimated impacts n sub-sector production in Egypt, 4th quarter, FY 2019-20 Source: COVID-19 Egypt multiplier model Note: Fourth quarter (April to June) of 2019/20 fiscal year; ICT=Information and Communication Technologies As implied, IPFRI found the services sector being hit hardest, falling by 10.9 percent, followed by industry at -8.3 percent; agriculture is the most resilient sector. The demand effect relates to diminished demand for Egypt’s exports from international markets and for Egypt’s imports due to lower domestic production (for intermediate goods) and lower domestic incomes (for final consumption goods). One would expect to observe a greater fall in demand for those of Egypt’s imports with a higher share of non-luxury final goods and for those of Egypt’s exports with a heavier use by sectors experiencing a greater fall in output in its importing trading partners. Empirically, however, IFPRI op cit. found that “impacts on Egypt’s agri-food system are less severe than elsewhere in the economy”. They forecast that “most damage will occur in nonfarm components of the agri-food system due to falling consumer demand. Although higher-income households face the largest income losses, lower-income households also will see their incomes decline significantly.” 59 IFPRI (2020). “Impact of COVID-19 on the Egyptian economy”. Middle East and North Africa REGIONAL PROGRAM POLICY NOTE 06, June. [Clemens Breisinger, Mariam Raouf, Manfred Wiebelt, Ahmed Kamaly, and Mouchera Karara]. 58 Putting these insights into practice, we can run a two-sided means-test on the null hypothesis that the pre-COVID-19 average labor intensity of production across the treatment sectors and synthetic compa￾rison sectors is statistically equal.60 If the null hypothesis is rejected then we may need to conclude that double differencing failed to remove the effect of COVID-19 from the trade data. Otherwise, we can choose to accept the alternative hypothesis that double differencing succeeded in removing the effect of COVID-19 from the trade data. If indeed the null hypothesis is rejected then we can endeavor to control for COVID-19 effects using this differential labor-intensity causal pathway of COVID-19 as well as the general COVID-19 indicators, and introduced, above. For example, for exports, we can modify Equation (2-4) as: where subscript s is for sector (product category); the 𝛾𝛾𝑘𝑘, 𝑘𝑘 = {0,1,2,3} are coefficients to be estimated; 𝐺𝐺𝑠𝑠 is the experimental group (TAIB-targeted = 1 or synthetic comparison = 0); 𝜆𝜆𝑠𝑠 0 is the pre-COVID-19 average labor productivity; and 𝜉𝜉𝑠𝑠 denotes independent and identically distributed standard errors. The hypothesis to test is whether the estimate of 𝛾𝛾1 is positive can be rejected at, say, a one-sided p-level of 0.05. 60 Again, subject to the caveat provided in Footnote 55. If infeasible at midline, then the adjustment mentioned here in the text would be performed at endline. 59 Table III.1: Data availability Variables Source(s) Level (e.g., country, sector) Availability Notes Y/N Period Format (e.g. pdf) 1. Value or quantity of production by specific sector (not just food/agriculture) Central Bank of Egypt (CBE) Country, Sector N Monthly, 2014 to 2020/21 PDF GDP at Factor Cost data available only for broad sec￾tors (Agriculture, forests and fishing, manufacturing etc.), not specific sectors as would be needed. While trying to access the data in Excel format through the CBE website it shows the error "You do not have permission to open this file" 2. Production of specific food or agricultural commodities CBE, CAPMAS, World Bank Country, Commodities N Annual, 2014 to 2020/21 Unknown Data disaggregated at the commodity level are unavail￾able. 3. Employment growth in agriculture sector and in other non￾agricultural sectors with similar capital and labor intensities CBE Country, Sector N Annual, 2014 to 2020/21 PDF Only total Employment data available for broad sectors (Government, Public Sector, Private). We would ideally need data disaggregated at the agriculture vs non￾agriculture sectors level. CAPMAS Country, Sector N Annual, 2014 to 2020/21 Unknown The CAPMAS site is not accessible. World Bank Country N Annual, 2015 to 2020 Excel, CSV Data available only for broad sectors (Agriculture, Industry, Service) Other - STATISTA Country, Sector Y Annual Excel, PDF The STATISTA site shows that some of these data are available; however, it does not show the data source, so unclear if it is a reliable source (paid access required). 4. Number of firms exporting food products CBE, World Bank (WITS) Country, Commodities N Annual Excel Data are not available CAPMAS Country, Sector N Annual, 2014 to 2020/21 Unknown The CAPMAS site is not accessible. 60 Annex IV. Data Collection Tools NFSA KII Questionnaire - Midline Introduction and Consent To start the interview, please read the following script: Hello and thank you for agreeing to speak with us. My name is _______ (interviewer name) and this is my colleague________. We work with NORC at the University of Chicago, which was hired to evaluate the Egypt TAIB Project implemented by Venture37. Egypt TAIB aims to expand trade of agricultural products domestically, regionally and internationally by building the capacity and supporting the sustainable development of a functioning National Food Safety Authority (NFSA), improving knowledge of and training around food safety and management, and promoting an improved policy and regulatory framework. In the context of this evaluation, we would like to ask you some questions about the National Food Safety Authority (NFSA), its mandate and purpose, and the challenges and opportunities it faces. This interview will last approximately an hour. Your participation is entirely voluntary, and you can choose not to answer a question or terminate the interview at any moment without providing a reason. Your perspective is very important to help Venture37 improve its programs to better support NFSA and the consumers of Egyptian products. The information we will be collecting through this interview will be kept safe by our team. Your responses will be kept anonymous and your identity will be kept strictly confidential, and will not be shared outside of the evaluation team. Other information that could identify you (e.g., specific stories you may tell) will be excluded from reports and other documents produced by our team and shared with Venture37. If you have any questions you may ask them now or later, even after the interview has started. If you wish to ask questions later, you may contact X, e-mail: X, phone: X. This questionnaire has been reviewed and approved by the Institutional Review Board of NORC which is a committee whose task it is to make sure that research participants are protected from harm. If you wish to find about more about these bodies, contact X, e-mail: X, phone: X. Do you agree to participate in this discussion today? Yes No Interview Datasheet Respondent name ______________________________ Respondent title/position________________________ Department ________________________ Interviewer _______________________________ Interview date _______________ Interview start time ____________ Interview end time _____________ Interview disposition codes _____________ 61 NFSA’s work with the TAIB Project I would like to first speak with you about NFSA’s work with TAIB. 1. Which activities have been most successful in your collaboration with TAIB, and why? • [Probe:] Which activities have been most successful in terms of passing new food safety regulations? • [Probe:] Which activities have been most successful in your collaboration with TAIB to build NFSA’s internal capacity? • [Probe:] Which activities have been most successful for training inspectors? 2. What support from TAIB has been most beneficial since the inception of the project from your point of view? 3. What do you think are the most useful ways that the TAIB project can continue to support NFSA? Challenges of the TAIB Project 4. What do you think are the principal challenges with TAIB’s collaboration with NFSA? • How do you think these challenges could be addressed moving forward? 5. What do you think are the principal external and contextual challenges that the TAIB project and NFSA will need to overcome to accomplish NFSA’s mission? 6. [Interviewer: Skip this question if necessary for time:] How has the COVID-19 pandemic affected your work? • How has it affected new policies and regulations that you are working on? • How has it affected internal capacity building for the agency? Opportunities and Sustainability 7. What is NFSA’s vision and principal priorities for the next two years? • What opportunities would you like TAIB to capitalize and work on to help achieve these visions and priorities? 8. What about NFSA’s vision and principal priorities over the longer term (e.g., the next 5 years)? How could TAIB support these longer-term priorities? 9. If NFSA is successful in accomplishing its priorities and mandate, what do you anticipate to be the impact or results of NFSA’s activities on knowledge and perceptions surrounding food safety, if any, with regard to: • Egyptian firms (both exporters and domestic-only actors)? • Imports of food from abroad? 10. What do you foresee to be key factors that will ensure that TAIB’s work with NFSA is sustainable? Closing Questions Do you have any recommendations for how the TAIB project can be improved? 62 TAIB Staff KII Questionnaire - Midline Introduction and Consent To start the interview, please read the following script: Hello and thank you for agreeing to speak with us. My name is _______ (interviewer name) and this is my colleague________. We work with NORC at the University of Chicago, which was hired to evaluate the Egypt TAIB Project implemented by Venture37. Egypt TAIB aims to expand trade of agricultural products domestically, regionally and internationally by building the capacity and supporting the sustainable development of a functioning National Food Safety Authority (NFSA), improving knowledge of and training around food safety and management, and promoting an improved policy and regulatory framework. In the context of this evaluation, we would like to ask you about your experience implementing the TAIB project with the NFSA, the challenges you have faced and the factors that have helped you be successful, as well as opportunities for improved implementation moving forward. This interview will last approximately an hour. Your participation is entirely voluntary, and you can choose not to answer a question or terminate the interview at any moment without providing a reason. Your perspective is very important to help Venture37 improve its programs to better support NFSA and the consumers of Egyptian products. The information we will be collecting through this interview will be kept safe by our team. Your responses will be kept anonymous and your identity will be kept strictly confidential, and will not be shared outside of the evaluation team. Other information that could identify you (e.g., specific stories you may tell) will be excluded from reports and other documents produced by our team and shared with Venture37. If you have any questions you may ask them now or later, even after the interview has started. If you wish to ask questions later, you may contact X, e-mail: X, phone: X. This questionnaire has been reviewed and approved by the Institutional Review Board of NORC which is a committee whose task it is to make sure that research participants are protected from harm. If you wish to find about more about these bodies, contact X, e-mail: X, phone: X. Do you agree to participate in this discussion today? Yes No Interview Datasheet Respondent name ______________________________ Respondent title/position________________________ Department ________________________ Interviewer _______________________________ Interview date _______________ Interview start time ____________ Interview end time _____________ Interview disposition codes _____________ 63 TAIB Project: Successes, Challenges, and Opportunities 1. What do you think have been the greatest successes of the TAIB project in its implementation so far? • Successes in relation to the passage of new laws and regulations? • Successes with internal capacity building for NFSA? • Successes with the training of NFSA inspectors? 2. What have been the primary challenges that you have faced in implementing TAIB? • For new laws and regulations? • For internal capacity building for NFSA? • For the training of NFSA inspectors? 3. How has TAIB’s role evolved since project inception? • What opportunities should the TAIB project capitalize on in the next two and a half years? • How would you recommend TAIB focus its resources moving forward to maximize its impact? NFSA Role and Purpose 4. How do you envision NFSA’s role in Egypt’s food safety space after the end of the TAIB project? • In the current Egyptian context, what do you think are major opportunities that NFSA can capitalize on to accomplish its mission? 5. In your opinion, is there any duplication between the role and activities of the NFSA and those of other GoE agencies and ministries? • [F] If so, which one(s)? 5. Do you know what the principal priorities of the NFSA are for the coming year or two? • [F] Do you think they are appropriate for NFSA to achieve its objectives? For TAIB to achieve its objectives? • [F] What about those over the longer term (for example, the next 5 years)? 6. What do you think of the current approach to food safety inspections? • Do you expect the NFSA to change how it approaches food safety inspections in the future? If so, how do you see this evolution? Project Implementation to Date 7. What do you think of the passage/development process for the new policies and regulations that you are working on with NFSA? Can you think of any ways that it could be more efficient or effective? 8. What are typical professional backgrounds for the newly trained inspectors? (past work experience and education) • How much food safety inspection knowledge did inspectors have prior to the trainings? • Did TAIB encounter any challenges training the inspectors? If yes, what are they? 9. What do you think of the design of the TAIB project? Can you think of any ways in which the design could be improved? • Do you have any lessons learned since 2018 that you could share with us that could potentially inform similar projects in the future? 10. How has the COVID-19 pandemic affected your work? • How has it affected trainings with inspectors? • How has it affected new policies and regulations that you are working on? • How has it affected capacity building activities for NFSA? 64 Sustainability 11. What do you think are the principal challenges of NFSA that will need to be overcome to accomplish its mission? • Do you think the NFSA has sufficient resources to carry out its mission? Why or why not? • Do you think the NFSA has sufficient internal capacity to carry out its mission? Why or why not? 12. What do you foresee to be key factors that will ensure that TAIB’s work with NFSA is sustainable? Closing Questions Are there any additional comments you would like to make? 65 Chamber of Food Industries KII Questionnaire - Midline Introduction and Consent To start the interview, please read the following script: Hello and thank you for agreeing to speak with us. My name is _______ (interviewer name) and this is my colleague________. We work with NORC at the University of Chicago, which was hired to evaluate the Egypt TAIB Project implemented by Venture37. Egypt TAIB aims to expand trade of agricultural products domestically, regionally and internationally by building the capacity and supporting the sustainable development of a functioning National Food Safety Authority (NFSA), improving knowledge of and training around food safety and management, and promoting an improved policy and regulatory framework. In the context of this evaluation, we would like to ask you some questions about any collaboration you have had with the NFSA or the Egypt TAIB project, and related challenges and opportunities for the future. This interview will last approximately an hour. Your participation is entirely voluntary, and you can choose not to answer a question or terminate the interview at any moment without providing a reason. Your perspective is very important to help Venture37 improve its programs to better support NFSA and the consumers of Egyptian products. The information we will be collecting through this interview will be kept safe by our team. Your responses will be kept anonymous and your identity will be kept strictly confidential, and will not be shared outside of the evaluation team. Other information that could identify you (e.g., specific stories you may tell) will be excluded from reports and other documents produced by our team and shared with Venture37. If you have any questions you may ask them now or later, even after the interview has started. If you wish to ask questions later, you may contact X, e-mail: X, phone: X. This questionnaire has been reviewed and approved by the Institutional Review Board of NORC which is a committee whose task it is to make sure that research participants are protected from harm. If you wish to find about more about these bodies, contact X, e-mail: X, phone: X. Do you agree to participate in this discussion today? Yes No Interview Datasheet Respondent name ______________________________ Respondent title/position________________________ Department ________________________ Interviewer _______________________________ Interview date _______________ Interview start time ____________ Interview end time _____________ Interview disposition codes _____________ 66 Egyptian Context 1. In your opinion, what are the principal food safety challenges facing Egypt today? • [F – if several] Which of these do you think is most pressing? NFSA Role and Purpose and the TAIB Project 2. Does the Chamber of Food Industries interact with the NFSA on a regular basis? • [F-Yes] How is the relationship between the Chamber and the NFSA? • [F-Yes] Do you feel that the priorities of the Chamber and the NFSA are aligned? Why or why not? • [F-No] Why not? Do you think there is something that should change? 3. In your opinion, what is the role of the NFSA in addressing food safety issues in Egypt? 4. Does the Chamber of Food Industries implement any activities with its members to help them pass NFSA’s inspection? Which activities? [Probes: trainings, pre-inspections, preparation/delivery of written materials] 5. In your opinion, is there any duplication between the role and activities of the NFSA and those of other GoE agencies and ministries? • [F] If so, which ones? 6. Have you worked directly with the TAIB project? Could you please tell me about any collaboration or interaction the Chamber of Food Industries had with them? • Do you think that the objectives of the TAIB project are aligned with the objectives and priorities of the Chamber of Food Industries? Why or why not? • What have been the impacts of this direct collaboration on your work? • What have been the impacts of this direct collaboration on your work? 7. Has the TAIB project resulted in any indirect impacts on your work for the food industry in Egypt? What are they? • What do you think that TAIB could do moving forward to improve the work of the NFSA and your own work? 8. Did you notice any changes to how the TAIB project or the NFSA operates due to COVID-19 that affected your activities? Challenges, Opportunities, and Sustainability 9. What do you think are the principal external and contextual challenges that NFSA will need to overcome to accomplish its mission? 10. In the current Egyptian context, what do you think are major opportunities that NFSA can capitalize on to accomplish its mission? 11. What do you foresee to be key factors that will ensure that NFSA’s work is sustainable? 12. What are some of your recommendations for the NFSA leadership to accomplish its mission? Closing Questions 13. How has the COVID-19 pandemic affected your work related to food safety? • Did you notice any changes to how the NFSA operates due to COVID-19 that affected your activities? Are there any additional comments you would like to make? 67 3rd Party Certification Companies KII Questionnaire - Midline Introduction and Consent To start the interview, please read the following script: Hello and thank you for agreeing to speak with us. My name is _______ (interviewer name) and this is my colleague________. We work with NORC at the University of Chicago, which was hired to evaluate the Egypt TAIB Project implemented by Venture37. Egypt TAIB aims to expand trade of agricultural products domestically, regionally and internationally by building the capacity and supporting the sustainable development of a functioning National Food Safety Authority (NFSA), improving knowledge of and training around food safety and management, and promoting an improved policy and regulatory framework. In the context of this evaluation, we would like to ask you some questions about how the food safety environment has changed under the NFSA and get your feedback on your collaboration with them. This interview will last approximately an hour. Your participation is entirely voluntary, and you can choose not to answer a question or terminate the interview at any moment without providing a reason. Your perspective is very important to help Venture37 improve its programs to better support NFSA and the consumers of Egyptian products. The information we will be collecting through this interview will be kept safe by our team. Your responses will be kept anonymous and your identity will be kept strictly confidential, and will not be shared outside of the evaluation team. Other information that could identify you (e.g., specific stories you may tell) will be excluded from reports and other documents produced by our team and shared with Venture37. If you have any questions you may ask them now or later, even after the interview has started. If you wish to ask questions later, you may contact X, e-mail: X, phone: X. This questionnaire has been reviewed and approved by the Institutional Review Board of NORC which is a committee whose task it is to make sure that research participants are protected from harm. If you wish to find about more about these bodies, contact X, e-mail: X, phone: X. Do you agree to participate in this discussion today? Yes No Interview Datasheet Organization name _____________________________________________ Organization address _____________________________________________ _____________________________________________ Respondent name _____________________________________________ Respondent title/position_______________________ Department __________________________________ Interviewer _______________________________ Interview date _______________ Interview start time ____________ Interview end time _____________ Interview disposition codes _____________ 68 Introductory Questions 1. Briefly, please tell me about your company. • How long have you been in business? • What are your activities and which products do you certify? Egyptian Context 2. Do you think food safety is a major factor in the ability of Egyptian firms to export and import? Why or why not? 3. Do you think food safety is a major factor in the ability of Egyptian firms to sell domestically? Why or why not? 4. What do you think of the current import control regulations in place? • What are the primary challenges that you have faced with the current import control regulations? • Can you think of any way that they could be improved? NFSA Role and Purpose 5. Does your company interact with the National Food Safety Authority (NFSA)? • [F-Yes] How has your certification work changed since you started working with the NFSA? • [F-No] Why not? Do you think that is something that should change? 6. Please tell me about the registration process with the NFSA. • Were there any challenges that you faced with this process? • Can you think of any ways that this process could be improved? 7. How is your relationship with the NFSA? • Have you faced challenges with your certification work with NFSA? • What could improve your relationship and make your collaboration easier? 8. Do you know what the principal priorities of the NFSA are for the next year or two? 9. Have you heard about a new potential export strategy from the NFSA? • [If yes:] What do you think of it? How do you think it would impact your work? • [If yes:] Do you have any feedback that you could share on how this strategy could be improved moving forward? 10. If NFSA is successful in accomplishing its priorities and mandate, what do you anticipate to be the impact or results of NFSA’s activities on knowledge and perceptions surrounding food safety, if any, with regards to: • Egyptian consumers? • Egyptian firms (both exporters and domestic-only actors)? • Imports of food from abroad? Closing Questions 11. How has the COVID-19 pandemic affected your certification work? • Did you notice any changes to how the NFSA operates due to COVID-19 that affected your business? Are there any additional comments you would like to make? 69 Food Businesses KII Questionnaire – Midline Introduction and Consent To start the interview, please read the following script: Hello and thank you for agreeing to speak with us. My name is _______ (interviewer name) and this is my colleague________. We work with NORC at the University of Chicago, which was hired to evaluate the Egypt TAIB Project implemented by Venture37. Egypt TAIB aims to expand trade of agricultural products domestically, regionally and internationally by building the capacity and supporting the sustainable development of a functioning National Food Safety Authority (NFSA), improving knowledge of and training around food safety and management, and promoting an improved policy and regulatory framework. In the context of this evaluation we would like to ask you some questions about how the food safety industry environment has changed under the NFSA and get your feedback on how this has affected your business. This interview will last approximately an hour. Your participation is entirely voluntary, and you can choose not to answer a question or terminate the interview at any moment without providing a reason. Your perspective is very important to help Venture37 improve its programs to better support NFSA and the consumers of Egyptian products. The information we will be collecting through this interview will be kept safe by our team. Your responses will be kept anonymous and your identity will be kept strictly confidential, and will not be shared outside of the evaluation team. Other information that could identify you (e.g., specific stories you may tell) will be excluded from reports and other documents produced by our team and shared with Venture37. If you have any questions you may ask them now or later, even after the interview has started. If you wish to ask questions later, you may contact X, e-mail: X, phone: X. This questionnaire has been reviewed and approved by the Institutional Review Board of NORC which is a committee whose task it is to make sure that research participants are protected from harm. If you wish to find about more about these bodies, contact X, e-mail: X, phone: X. Do you agree to participate in this discussion today? Yes No Interview Datasheet Organization name _____________________________________________ Organization address _____________________________________________ _____________________________________________ Respondent name _____________________________________________ Respondent title/position_______________________ Department __________________________________ Interviewer _______________________________ Interview date _______________ Interview start time ____________ Interview end time _____________ Interview disposition codes _____________ 70 Introductory Questions 1. Briefly, please tell me about your business. • What products do you import and/or export? • Which countries do you work with outside of Egypt? • How long have you been in business? Egyptian Context 2. Do you think food safety is a major factor in the ability of Egyptian firms to export and import? Why or why not? • What about for your own business specifically? 3. Do you think food safety is a major factor in the ability of Egyptian firms to sell domestically? Why or why not? • What about for your own business specifically? Relationship with NFSA 4. How has the establishment of NFSA affected your business? 5. How have food safety inspections changed? How has that impacted your work? • Have you worked directly with NFSA food inspectors? . [If yes:] How would you characterize your relationships with them? • Can you think of any way in which food safety inspections could be improved? 6. [If the company is an importer:] What do you think of the risk-based import control system? • What are the primary changes under this system that have impacted your business? How have they impacted your business? • Have you faced challenges with the risk-based control system? What are they? 7. What about other changes to food safety regulations since 2017? Can you think of any that have affected your business? • Have you faced challenges with any of them? Which ones? 8. Do you know what the principal priorities of the NFSA are for the next year or two? 9. Have you heard about a new potential export strategy from the NFSA? • [If yes:] What do you think of it? How do you think it would impact your work? 10. If NFSA is successful in accomplishing its priorities and mandate, what do you anticipate to be the impact or results of NFSA’s activities on knowledge and perceptions surrounding food safety, if any, with regards to: • Egyptian consumers? • Egyptian firms (both exporters and domestic-only actors)? • Imports of food from abroad? Closing Questions 11. How has the COVID-19 pandemic affected your business? • Did you notice any changes to how the NFSA operates due to COVID-19 that affected your business? Are there any additional comments you would like to make? 71 USDA: KII Questionnaire – Midline Introduction and Consent To start the interview, please read the following script: Hello and thank you for agreeing to speak with us. My name is _______ (interviewer name) and this is my colleague________. We work with NORC at the University of Chicago, which was hired to evaluate the Egypt TAIB Project implemented by Venture37. Egypt TAIB aims to expand trade of agricultural products domestically, regionally and internationally by building the capacity and supporting the sustainable development of a functioning National Food Safety Authority (NFSA), improving knowledge of and training around food safety and management, and promoting an improved policy and regulatory framework. In the context of this evaluation, we would like to ask you about the Egypt TAIB project, how well it is aligned with USDA objectives, and its challenges and opportunities to improve moving forward. This interview will last approximately an hour. Your participation is entirely voluntary, and you can choose not to answer a question or terminate the interview at any moment without providing a reason. Your perspective is very important to help Venture37 improve its programs to better support NFSA and the consumers of Egyptian products. The information we will be collecting through this interview will be kept safe by our team. Your responses will be kept anonymous and your identity will be kept strictly confidential, and will not be shared outside of the evaluation team. Other information that could identify you (e.g., specific stories you may tell) will be excluded from reports and other documents produced by our team and shared with Venture37. If you have any questions you may ask them now or later, even after the interview has started. If you wish to ask questions later, you may contact X, e-mail: X, phone: X. This questionnaire has been reviewed and approved by the Institutional Review Board of NORC which is a committee whose task it is to make sure that research participants are protected from harm. If you wish to find about more about these bodies, contact X, e-mail: X, phone: X. Do you agree to participate in this discussion today? Yes No Interview Datasheet Respondent name ______________________________ Respondent title/position________________________ Department ________________________ Interviewer _______________________________ Interview date _______________ Interview start time ____________ Interview end time _____________ Interview disposition codes _____________ 72 Introductory Questions 1. How long have you been working with USDA Egypt? 2. In your opinion, what are the principal food safety challenges facing Egypt today? • [F – if several] Which of these do you think are most pressing? The TAIB Project 3. How have TAIB’s objectives and priorities evolved since the beginning of the project? 4. What do you think have been the greatest successes of the TAIB project in its implementation so far? • Successes in relation to the passage of new laws and regulations? • Successes with internal capacity building for NFSA? • Successes with the training of NFSA inspectors? 5. What have been the primary challenges that TAIB has faced? • For new laws and regulations? • For internal capacity building for NFSA? • For the training of NFSA inspectors? 6. What do you think of the design of the TAIB project? Can you think of any ways in which the design could be improved? • Do you have any lessons learned since 2018 that you could share with us that could potentially inform similar projects in the future? NFSA’s Role, Challenges, and Opportunities 8. What do you think are the principal challenges that NFSA will need to overcome to accomplish its mission? • Do you think the NFSA has sufficient resources to carry out its mission? Why or why not? • Do you think the NFSA has sufficient internal capacity to carry out its mission? Why or why not? 9. What do you foresee to be the key factors that will ensure that NFSA’s work is sustainable? Trade and Broader Context 10. How have the outcomes of the TAIB project affected USDA policies? • In Egypt? • In the region? • If the TAIB project is successful in the rest of its implementation, how will this affect trade in Egypt or the region? 11. How would a successfully operating NFSA and a unified food safety regulation in Egypt affect US trade with Egypt? 12. Besides the COVID-19 pandemic, are there any political or other non-food sector related developments in Egypt that could impact the outcomes of the TAIB project? Closing Questions Are there any additional comments you would like to make? 73 FGD Guide for NFSA Inspectors - Midline Introduction and Consent To start the interview, please read the following script: Hello and thank you for agreeing to speak with us. My name is _______ (interviewer name) and this is my colleague________. We work with NORC at the University of Chicago, which was hired to evaluate the Egypt TAIB Project implemented by Venture37. Egypt TAIB aims to expand trade of agricultural products domestically, regionally and internationally by building the capacity and supporting the sustainable development of a functioning National Food Safety Authority (NFSA), improving knowledge of and training around food safety and management, and promoting an improved policy and regulatory framework. In the context of this evaluation, we would like to discuss how your work has changed under the NFSA, the trainings that you have received, and any opportunities to improve food safety inspections in the future. . This discussion will last approximately an hour. Your participation in this focus group is entirely voluntary, and you can choose not to answer a question or leave the focus group interview at any moment without providing a reason. Your perspective is very important to help Venture37 improve its programs to better support NFSA and the consumers of Egyptian products. The information we will be collecting through this focus group discussion will be kept safe by our team. Your responses will be kept anonymous and your identity will be kept confidential, and will not be shared outside of the evaluation team. Other information that could identify you (e.g., your education and previous employment) will be excluded from reports and other documents produced by our team and shared with Venture37. We ask that everyone here respect each person’s privacy and confidentiality, and not repeat what is said during this discussion. But, please remember that other participants in the group may accidentally share what was said. If you have any questions you may ask them now or later, even after the discussion has started. If you wish to ask questions later, you may contact X, e-mail: X, phone: X. This questionnaire has been reviewed and approved by the Institutional Review Board of NORC which is a committee whose task it is to make sure that research participants are protected from harm. If you wish to find about more about these bodies, contact X, e-mail: X, phone: X. Do you agree to participate in this discussion today? Yes No Interview Datasheet Site name ______________________________ Governorate _____________________________ # of participants: Female (___); Male (____) Facilitator _______________________________ Note taker _______________________________ FGD date ________________________________ FGD start time ____________ FGD end time _____________ Recording file name _________________________ 74 Inspection Regime We are going to discuss the risk-based food inspection approach and the impacts of the TAIB project on your work as inspectors. As NFSA inspectors, you are uniquely positioned to speak about this (risk based) approach. 1. Has the support of the TAIB project to NFSA led to changes in how you do your inspections? If so, in what way? 2. Have the TAIB trainings made your inspection process easier? If yes, how? • What kinds of challenges do you face with the firms that you inspect? • Is there anything that could be changed to improve your interactions with the firms that you inspect? [Probe: changes to your scope of work? Your inspection checklists? The resources that you have available?] 3. Do you think that the new guidelines developed by TAIB for inspectors/ food processors are clear? • What do you think of the new scope of work or inspection checklists? Are they effective? o Have you faced any challenges using them? o Can you think of any ways in which they could be improved? 5. What additional support can TAIB provide to improve how you carry out your inspections (i.e. protocols, SOPs, tools, etc.)? a. Do you feel that you have sufficient resources to do your work? i. [Probes: Specifically regarding technical equipment? Specifically regarding availability of transportation for inspectors? Specifically regarding the processes and procedures used within the agency? Specifically regarding data analysis and use of data?] Trainings Next we will discuss the training you have received from TAIB or a TAIB-trained trainer. 6. Looking at all the training you’ve received under the TAIB project in the past two years, have they been sufficient to accomplish what you are tasked to do by the NFSA? Why or why not? • [F] What are important gaps you’d like to see addressed? 7. Now I’d like to ask you about the training on specific topics such as HACCP, Good Manufacturing Practices, inspection protocols, etc. you may have received. • Which of these training topics did you find most useful? • Which of them did you find least useful? • Is there any information or are there any topics that you think should have been covered in these trainings but was not? 8. For those that participated in in-person trainings before the COVID-19 pandemic, what did you think about the format and length of the trainings? • What about those that participated in remote trainings during the pandemic? What did you think about the format and length? Closing Questions Last, we would like to ask you a few general questions about food safety in Egypt. 75 9. In your experience, what are the most significant challenges to improving food safety in the Egyptian market? • [F] What about for importers and exporters specifically? 10. In your experience, what are the most significant opportunities to improving food safety in the Egyptian market? • [F] What about for importers and exporters specifically? Are there any additional comments you would like to make? 76 Annex V. COVID-19 Safety Protocols The following steps will be taken to ensure the safety of NORC in-country consultants and all participants in qualitative data collection in light of the COVID-19 pandemic: Due diligence: Confirming national or local government have not put restrictions in place on movement or businesses during the expected data collection period. Where available, NORC will also review national/local government categorization of risk zones/clusters. Limiting staff travel: Consultant should conduct interviews in one locality or as few as possible. In the event of a positive test at any time during data collection: Individuals are required to stay home when sick, and should get tested for COVID-19 if they are experiencing symptoms or have been exposed to someone who tested positive within 14 days of the exposure. Consultants will alert NORC if they test positive, and they will alert any individuals that they have interacted with in person within 14 days before the diagnosis. If they test positive then they will quarantine: Quarantine Protocol: • Consultants will be quarantined for 14 days under the following circumstances: o After reporting exposure to COVID-19. This includes confirmed or suspected exposure. o After exhibiting any symptoms, including fever, cough, shortness of breath, loss of taste or smell, nausea, diarrhea, headache, fatigue, or body aches. Consultants are expected to monitor their own symptoms every day before leaving for the field. o After testing positive for COVID-19. • Consultants can resume fieldwork after 14 calendar days if (1) they are not exhibiting any symptoms AND (2) they have tested negative for COVID-19. The test should be done as soon as symptoms develop, so the individual can be treated, or 14 days after exposure if no symptoms develop, to confirm they are not infected. • The consultant will report any events that lead to being quarantined to NORC. During Interviews: • Sanitize all data collection instruments (electronic device, pens etc.) and personal items (such as glasses, cell phones, and keys) before the start and after the end of each KII/FGD. • Practice physical or social distancing according to local health and safety guidance. o Greet respondents in a friendly, but distant way that is culturally appropriate. o Physical exchange of documents or devices should be avoided as much as possible. • Wear a mask at all times (excluding meals); o The use of face masks is mandatory, if culturally appropriate. Using the mask correctly is essential to avoid contagion. Likewise, consultants must be careful when removing their masks in order to avoid contact with contaminated areas and/or the dispersion of the infectious agent. Cloth masks can be used. • Suggest all respondents/participants to wear a mask for the duration of the KII/FGD, if possible and culturally appropriate. o Provide masks to all KII/FGD participants that do not already have one. • Refrain from touching mouth, nose and eyes. 77 • Practice proper hand washing protocol: o Hands should be washed between interviews, before and after using the bathroom, before and after eating, and after sneezing and coughing. o Carry glycerinated alcohol (minimum 60% and maximum 95%), or antibacterial gel at all times.  Hand hygiene with glycerinated alcohol or antibacterial gel should be performed until the hands are visibly clean. It is important to remember that glycerinated alcohol and antibacterial gel do not replace adequate hand washing, but it should be used when there is no possibility of cleaning with soap and water. • Conduct FGD in an indoor space, large enough so that (up to) 8 participants and 1 data collector can sit in a horseshoe or U-shape and maintain 6 feet (2 meters) between everyone in the room. The data collectors will sit at the head of the group and the digital recorder will be placed in the center of the group to best record sounds. For KIIs, the space must accommodate 2 individuals with 6 feet (2 meters) between them. • Limit provision of refreshments to study participants, to the extent possible. Where refreshments must be provided, consultant will consider how best to distribute refreshments in a way that limits people interacting physically and ask participants to wash their hands thoroughly both before and after eating (e.g. by setting the refreshments out on a table and having staff/participants come one-at-a-time). Use private transport whenever possible: • For private vehicles: Surfaces which individuals come into contact with will be disinfected daily, such as the door handles, steering wheel, gear lever, seat belt buckles, spokes, etc. o Consultants will wash their hands or use hand sanitizer immediately after finishing refueling • For public transport: o Consultants must wear face masks and, as far as possible, maintain a minimum distance of two meters from other people. o Windows will be kept down, whenever possible • For motorcycles and bicycles: o Staff must wear face masks and disinfect the handlebars, helmet, gloves, glasses, and knee pads, among other