October 2013 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Evan Scott Thomas, Melissa Chiappetta, Joseph Limange, and Ben Mazzotta of Social Impact, and Bechir Rassas and James Maraga of Management Systems International. AFRICA TRADE HUBS EXPORT PROMOTION EVALUATION Evaluation Report Photo Caption: Employees from a USAID-beneficiary Ghanaian apparel and textile firm hard at work. (Photo Credit: Melissa Chiappetta) AFRICA TRADE HUBS EXPORT PROMOTION EVALUATION EVALUATION REPORT October 2013 USAID Evaluation Services IQC No: AID-RAN-I-00-09-00019 Task Order No: AID-OAA-TO-12-00040 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. iv Africa Trade Hubs Evaluation Report CONTENTS Acknowledgments...............................................................................Error! Bookmark not defined. Contents........................................................................................................................................ iv Figures and Tables ........................................................................................................................ v Acronyms...................................................................................................................................... vi I. Executive Summary................................................................................................................... 1 Evaluation Purpose and Questions................................................................................................................. 1 Project Background............................................................................................................................................ 1 Evaluation Design, Methods and Limitations................................................................................................ 1 Key Findings and conclusions........................................................................................................................... 2 Recommendations .............................................................................................................................................. 3 II. Evaluation Purpose & Evaluation Questions.......................................................................... 5 Evaluation Purpose ............................................................................................................................................. 5 Evaluation Questions......................................................................................................................................... 5 III. Project Background................................................................................................................. 6 Development Goal ............................................................................................................................................. 6 Development Hypothesis................................................................................................................................. 6 Export Performance........................................................................................................................................... 7 Project Context.................................................................................................................................................. 7 IV. Evaluation Methods & Limitations....................................................................................... 14 Evaluation Methods..........................................................................................................................................14 Evaluation Scope ...............................................................................................................................................15 Evaluation Limitations......................................................................................................................................16 V. Findings, Conclusions & Recommendations ........................................................................ 24 West Africa Trade Hub Impact Evaluation................................................................................................. 24 East and Central Africa Trade Hub Performance Evaluation ................................................................. 36 Southern Africa Export Trends.....................................................................................................................51 Summary Findings and Conclusions .............................................................................................................52 Recommendations............................................................................................................................................57 Annexes........................................................................................................................................ 60 Annex I: Evaluation Scope of Work.............................................................................................................61 Annex II: West Africa Evaluation Methods and Limitations...................................................................71 Annex III: Data Collection Instruments ......................................................................................................87 Annex IV: Sources of Information ............................................................................................................. 101 Annex V: Additional Lessons Learned from WATH Survey............................................................... 135 Annex VI: Data Monitoring and Evaluation for Future Trade Hubs.................................................. 140 Africa Trade Hubs Evaluation Report v FIGURES AND TABLES Figures Figure 1: Sub-Saharan Africa Exports to the United States in USAID-Supported Sectors..............................7 Figure 2: ECATH Results Framework ........................................................................................................................11 Figure 3: Schematic of Impact Evaluation...................................................................................................................24 Figure 4: West African Exports to the United States in USAID-Supported Sectors......................................27 Figure 5: Percent Change in Sales and Exports in Assisted vs. Non-Assisted Firms.......................................28 Figure 6: Export Growth by Market Destination ....................................................................................................32 Figure 7: East and Central African Exports to the United States in USAID-Supported Sectors..................38 Figure 8: ECATH Firms’ CAGR for Sales in Periods 1 and 2 ...............................................................................40 Figure 9: ECATH Firms’ Mean Absolute Annual Sales Growth in Period 1 and Period 2.............................41 Figure 10: ECATH CAGR for Exports Periods 1 and 2.........................................................................................43 Figure 11: ECATH Firms’ Mean Absolute Annual Export Growth in Period 1 and Period 2.......................44 Figure 12: ECATH Firm Export Diversification in Periods 1 and 2.....................................................................47 Figure 13: ECATH Firms’ CAGR for Employment in Periods 1 and 2...............................................................49 Figure 14: ECATH Firms’ Absolute Annual Growth in Employment for Periods 1 and 2 ............................49 Figure 15: Southern Africa Exports to the United States in USAID-Supported Sectors (in $ millions) ....52 Figure 16: Challenges in Production and Export................................................................................................... 135 Figure 17: Change in Constraints to Export—Treated Firms........................................................................... 136 Figure 18: Type of Assistance Received—Treated vs. Untreated .................................................................... 137 Figure 19: Beneficiary-Reported Type of Assistance with Most Impact—Treated vs. Untreated............ 139 Figure 20: Example Results Chain............................................................................................................................. 141 Tables Table 1: Potential Spillover Effects of WATH Assistance......................................................................................19 Table 2: Surveyed Firms That Have Received Assistance from Sources Other than WATH.....................19 Table 3: Selected West Africa Trade Hub Results in Ghana and Other Countries, 2007–2011 ...............25 Table 4: Average Sales and Exports at Baseline and Endline for Treatment and Comparison Firms........28 Table 5: Exports at Baseline and Endline for Treatment and Comparison Firms, by Value Chain ($’000)29 Table 6: Ghanaian Exports (in US$ million) to the United States by Sector....................................................30 Table 7: Average Number of Export Market Destinations at Baseline and Endline for Treatment and Comparison Firms, by Value Chain....................................................................................................................31 Table 8: Average Size of Major Export Market, by Destination, at Baseline and Endline for Treatment and Comparison Firms, by Value Chain ($’000).............................................................................................32 Table 9: Average Number of Workers at Baseline and Endline for Treatment and Comparison Firms..33 Table 10: Average Number of Total Workers at Baseline and Endline for Treatment and Comparison Firms, by Value Chain............................................................................................................................................34 Table 11: Average Number of Women Workers at Baseline and Endline for Treatment and Comparison Firms, by Value Chain............................................................................................................................................35 Table 12: AGOA-Related Exports Attributed to USAID Trade Hubs’ Export Promotion Assistance.....55 Table 13: West African Trade Hub Attributed Exports and Export Promotion Expenditures by Value Chain, FY 2010–2012 ............................................................................................................................................56 Table 14: Inferential Problems That Result from Missing Data ............................................................................75 Table 15: Reported Constraints to Export............................................................................................................ 136 Table 16: Reported Type of Assistance Received ................................................................................................ 138 vi Africa Trade Hubs Evaluation Report ACRONYMS ACA African Cashew Alliance AGCI African Global Competitiveness Initiative AGOA African Growth and Opportunity Act CAGR Compounded Annual Growth Rate CLA Collaborating, Learning and Adapting COMPETE Competitiveness and Trade Expansion Project COR Contracting Officer Representative ECA East and Central Africa ECATH East and Central Africa Trade Hub ERQ Export Readiness Questionnaire EU European Union FY Fiscal Year GSA Global Shea Alliance HS Harmonized System IE Impact Evaluation IEHA Initiative to End Hunger in Africa IQC Indefinite Quantity Contract ITC International Trade Centre KII Key informant interview LIDI Leather Industry Development Institute of Ethiopia M&E Monitoring and Evaluation MSI Management Systems International NGO Non-Governmental Organization PE Performance Evaluation PIR Project Intermediate Result PMP Performance Monitoring Plan QED Quasi-Experimental Design RATES Regional Agricultural Trade Expansion Support REDSO Regional Economic Development Support Office RFP Request for Proposals SI Social Impact, Inc. SME Small and medium enterprises SOW Scope of Work TCB Trade Capacity Building TCP Trade Competitiveness Project TRADE Trade for African Development Initiative Africa Trade Hubs Evaluation Report vii UN United Nations USAID U.S. Agency for International Development USD U.S. Dollar USITC U.S. International Trade Commission USTR U.S. Trade Representative WA West Africa WATH West Africa Trade Hub Africa Trade Hubs Evaluation Report 1 I. EXECUTIVE SUMMARY EVALUATION PURPOSE AND QUESTIONS The United States Agency for International Development (USAID) contracted with Social Impact, Inc. (SI) to conduct an evaluation to assess the effectiveness of firm-level export promotion activities of the Africa Trade Hubs Projects, which had been in operation for more than a decade. The questions in the scope of work (SOW) for the assignment mandate an evaluation of USAID firm level intervention effects in terms of sales, exports, export diversification and job creation, as well as sectoral and regional effects and cost-effectiveness. PROJECT BACKGROUND USAID’s Africa Trade Hubs operate under the development hypothesis that trade access provided under the African Growth and Opportunity Act (AGOA), coupled with USAID technical assistance and training activities, will help achieve the development goal of expanding nontraditional exports from sub￾Saharan Africa to the United States and other destinations. EVALUATION DESIGN, METHODS AND LIMITATIONS The SOW for this assignment notes a preference for an impact evaluation (IE) utilizing quantitative analysis, if feasible, supplemented with qualitative evaluation methods to enrich the findings and conclusions. The evaluation team found that the Trade Hub implementing partner in West Africa (WA), which was the contractor since inception of the original Trade Hub contract, had maintained questionnaires from companies applying to work with the Hub, and that these questionnaires could be utilized as a baseline to form treatment and comparison groups. The evaluation team, in consultation with USAID, determined that a quasi-experimental IE could therefore be attempted in West Africa. The team found that relative to the other West African countries where the program was implemented, a reasonably large proportion (28.3 percent) of all the firms assisted by the Hub were located in Ghana, and based on this, the team conducted a firm-level survey of 128 firms in and around Accra. The sample of firms comprised almost all of the firms in and around Accra that were known to have ever applied for assistance from the Trade Hub, including both assisted and unassisted firms representative of all the sectors in which Trade Hub export promotion support has been provided. In East and Central Africa (ECA), there were no such comparable baseline data, and no list of comparison firms, either past or present. Moreover, the current Trade Hub implementing partner had been in place only since 2009 and had only assisted 96 firms across all the countries in the region. Accordingly, the evaluation team, again in consultation with USAID, determined that an IE was not feasible, and instead conducted a performance evaluation using quantitative techniques to compare firm￾level indicators before and after assistance was provided. The main quantitative technique the team used was to conduct a telephone survey of assisted firms across the region. Of the 96 firms contacted, 64 responded, representing 67 percent of all assisted firms. In Southern Africa, firm-level export promotion support ended in 2012, and according to USAID the current implementing partner had no information on prior firm-level support activities performed by the previous implementing partner, so the evaluation was limited to examination of regional trends and value-chain cost-effectiveness. 2 Africa Trade Hubs Evaluation Report The evaluation team acknowledges that both the IE of the WA Trade Hub (WATH) and the performance evaluation of the ECA Trade Hub (ECATH) are subject to potential methodological biases that may weaken some of the evaluation’s findings. In the WATH IE, the “application for assistance” questionnaires, intended to be used as baseline data, suffered from many gaps. Thus, both the baseline and the endline data had to be reconstructed, resulting in concerns about potential recall bias. Other sources of concern include the potential for selection bias—because the Trade Hub model is to try to select those firms most likely to succeed—as well as the potential for bias due to non-uniform baseline years and “spillover” of beneficial effects of USAID export promotion support activities to non￾beneficiaries. Similarly, in the ECATH performance evaluation, there are concerns over the potential for recall bias, selection bias and bias due to differing baseline years. KEY FINDINGS AND CONCLUSIONS Sales and Exports Based on analysis of responses from firms surveyed in and around Accra, USAID export promotion assistance appears to have positively affected the sales and export performance of the firms assisted by the WATH, both in the aggregate and in each of the value chains supported. Although statistical tests of these effects were inconclusive, even in sectors in which the exports of surveyed firms actually fell on average (likely due to market factors outside of Trade Hub assistance), exports from treatment firms fell by less on average than did those of the comparison group. The findings from the performance evaluation of firms assisted by the ECATH suggest that USAID￾assisted firms performed relatively well in ECA as well. This occurred despite the fact that the ECATH￾assisted firms were active predominantly in the apparel and textiles sector, which experienced declining exports to the United States from the sub-Saharan Africa region as a whole and stagnating exports from the ECA region during the period under review. Export Diversification The survey of firms in and around Accra also offered concrete evidence that firms assisted by the USAID Trade Hub appear to have been more successful in diversifying export markets than did unassisted firms. Perhaps more important, the average size of export markets by destination increased more rapidly for assisted firms than for unassisted firms, with major gains occurring in exports to the European Union (EU) and West Africa among firms in the treatment group—gains that were not mirrored among firms in the comparison group. The conclusion that these findings may be representative of sub-Saharan Africa more generally tends to be supported by the findings of the performance evaluation from the ECATH, in which assisted firms in all value chains except floriculture increased their export destinations (countries to which the firms exported), on average, between Period 1 (before receiving USAID Trade Hub assistance) and Period 2 (after receiving the assistance). Job Creation Survey results from the WATH IE indicated that employment by treatment group firms declined on average, while employment by comparison group firms increased. This counterintuitive result may be explained by the fact that gains in employment in certain value chains were more than offset by job losses in apparel and textiles, which are labor-intensive value chains wherein overall exports to the United States, as well as the number of export markets, have both declined significantly in recent years. By way of contrast, the findings from the performance evaluation carried out in ECA provide evidence that USAID Trade Hub assistance is correlated with job creation. Whereas 34 percent of the firms Africa Trade Hubs Evaluation Report 3 reported increasing employment in the period before starting to work with the Trade Hub, 62 percent increased employment in the period after the assistance began. In terms of women’s share of employment, the ECATH firm level survey found that while it remains much higher than men’s—at 73 percent of the workforce among firms sampled by the evaluation team’s phone interview—this percentage has remained the same since the year before firms first began working with the Trade Hub. However, the WATH survey found that women’s share of employment among assisted firms declined by 27 percent after working with the Trade Hub. This is most likely explained by the decline in employment in the apparel and textiles sector—which employs an overwhelming predominance of women. Cost Effectiveness The evaluation team encountered considerable difficulty in obtaining comparable data on exports and expenditures from the three Trade Hubs that could be utilized to make comparisons of the cost￾effectiveness of USAID trade promotion support. Sectoral and Regional Effects The evaluation team was only able to gather information about attributed exports by value chain from the WATH. Findings demonstrate that more resources were being channeled to those value chains that produced the most exports per USAID dollar spent. To understand what happened in ECATH, the team conducted regression and correlation analysis and did not find any statistically significant relationships between export or employment growth and sector. Qualitative findings also did not identify any sectoral differences in outcomes in ECA. Exports to the United States in sectors supported by the Trade Hubs increased for all three Hubs (and more so than exports to any other region). The most dramatic increases appear to be in the food and food ingredients sector in both WA and ECA, and in manufactured goods other than apparel in Southern Africa. RECOMMENDATIONS • USAID should require future Trade Hub implementing partners to submit a detailed Performance Monitoring Plan (PMP), supported by a development hypothesis and results framework, at the project’s outset. The PMP should include outcome-level indicators and input- and output-level indicators, as well as indicators aimed at measuring whether or not theory-of-change assumptions hold true. • USAID should require the Trade Hubs to monitor both surviving and failing firms for results reporting purposes and follow up with site visits and qualitative surveys to both types of firms to enrich USAID and Trade Hub knowledge of what works and what does not in firm level export promotion assistance. When a firm goes out of business, implementing partners should make a concerted effort to interview the business owner to understand the reason behind the firm’s closure. • Given the multitude of potential methodological biases and restricted sample size, any future IE of Trade Hub programming should be designed alongside the project. This is particularly critical for development of an approach to measure spillovers, which is an important element of the Trade Hubs’ program design, yet one that can lead to biased evaluation results if unaccounted for. • Collection of longitudinal data on client exports and employment should be required of Trade Hubs’ clients so that cost-effectiveness of support to different value chains can be documented. 4 Africa Trade Hubs Evaluation Report Cost-effectiveness of alternative interventions and implementation approaches should be documented to help identify interventions that use resources most efficiently. • USAID should maintain a centralized database of all Trade Hub reports, including Requests for Proposals (RFPs), work plans, annual and quarterly reports and PMP data in order to ensure institutional memory for future implementing partners and evaluation implementing partners. • In line with its policy of encouraging “collaborating, learning and adapting (CLA),” USAID should periodically assess whether the value chains that the Trade Hubs are supporting actually are those with the greatest potential to generate exports and employment, and if needed, guide the implementing partners to make midstream corrections accordingly. • USAID should consider requiring the Trade Hubs to continue market-linkage work with firms either directly or through associations. Quantitative evidence from both the WATH and ECATH shows that firms report that this type of assistance has had the most impact on increasing exports. Qualitative interviews demonstrated that the best method for assisting firms in this regard is through a step-down process, meaning firms are supported more heavily (financially and technically) when they first begin working with the Trade Hub (or association) and then, slowly, support is lessened until firms are supporting themselves entirely. • USAID should consider requiring the Trade Hubs to expand efforts to reduce firm constraints to exporting by facilitating firm level access to finance and reducing regulatory barriers to exports. Results of regression analysis from the ECATH show that firms that have received support in accessing finance were more likely to have increased exports than firms that did not. In addition, access to finance and regulatory barriers were highlighted by firms in both ECA and WA as the main barriers to firms’ export growth. Africa Trade Hubs Evaluation Report 5 II. EVALUATION PURPOSE & EVALUATION QUESTIONS EVALUATION PURPOSE The purpose of this evaluation was to assess the effectiveness of firm level export promotion activities fielded by the USAID Africa Trade Hub projects, based on more than a decade of experience following their inception. The evaluation objective was to inform project design, reporting parameters and evaluation methodologies for future projects. EVALUATION QUESTIONS The following, drawn from the SOW for this assignment, is a synopsis of the key questions addressed by the evaluation: • Firm Level Effects: Did exports/revenues increase as a result of USAID export assistance? Was there a statistically significant difference in export performance by assisted firms versus overall regional/global export trends in those sectors/products (i.e., more than in entities that were not assisted)? Is there a statistically significant difference in export/revenue performance between firms that received assistance and those that did not? • Export Diversification: Has trade/export assistance delivered by the Trade Hubs helped to diversify trade in assisted sectors/firms? Was there a statistically significant difference in the export destination or export product mix of assisted firms in comparison to overall export trends in this product/sector? • Job Creation: Did the increase in exports/revenues lead to the creation of jobs in assisted firms? Is there evidence that this job creation exceeded overall job creation in the sector (i.e., exceed the number created in firms in the same area/industry that were not assisted)? Was there a statistically significant improvement in job creation in assisted firms from the period before the assistance began to the period following the assistance? • Expenditures: What is the relationship between USAID Trade Hub expenditures on export assistance and export/revenue and job creation performance (ratios of expended costs to exports/revenues/jobs generated)? To what extent can changes in export performance or job creation be attributed to USAID Trade Hub expenditures? • Sectoral Effects: Is trade/export assistance delivered by the Trade Hubs relatively more effective within specific product sectors (e.g., traditional agriculture products, nontraditional agriculture products, manufactured goods, services, etc.)? • Regional Effects: Has trade/export assistance delivered by the Trade Hubs been relatively more effective in generating exports/revenues/jobs in specific countries/regions or more effective in generating exports to certain destination markets over others (e.g., traditional agriculture products, nontraditional agriculture products, manufactured goods, services, etc.)? Each of these questions is addressed in detail in the following sections. Section III presents background and context on each of the three regional Trade Hubs. Section IV outlines evaluation methods and limitations. Section V presents detailed findings, conclusions and recommendations. Annexes provide additional detail on evaluation methods and limitations, data collection instruments and sources of information. 6 Africa Trade Hubs Evaluation Report III. PROJECT BACKGROUND DEVELOPMENT GOAL As confirmed by the U.S. Trade Representative, one of the goals of AGOA was to help diversify sub￾Saharan Africa exports to the U.S., specifically in reference to the predominance of petroleum exports in total sub-Saharan Africa exports.1 Utilizing the AGOA umbrella, USAID’s Trade Hub projects pursued the development goal explicitly to help partner countries expand “nontraditional” exports to the U.S. from sectors other than petroleum and minerals. DEVELOPMENT HYPOTHESIS USAID’s Africa Trade Hubs operate under the development hypothesis that AGOA trade access, coupled with USAID technical assistance and training activities, will help achieve the development goal of expanding nontraditional exports from sub-Saharan Africa to the U.S. and other destinations. USAID activities, which are expanded upon in subsequent sections, include support for improvement of the business-enabling environment, facilitation of infrastructure improvements, improving access to credit and export promotion technical assistance and training provided directly to individual firms. USAID identifies nontraditional value chains with the greatest export potential and,2 through export promotion support from the Trade Hubs, provides technical and training support to firms within those value chains that appear capable of taking advantage of that potential. Subsidiary objectives relevant to USAID support worldwide, such as providing support to women-owned firms, women’s employment and avoiding environmentally unsustainable activities, are monitored closely. The success of demonstration firms, or “champions,” in targeted value chains provides incentives to other firms to emulate their business practices, either as new market entries or in order to maintain their market share in the face of increased competition. The result is to “scale up” the export promotion successes across sectors and regions, thereby contributing to the growth of nontraditional exports from sub-Saharan Africa more generally.3 1 See USTR, “Trade and Investment Policy Toward Sub-Saharan Africa and Implementation of the African Growth and Opportunity Act,” May 2008. 2 An example of this approach may be found in O’Keefe, Thomas, “A Study of Revealed Comparative Advantage for AGOA￾Eligible SADC Countries: Identifying Products Afforded Preferential Access to the U.S. Market Under the African Growth and Opportunity Act,” submitted by Chemonics International, Inc. to the Regional Activity to Promote Regional Integration through Dialogue and Policy Implementation, November 2001. 3 In “From Aid To Trade: Delivering Results; A Cross-Country Evaluation of USAID Trade Capacity Building,” prepared by Molly Hageboeck, MSI, Inc., November 2010, evaluators found that an additional $1 of USAID trade capacity building assistance was associated with a $42 increase in the value of developing country exports from the sectors supported within two years. Africa Trade Hubs Evaluation Report 7 EXPORT PERFORMANCE While U.S. International Trade Commission (USITC) data indicate that petroleum and minerals products continue to dominate overall exports from sub-Saharan Africa to the United States, the share of nontraditional exports (i.e., all others) in relation to total exports has been expanding in recent years (from a low of 7.7 percent in 2007 to 11.0 percent in 2011). Figure 1: Sub-Saharan Africa Exports to the United States in USAID-Supported Sectors Figure 1 demonstrates that, with the exception of the apparel and textiles sector, nontraditional (i.e., non-petroleum and non-mineral) exports from sub-Saharan Africa to the U.S. have increased substantially during the past decade, recovering their upward trend after the Great Recession in 2009 to reach nearly $8 billion in 2011, more than double the levels achieved in 2001. What is not clear from this review of export performance, however, is the extent to which USAID Trade Hub assistance, and in particular its export promotion assistance, influenced this success. This is the principal question addressed by this evaluation. PROJECT CONTEXT AGOA was signed into law on May 18, 2000, as Title I of the Trade and Development Act of 2000. The Act offers tangible incentives for African countries to continue their efforts to open their economies and build free markets. USAID’s Africa Trade Hubs were subsequently first developed to implement the objectives of President George W. Bush’s $70-million, four-year Trade for African Development (TRADE) Initiative (FY 2002–2005), which sought to promote U.S.-African business linkages, enhance the competitiveness of African products and services, expand the role that trade can play in African poverty-reduction strategies, improve the delivery of public services supporting trade, build African capacity for trade policy formulation and implementation and strengthen the enabling environment for $0.0 $2.0 $4.0 $6.0 $8.0 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Sub-Saharan Africa Exports to the United States in Sectors Supported by USAID (in $ billions) Manuf. Goods (other than food and apparel) Food and Food Ingredients Apparel and Textiles Other 8 Africa Trade Hubs Evaluation Report African businesses. To operationalize the TRADE initiative, USAID contracted with implementing partners to establish three regional Trade Hubs in West, East and Central, and Southern Africa. The five year African Global Competitiveness Initiative (AGCI) followed TRADE in July 2005, and was designed to build upon the successes of TRADE. In 2011, the African Competitive Competitiveness and Trade Expansion Initiative (ACTE) was announced as a mechanism to further support African trade capacity building. West Africa Trade Hub The WATH was established in Accra, Ghana, in November 2003 under the direction of USAID/West Africa and was implemented by CARANA Corporation. A new office was added under a separate contract in November 2005 in Dakar, Senegal. Although managed independently, the Dakar office was to work in close collaboration with the WATH in Accra. The Dakar office was assigned the responsibility for export business development work in fish and seafood, and in April 2007 took over work started in Accra in specialty foods. The Dakar office was reduced to a one-person office in 2010 due to lack of funding. The WATH has had several objectives: • Develop and implement private-sector business support strategies for increasing trade under AGOA and with international markets; • Facilitate the effective implementation of customs reform and harmonization; • Support trade-facilitation initiatives and the provision of business services to promote the creation of a more globally competitive West Africa; • Reduce non-tariff barriers to increased intraregional trade; • Encourage greater intraregional and external agricultural trade including the development of sanitary, phytosanitary and food safety standards and practices that comply with import requirements of the U.S. and the EU; and • Increase awareness of West African investment opportunities. The objectives were addressed through four technical components: 1. Business environment; 2. Infrastructure; 3. Enterprise development; and 4. Crosscutting components, including communications and outreach and monitoring and evaluation. Under the business environment component, the WATH worked closely with select export-ready firms to promote exports to U.S. markets on a priority basis, as well as to other international and regional markets when possible. A follow-up WATH project, also based in Accra with a satellite office in Dakar, was initiated in 2007 and also implemented by CARANA Corporation. The new project had four components: trade and investment capacity (business), transportation infrastructure, financial services and business environment. The business component has had seven focus areas: AGOA services; Market Linkages; Cashew and the African Cashew Alliance (ACA); Shea; Home Décor and Fashion Accessories; Apparel; Africa Trade Hubs Evaluation Report 9 and Specialty Foods. While the business component involved working directly with companies in specific value chains, the other components cut across all value chains. Under AGOA services, the WATH provides technical assistance to trade support institutions through a network of 18 AGOA resource centers working with export-ready companies, customs services and governments. Assistance focuses on export procedures and documentation, and at a higher level, sectoral export strategies to help eligible countries take better advantage of AGOA. Using a business-oriented approach, the market linkages component provides direct assistance to suppliers through training and technical support to enhance competitiveness and meet market needs in terms of quality and timing of supplies to buyers. With the WATH assistance, the ACA was founded in 2006 as a business alliance devoted to promoting a globally competitive African cashew industry. ACA provides technical assistance, facilitates investments, promotes standards and market linkages and acts as a platform for sharing information and best practices. The Global Shea Alliance (GSA) was founded in an October 2010 meeting in Ghana, where over 50 key players in the shea industry signed an agreement to form an international industry alliance. GSA was formally inaugurated at the annual international shea conference in Accra in 2011, which was attended by the leading buyers. The WATH attention in the home décor, fashion accessories and apparel sectors focused on strengthening capacity at key points along the value chain and on facilitating collaboration across the region. The WATH assistance placed priority on connecting companies, nationally and regionally, and linking them with experienced export agents, thus enabling groups of companies to fulfill large international orders that no single company could fulfill. In the specialty foods value chain, the WATH assisted public and private organizations to jointly define common regional approaches to overcoming technical and commercial challenges, and to actively develop plans to increase their market shares. Products have included processed cereals, baobab, exotic jams, coffee and moringa. East and Central Africa Trade Hub USAID initiated the ECATH in late 2002 through a one-year sole-source task order to Chemonics International, which set up the Hub in Nairobi, Kenya. Programmed under USAID’s Regional Economic Development Support Office (REDSO) Strategic Objective 5, the ECATH sought to achieve regional food security through building African capacity. However, its major components—1) trade policy capacity building, 2) working with the private sector to increase exports under AGOA and 3) improving the efficiency/reducing the cost of trade-related transportation—closely reflected TRADE objectives. The project also included significant activities at the firm level under its second component, which provided firms with technical assistance related to product design, trade show support, market research education and AGOA rules and regulations. In 2003, USAID/REDSO procured a five-year follow-on project to the Chemonics-run Trade Hub through a contract awarded to a consortium led by Bearing Point. That project included the same components as the earlier Chemonics contract and ended in 2008. Through the AGCI in 2005, the U.S. pledged an additional $200 million in assistance to promote trade in Africa. AGCI sought to expand upon the activities provided under the first two iterations of the ECATH through activities focused on strengthening the enabling environment for businesses in Africa, increasing firm level access to finance, developing regional infrastructure (transportation, energy, communication technology, etc.) and continuing (and expanding upon) the firm level support started under the first iteration of the Trade Hub through the provision of capacity building services related to AGOA, support 10 Africa Trade Hubs Evaluation Report with market linkages (including both trade shows and inward buyer missions) and private-sector partnerships and value addition support to smallholders within a set of targeted agricultural value chains. When the first generation of the ECATH ended in 2008, USAID/East Africa expanded the scope of the Trade Hub project to facilitate coordination between the ECATH and other regional trade and business-support projects. USAID/East Africa awarded a four-year (2009–2013) contract to Chemonics International for an umbrella project that combined the ECATH and the Regional Agricultural Trade Expansion Support (RATES) projects into one more comprehensive project called the Competitiveness and Trade Expansion project (COMPETE). This more holistic approach combined the Mission’s agriculturally led economic growth model with trade facilitation initiatives while reducing the Mission’s management burden. It also worked to align two presidential initiatives—the Initiative to End Hunger in Africa (IEHA) and AGCI—under a single contract. Under the third iteration (the AGCI-supported iteration) of the Trade Hub, the ECATH became a subcomponent of COMPETE and focused exclusively on national- and firm level AGOA promotion activities. The trade policy and transit facilitation work formerly done by the ECATH transitioned to COMPETE, along with RATES’ work supporting strategic agricultural value chains, access to finance (which has never been fully funded under the Trade Hub projects) and infrastructure.4 Today, the COMPETE project has four main goals: 1. Better integration of national and regional markets; 2. Increased trade between the U.S. and ECA; 3. Increased knowledge management and information sharing and 4. Increased capacity of African regional partners. Each of these goals and its corresponding activities is outlined in COMPETE’s 2013 Results Framework, shown in Figure 2.5 This evaluation focuses on Project Intermediate Result (PIR) 2.2—U.S. and ECA firms taking advantage of opportunities under AGOA. Under this PIR, the ECATH works closely with associations and selected export-ready firms to promote exports to U.S. markets on a priority basis and to other international and regional markets when possible. COMPETE works with firms and associations in these sectors: • Textiles and apparel • Cut flowers (floriculture) • Specialty foods • Home décor and fashion accessories • Leather and footwear6 4 DAI/Nathan Group. “African Trade Hub Best Practices Review: Building on Successes and Lessons Learned for the Next Generation of the Trade Hubs,” July 2010. 5 Chemonics International, Inc. “EATH Annual Progress Report, October 2011-September 2012: Competitiveness and Trade Expansion Project,” April 2013. Africa Trade Hubs Evaluation Report 11 Figure 2: ECATH Results Framework Firms are selected to receive ECATH assistance through an application process that assesses the firms’ level of “export readiness,” a largely subjective concept according to ECATH staff. The following criteria are considered in selecting firms: • Product range and quality: Firms must have an in-house designer to receive ECATH support. They also must produce consistently high-quality products. The ECATH employs three U.S.-based consultants who assess the quality of products firms produce. • Human capacity: Firms must have someone who is able to present the firm’s products to clients and competently describe how they are made. 6 This sector was just added in 2012 to support a few Ethiopian firms, as the ECATH saw great potential in this sector given the low cost and abundant supply of leather in Ethiopia and the rapid growth of exports in this sector. The growth was partly due to a recent Ethiopian Government policy that requires Ethiopian leather companies to process leather for exports rather than selling the raw good. The sector is still not an official ECATH-supported sector, and only firms in Ethiopia receive support from the ECATH under this sector. 12 Africa Trade Hubs Evaluation Report • Export history: While firms don’t have to have a history of exporting, the ECATH prefers that firms at least have regional experience with exports. An ECATH staff member said that it is difficult to think a firm could export to the U.S. if it hasn’t exported anywhere before. Nonetheless, according to the ECATH staff, if the firm’s human capacity and product quality are exceptional, a lack of export history can be overlooked. Evaluators also observed this among several—mostly designer-led textile and apparel—firms. • Cost sharing: Firms have to want assistance and be willing to engage in cost-sharing for the ECATH-supported activities such as trade shows. • Commitment: Finally, firms must be committed to increasing their exports, as demonstrated through their willingness to engage in cost sharing, but also through their agreement to continue working with the Trade Hub for at least two to three years. ECATH supports firms directly through the following activities under PIRS 2.2: • International trade shows: the ECATH organizes and coordinates Origin Africa Pavilions at trade shows and provides cost-sharing support for export-ready ECA firms interested in exporting to the U.S. to attend the shows and connect with buyers. A review of the ECATH beneficiary lists (firms who have received assistance under COMPETE since 2009) shows that 59 of the 96 firms (61 percent) receiving assistance received support with attending a trade show. • Direct buyers missions: COMPETE organizes targeted, sector-specific missions composed of motivated U.S. buyers interested in sourcing from Africa. According to evaluator analysis of ECATH-provided beneficiary lists, 42 of the 96 firms (44 percent) receiving support from the ECATH received support through direct inward buyer visits to their firms. • Technical assistance: the ECATH provides product-development technical assistance to firms to adapt their products to the international market and ensure the consistent quality of goods. The Trade Hub has also provided business planning and logistical support to firms, as well as assistance with accessing finance, obtaining certifications and meeting environmental/compliance regulations. ECATH’s list of beneficiary firms shows that 59 of the 96 firms (61 percent) that received assistance since 2009 received some sort of technical assistance. • Origin Africa events: Origin Africa is a brand established by the Trade Hubs to raise awareness and campaign to change perceptions about doing business in Africa. Origin Africa seeks to put Africa on the map as a preferred sourcing destination for textiles and apparel, cut flowers, specialty foods and home décor/fashion accessories. Origin Africa works through buyer awareness outreach campaigns, regional events (including conferences, trade shows and fashion shows, with the goal of building domestic and regional markets for African-produced fashion to set the stage for growth) and international events. In addition to the above, the ECATH supports firms indirectly through Regional Trade Associations, policy advocacy activities and public/private sector support. However, this evaluation, as mentioned, focuses on direct firm level assistance. Southern Africa Trade Hub USAID’s Southern Africa Trade Competitiveness Project (TCP), also referred to as the USAID Trade Hub and implemented by CARANA Corporation from 2004–2010, worked with the private sector to promote exports from key sectors of the Southern African economy to global markets. The project emphasized private-sector, market-led approaches to achieving export competitiveness and regional trade in agriculture, including food security. Africa Trade Hubs Evaluation Report 13 The Southern Africa Trade Hub, based in Gaborone, Botswana, worked to increase the competitiveness of selected industries and value chains, specifically apparel, specialty foods and fresh produce. The Hub assisted 500 firms in the Southern Africa region with advice and information about exporting to the U.S. and laid the groundwork for nearly 2,000 buyer-seller relationships through trade shows, business-to￾business events and buyer engagement. Key elements of the approach included: • Supporting women entrepreneurs • Promoting regional integration • Branding Africa (“Source Africa”) • Increasing competitiveness • Increasing food security • Agricultural training • Facilitating investment • Supporting public-private partnerships When the current Trade Hub contract was awarded to AECOM in October 2010, USAID required the implementing partner to re-focus its efforts on policy, trade facilitation and institutional reform, leaving firm level assistance and value-chain assistance to USAID's bilateral projects. There is residual support of a flagship trade event once a year, and the Trade Hub continues to work within the value chains of grains (groundnuts, maize and soy) and the textiles and apparel sector, but even in the flagship event, the work is focused on systemic changes, institutional reform and alleviating constraints.7 7 It is for this reason that the scope of work for this assignment mandated that the impact portion of the evaluation “...will focus primarily on the activities of the West Africa Trade Hub and the East and Central Africa Trade Hub because their activities have focused heavily on direct assistance to firms for the purpose of improving export performance.” 14 Africa Trade Hubs Evaluation Report IV. EVALUATION METHODS & LIMITATIONS EVALUATION METHODS The SOW for this evaluation suggested the need to use an IE methodology to answer the evaluation questions. However, recognizing that implementation of an IE according to the most rigorous standards may not be feasible, the evaluation team considered other possible quantitative measurement methods during evaluation design, including a performance evaluation of assisted firms before and after they received USAID assistance. These methods and the ultimate selection of designs by the Trade Hub are discussed in more detail below. For all Trade Hubs, however, the evaluation team used a mixed-method evaluation design, coupling qualitative evaluation methods with the aforementioned quantitative methods to enrich and triangulate findings and conclusions. The USAID Evaluation Policy defines an IE as an evaluation that addresses attribution through estimating the counterfactual, or what would have happened in the absence of the project. That is, an IE seeks to identify the impacts that can be linked to a given project rather than attributed to any of a myriad other possible causes, including for example, other donor projects or economic or political changes. This is typically achieved by comparing the changes in outcomes for a “treatment group” that receives USAID support to a “comparison group” that does not. To the extent that the treatment and comparison groups are similar before the project and could be expected to have had similar trajectories of change in the absence of the project, the difference in their outcomes can be attributed to the project. In keeping with standard operating procedures, USAID required the Trade Hub implementing partners to perform baseline surveys of potential beneficiaries prior to commencement of project activities; however, these surveys did not include a control group or a rigorously defined sample frame. Therefore, the only avenue available for the evaluation team to perform an IE in accordance with USAID’s Evaluation Policy was to attempt to reconstruct such a baseline, both for treatment and comparison groups, and to couple this with an endline survey of both groups. Since participating firms were selected based on their perceived export readiness, an experimental evaluation design with random assignment is not feasible. Accordingly, the evaluation team had to consider quasi-experimental designs, which face formidable challenges. The first is, ex post facto, to identify respondents for both the treatment and comparison groups that truly were similar prior to the project, thus reducing selection bias. A second challenge relates to sample size for both groups, especially if the SOW requires analysis of sub-samples from both groups (e.g., to disaggregate the impact causality analysis by value chain or country, or other parameters such as gender of employees).8 A third challenge relating to the absence 8 In East and Central Africa, for example, the entire population of firms assisted by the current Trade Hub implementing partner was only 96 firms from all the countries in that region. Africa Trade Hubs Evaluation Report 15 or weakness of baseline data is the potential for recall bias; that is, respondents may have either faulty or selective memory of data from periods that may be a decade or more in the past. EVALUATION SCOPE For this evaluation, the evaluation team determined that despite the lack of any baseline surveys on either beneficiaries or non-beneficiaries, it would be possible to move forward with an IE utilizing a quasi-experimental design for evaluating Trade Hub performance in West Africa but not in the other regions. One reason an IE was deemed possible in West Africa was that the Trade Hub implementing partner maintained data from application forms that firms filled in when they were first contacted. The WATH implementing partner maintained the original application forms (called Export Readiness Questionnaires [ERQ]) from each of the firms that applied to work with it, and those forms included information about the firm’s sales, exports, employment and destination of exports at the time of the application. This meant that there would be a list not only of firms that worked with the Trade Hub and received the benefit of export promotion technical assistance and training, but also of those that had not (a pool of potential comparison firms). Moreover, the evaluation team believed that data from these application forms might be utilized to cross-check recalled baseline information obtained from surveying the firms, thereby reducing the potential for recall bias.9 It was also the case that in West Africa, the Trade Hub implementing partner worked with a relatively large number of firms, and that a large proportion of these, representing all of the value chains of interest to USAID, were located in and around Accra. Therefore, it would be possible through site visits to identify and survey in depth relatively large samples of USAID beneficiaries and non-beneficiaries in and around Accra representing all the sectors to which USAID provided export promotion assistance. Unfortunately, selection of the “treatment” group members by the implementer was not random, so that a selection bias may have existed in favor of offering treatment to firms that were deemed by CARANA to have stronger prospects for success (see Picking Winners below). In contrast, in both ECA and Southern Africa, the original implementing partners were not selected for the second round of Trade Hubs contracts, and accordingly any data maintained on firms that applied for support were no longer accessible.10 The implementing partner for the ECATH had been in operation since 2009. Partly due to the more restricted period of implementing partner coverage, the number of firms that the Trade Hub worked with was relatively small, especially outside the apparel and textiles sector, and there was no large concentration of any firms in any given country of the region. There were no baseline data on firms 9 The field surveyors were given copies of the information originally supplied in intake questionnaires and instructed to use that information to cross-check the responses given during the field survey. The evaluation team also endeavored to enlarge the comparison group by surveying firms with a comparable profile that had never been in contact with the West Africa Trade Hub and were willing to be included in the evaluation survey, an exercise that yielded only seven firms in addition to those that had completed initial ERQs. 10 USAID’s standard operating procedures do not include maintaining detailed information compiled by USAID implementing partners, such as baseline surveys, performance management data or endline surveys, once their contracts end. 16 Africa Trade Hubs Evaluation Report assisted in any of the countries and the implementing partner had not maintained a list of firms that had applied to work with the implementing partner but did not receive assistance. In other words, there was no identifiable comparison group and no data on non-beneficiaries in baseline periods that could be used to cross-check survey results from either beneficiaries or non-beneficiaries to guard against recall bias. Based on these data limitations, the evaluation team determined that it would not be possible to perform an IE using a quasi-experimental evaluation design methodology in ECA and advised USAID accordingly. Instead, it was determined that evaluation work in that region would be confined to a performance evaluation using a firm level telephone survey of beneficiaries to gather data for analysis of changes in firm level performance before and after each firm had received export promotion assistance. This was supplemented by interviews of key informants and beneficiaries in Kenya and Ethiopia. In Southern Africa, the Trade Hub implementing partner had been in place since 2010, but received instructions from USAID in 2012 to re-focus efforts away from firm level assistance and toward work with associations, leaving firm level export promotion work to USAID bilateral missions in the region. In the absence of either baseline or endline data on firms assisted, and without the possibility to conduct either firm level surveys or interviews, the evaluation team determined in consultation with USAID that the most that could be achieved would be to review trends in export performance in that region. Details on the methodology undertaken are included in the discussion of each Trade Hub. EVALUATION LIMITATIONS The lack of baseline information and a previously defined and measured comparison group of firms, as described above, led to several evaluation limitations, as did the limited resources available for this evaluation. Additionally, similar to many other development initiatives, the Trade Hubs were not designed to demonstrate that their interventions were the sole source of documented changes in outcomes. The Trade Hubs’ firm level support activities were not designed to work in isolation, but in combination with a variety of other components (such as financial services, road governance and the business environment). The Trade Hub projects also have worked in close collaboration with a variety of other USAID and non-USAID development initiatives to achieve joint results.11 This fact, paired with the specific limitations discussed below, complicated the ability of the evaluation team to produce causal evidence. The following section discusses the limitations that contributed to concerns regarding the validity of the findings presented in this evaluation. Whenever possible, methods used to reduce or mitigate these concerns are also presented. For information on ways that these limitations might be avoided in future Trade Hub evaluations, see Annex VI. 11 For these reasons the contributions of the Trade Hubs to the overall results achieved by the firms that they have assisted should be regarded as contributions, not as evidence that the Trade Hubs’ firm level project interventions were the only cause of the results achieved. Africa Trade Hubs Evaluation Report 17 1. West Africa Trade Hub IE Recall Bias Recall bias, which results from having to ask respondents for information from a period many years in the past, is inevitable when baseline information has not been collected. The evaluation team planned to mitigate this potential bias by cross-checking the responses of both treatment and comparison group respondents against the information they had provided in their original application forms to the Trade Hub (ERQs). Unfortunately, because a significant amount of information was missing in those questionnaires, the evaluation could not effectively compare the ERQ responses against the survey responses, and so the issue of recall bias could not be eliminated.12 Nonetheless, there is no evidence that treatment group firms were subject to greater recall bias than were comparison group firms, reducing this concern. Selection Bias For a strong IE, the comparison group should have the same characteristics as the treatment group, with the treatment being the only differentiating factor. When this relationship does not hold, project estimates may be biased because the results will not reflect the true impact of the project since they also incorporate the effect of those other differences. Two potential sources of selection bias were identified for this IE: • Picking winners: Inherent in the USAID development hypothesis is the concept of “picking winners,” or those firms most likely to succeed in exporting, meaning that comparison group firms might have been rejected by the Trade Hub because they were perceived as less likely to succeed.13 This is further evidenced by the fact that the evaluation team found that more members of the comparison group than of the treatment group had zero exports in their baseline year. Moreover, although partly counterbalanced by a few very large exporters in the comparison group, median exports in the baseline survey for the comparison group were systematically smaller than those for the treatment group.14 The evaluation team attempted to address this through differences in differences comparisons,15 but it is possible that this selection bias could still affect results. • Failure rates: Due to the fact that the survey (which captured both baseline and endline results) was only administered to surviving firms (i.e., still in existence at the time of the survey), no 12 For example, 60 percent of responses were missing in the ERQs on the percentage of sales exported and 54 percent were missing for the male-to-female employee ratio. Critically, the date of application was not recorded, so the team could not identify whether the survey recall period matched the timing of the ERQ. 13 The evaluation team hastens to add that there is no suggestion that this development model is in error, because of course it makes sense to attempt to identify and work with those firms that are most likely to succeed, and which can serve as “industry champions” whose eventual successes other firms may emulate. 14 It is also worthwhile noting that ‘picking winners’ was an explicit part of USAID’s development model, and so this finding supports a conclusion that the export promotion project was implemented successfully. 15 In differences comparisons, the change (over time) for the treatment group is compared against the change in the comparison group. Accordingly, differences in initial levels should be canceled out, unless these differences would lead to different rates of change independent of the project. 18 Africa Trade Hubs Evaluation Report information could be collected on firms that went out of business. The failure rate for new enterprises in the United States is around two thirds over a 10-year period, according to the Small Business Administration.16 If a similar trend holds in and around Accra, selection bias could occur if it were the case that the firms in either group were larger, better established and more likely to survive than in the other group. However, in the samples collected, there were no statistically significant differences between the two groups in terms of firm size (measured by profiles of number of employees) or the median year in which firms were founded, which mitigated this concern. Different Baseline Years Causal analysis of treatment and comparison groups assumes that both have the same baseline and endline years or that the baseline and endline years are randomly distributed in both groups, so that they have broadly similar exposure to external influences. Because the period covered by the survey analysis was more than a decade, it is inevitable that some firms would either enter the market or begin working with the WATH, or both, in years following 2001. Indeed, the baseline years of firms in the treatment and comparison groups did not appear to be randomly distributed, with many more comparison group firms reporting their baseline year as 2001 than in the treatment group. These differences in baseline year introduce an additional potential source of bias in the difference-in￾differences approach, because exogenous factors, like the derogation of the Multi-Fibre Arrangement in 2005, or the global recession in 2009, could skew the analysis if there were more firms in the comparison group than in the comparison group with baselines in those years. Spillover Effects Since the amount of support that USAID can provide to individual firms within a sector or value chain is inevitably small in comparison to the sectors themselves, USAID’s firm level assistance is intended to produce a “scaling-up” effect among firms not receiving direct assistance, as they begin to emulate the best practices of successful firms that have received direct assistance. While this is an explicit and positive aspect of the development hypothesis, such “spillover effects” may confound an IE attempting to isolate the impact of USAID firm level assistance by focusing on the performance of directly assisted firms as compared with unassisted firms. The evaluation team determined that to the extent such effects existed, they would tend to reduce the differences between the comparison and treatment group firms, biasing causality tests toward inconclusiveness. In order to better understand the potential for—and extent of—spillover effects, the team asked survey respondents a series of questions associated with changes in the business climate. As described in Table 1, the team found that most firms reported an increase in the number of competitor firms, input-supply firms and buyers. These numbers support the possibility that Trade Hub activities have had spillover effects. 16 See http://www.sba.gov/sites/default/files/sbfaq.pdf Africa Trade Hubs Evaluation Report 19 Table 1: Potential Spillover Effects of WATH Assistance In the past several years, have you noticed a… No Change Increase Decrease Change in the number of Ghanaian companies selling products like yours / you must compete with? 9% 74% 16% Change in the number of companies that supply inputs to firms like yours? 16% 60% 20% Change in the number of companies that offer transport services to firms like yours? 36% 39% 9% Change in the number of buyers of your product? 11% 57% 31% Other Donor Assistance Firms in either the treatment or comparison groups might also have received export promotion assistance from entities other than the WATH, complicating the analysis of the impact of USAID Trade Hub assistance. In fact, firms in both groups reported receiving assistance from other entities (see Table 2), but comparison group firms—in addition to receiving no USAID export promotion assistance—also reported receiving less non-USAID export promotion assistance than did treatment group firms, by a statistically significant margin. Based on this finding, if the treatment group experienced greater success in exporting than the comparison group, it would be fair to conclude that it was the combination of assistance received from different sources, and not only that received from USAID’s Trade Hub, that was the cause. Table 2: Surveyed Firms That Have Received Assistance from Sources Other than WATH Source of Assistance Treatment Firms Comparison Firms International buyers 30% 16% State-owned firms, incl. banks 19% 9% Government agencies 45% 32% Private Firms 48% 39% Multilateral organizations 7% 2% NGOs 19% 18% Friends and family 49% 43% 20 Africa Trade Hubs Evaluation Report Small Sample Size While evaluators attempted to survey all firms in and around Accra that had ever been in contact with the WATH, plus additional firms identified for inclusion in the comparison group by the Ghana Export Promotion Authority and the various trade associations dealing with the WATH value chains, the total list thus compiled contained 208 firms, of which slightly less than two thirds were successfully surveyed through on-site visits. The survey yielded 84 treatment group firms and 44 comparison group firms, a sample size that, given the requirement to disaggregate the statistical analysis across the value chains of interest to the Trade Hub, was relatively small for purposes of complex statistical analysis. 1. East and Central Africa Performance Evaluation Recall Bias While the ECATH provided data on attributed exports, as described above, the definition of “attributed exports” could vary from firm to firm. Additionally, there could be additional indirect effects of Trade Hub support that firms did not recognize as being a result of the ECATH support through this method of measurement. An example of this would be if a firm had received technical assistance that allowed the firm to increase the quality of its products or marketing, which may or may not have led to additional exports. As such, the evaluation team determined that it would be necessary to collect data from firms on total exports (among other indicators) to determine the apparent effects of ECATH support. Since no baseline data or data for subsequent years were available for the indicators under consideration, the evaluation team had to generate those data through interviews with survey respondents. As with the firm level survey in West Africa, the telephone survey of beneficiary firms in ECA asked respondents to recall data from as many as 11 years ago on sales, employment, export levels and export destinations, introducing the potential for recall bias into the analysis. Unfortunately, there were no objectively verifiable data against which to cross-check their responses, but evaluators did try to mitigate the potential for recall bias by collecting data only on specific years associated with key events (such as a firm opening its doors, beginning to work with the Trade Hub and the last year of assistance—all of which should be easy to remember and, thus, less prone to subjectivity). Selection Bias Two potential sources of selection bias were identified: 1. Self-selection: There is likely some selection bias in the quantitative survey sample since firms essentially self-selected into the sample (by choosing to take part in the telephone survey; some firms, on the other hand, refused to participate). Self-selection often results in survey respondents who are either very pleased with or very upset about the support they have received. 2. Failure rates: Only firms still in operation replied to the telephone survey, prompting the concern that they would represent only the most successful of firms that had received USAID support. However, since the number of respondents surpassed non-respondents, and also since evaluators found mixed results with outcomes (potentially eliminating the concern that only the most or least successful firms responded to the survey), evaluators determined that selection bias is likely not a critical issue. Outlier Years Comparing firm level results from Period 1 (the period before receiving USAID Trade Hub assistance) with Period 2 (the period after receiving it) might not be fair if the years used to calculate these growth figures were not typical years. For instance, a firm could have experienced a dramatic drop in sales one Africa Trade Hubs Evaluation Report 21 year due to drought or a spike in oil prices. If this were the case, evaluators might be over- or under￾estimating average growth. Since most firms could not provide detailed annual sales and export figures for multiple years over time, the evaluation had to rely on their recall from some key years. However, evaluators tested 2012 results to ensure outcomes in that year were consistent with outcomes in the previous two years (2010 and 2011), and evaluators found that the outcomes were consistent, which helped to alleviate concerns, at least regarding endline data. Small Sample Size While evaluators were able to track down more than 60 percent of the beneficiary population of firms supported by the ECATH since 2009, the total population of beneficiary firms in the ECATH since 2009 is small. Thus, when results were disaggregated by country and sector, evaluators sometimes found that they had fewer than five firms in any one group (i.e., disaggregated by either sector or country), which makes it impossible to identify statistically significant results. Other Donor Support As is also true with the WATH, ECATH interventions were not provided in isolation. Many firms that received ECATH support also received support from other donor organizations and NGOs. Thus, just because a firm appears to have increased its sales and exports between Period 1 and Period 2 does not mean that the Trade Hub had anything to do with this change. It is quite possible that differential outcomes in Period 2 compared to Period 1 were affected by other donor organizations and NGOs. In fact, 10 out of 11 firms (91 percent) interviewed through key-informant interviews (KIIs) reported having received assistance from at least one other donor organization or NGO, sometimes with the same goal of helping firms to increase exports. Existence of Many Zeros Because of the years for which data was collected (most specifically, the year that the firms began operating and the year that the firms began working with the Trade Hub), the survey dataset contains several zero values for sales and exports. The result is that it was not always possible to calculate the compounded annual growth rate (CAGR) of outcomes for each firm. Therefore, analysis under this PE using this method often relies on a smaller number of firms than the total sample size of 61. The total number of firms represented in each analysis is provided throughout the Findings section of the report to make this clear. Since there are many zeros, the evaluation team instead considered relying on the average absolute annual growth rate of outcomes for each firm. However, this method can result in outcomes largely driven by only a few outlier firms that are either very large or that saw significant growth/declines. As such, the team presents both types of analysis in this report to allow readers to better understand the intricacies of the data. Unrepresentative Qualitative Sample The ECATH provides assistance to firms from six countries: Kenya, Ethiopia, Mauritius, Uganda, Rwanda and Tanzania. While the evaluation team was able to collect quantitative phone-survey data from firms in each of these countries, due to resource constraints the team was only able to conduct in-person qualitative interviews in Kenya and Ethiopia, which were focused in the capitals of each country. 22 Africa Trade Hubs Evaluation Report Fortunately, Kenyan and Ethiopian firms account for nearly 50 percent of all firms supported by the ECATH since 2009. Although the phone survey included respondents from each country, it is possible that firms from the other countries may have reported different perceptions of Trade Hub assistance, and it is important to take this into consideration when viewing the qualitative results. Similarly, the evaluation team was unable to interview any floriculture firms, due to the fact that they are usually located outside of the capital cities.17 This value chain is therefore not represented in the qualitative analysis. Finally, the evaluation team was only able to interview one specialty food company, and though this number is representative of the population (given the sample size), it is possible that this one firm is an industry outlier.18 Lack of Data on Firms Supported under Previous Iterations of the Trade Hubs The evaluation team was unable to contact firms that were supported under the previous iteration of the Trade Hub but that were no longer receiving ECATH support because the ECATH did not have records for these firms. While evaluators were able to obtain a list of the firms supported by Bearing Point (from 2003–2008), data collectors found it difficult to obtain contact information for the firms. 2. Additional Limitations Trade Trends Assessment The evaluation team experienced difficulty assessing the performance of firms assisted by the Trade Hubs against trade trends in each region because Trade Hubs had not collected longitudinal data on the actual domestic sales, exports or employment of their clients. The evaluation team found that, instead, Trade Hub implementing partners were expected to report only on results “attributable” to USAID assistance. The premise was that the clients themselves would be best placed to assess how much of their exports, employment, etc. was attributable to USAID assistance, and that they would report the data as such. Unfortunately, “attributed” exports or employment were not concepts that could be compared with actual exports or employment, and may have either greatly over- or underestimated actual exports and employment.19 Even with recall surveys, the evaluation team was unable to recoup actual exports and employment data from the all the implementing partners’ clients, or even from all those still surviving and working with the implementing partner. The evaluation team felt that comparison of baseline and endline export “trends,” derived from the evaluation’s firm level surveys, with national or regional export trends, would be inappropriate for the following reasons. In West Africa, the evaluation survey was confined to firms in and around Accra, and these may or may not have been representative of assisted firms in the West Africa region as a whole.20 In addition, the concern that only surviving firms willing to respond were surveyed by the evaluation 17 The team did capture information on floriculture firms in its quantitative survey, however. 18 The quantitative survey did not include any footwear companies, as these firms have only been receiving support for a very short period of time. Thus, it is unlikely that the team would have been able to identify impact at this early stage. 19 A correspondent from the WATH, for example, believed that some firms were only attributing exports in the years in which they received direct assistance from USAID, thereafter reporting zero “attributed” exports. 20 They do, however, appear to be representative of the performance of Ghanaian firms, as outlined in “Summary Findings and Conclusions.” Africa Trade Hubs Evaluation Report 23 team in either region means that survey trends observed among those firms would be inherently biased in comparison with national or regional average trends. Cost-Effectiveness Assessment For all three regions, the evaluation team proposed to compare the results that Trade Hub implementing partners attributed to firm level export promotion assistance—in terms of increases in sales, exports and employment—with the cost of achieving those results. The intention was to make these comparisons in the aggregate, and also by value chain and country, to see whether assistance to certain value chains or countries was more effective. However, with the exception of data provided by the WATH for the period 2010–2012, it was not possible to obtain estimates of attributed results from the Trade Hubs by value chain or by country. Given that USAID’s Trade Hub implementing partners were not required to collect and report data on either expenditures or results by value chain and by country supported, they did not do so. It should be noted that, when the data were requested, all three implementing partners were able to estimate expenditures by value chain for each of the previous years of their contract operations without too much difficulty, and some were able to allocate those expenditures by country. The difficulty was that, having never tracked the exports or employment of their clients by value chain or country, they found it impossible to reconstruct these data. Nonetheless, it was possible, through reference to their annual reports, to obtain estimates of the increases in total exports that each implementing partner reported to have produced through USAID Trade Hub operations, and these were used to obtain rough estimates of total exports promoted per USAID dollar spent in each region. In West Africa, it was also possible to obtain rough estimates of attributable increases in exports per USAID dollar by value chain, based on information provided by the implementing partner. If USAID wishes to conduct more rigorous cost-effectiveness analysis in the future, it should follow the recommendations provided in Annex VI related to tracking costs at the project level and outcomes at the firm level. 24 Africa Trade Hubs Evaluation Report V. FINDINGS, CONCLUSIONS & RECOMMENDATIONS This section presents findings and conclusions, subject to the caveats presented above about data sources and limitations. These are followed by a summary of integrated findings and conclusions according to the evaluation questions contained in the assignment scope of work. WEST AFRICA TRADE HUB IMPACT EVALUATION 1. Methodology This evaluation utilized a firm level survey to compare a group of firms that received WATH technical assistance and a group of firms that did not receive such technical assistance. Since we will compare the change from baseline to follow-up in the treatment group versus the comparison group, we must collect data from both groups at two points in time (see Figure 3). Figure 3: Schematic of Impact Evaluation Due to resource and time constraints, the survey focused on Accra, Ghana. As shown in Table 3, Ghana has been a major focus of the WATH interventions. During the period 2007–2011, Ghana accounted for one half of the project’s value of exports facilitated and one quarter of intraregional trade facilitated. Similarly, Ghana’s share of companies that received WATH technical assistance is 28 percent (31 percent for women). The country’s share in other areas is also sizable: 20 percent of trade and investment capacity training conducted, 11 percent of value of investment facilitated and 8 percent of jobs created. Africa Trade Hubs Evaluation Report 25 Table 3: Selected West Africa Trade Hub Results in Ghana and Other Countries, 2007–201121 Results All Countries Ghana % of Total Companies that have received technical assistance Numbers Dollars Numbers Dollars Total 3,244 918 28% Women-owned 1,027 321 31% Value of exports facilitated $178,491,502 $90,907,553 51% Value of intraregional trade facilitated $4,073,431 $1,074,270 26% Value of investment facilitated $53,776,057 $6,183,923 11% Jobs created Total 9,957 1,835 18% Women-owned 7,137 960 13% Trade and investment capacity training Total 8,933 1,762 20% Women-owned 2,601 570 22% Selection of Treatment and Comparison Group Firms The list of treatment firms was compiled from the WATH 2006–2011 exporter directories, which consisted of all Accra-based export firms that benefited from WATH assistance. The comparison group consisted of all firms that had completed an ERQ but that were not included in the exporter directories because they had not received any assistance from the WATH. The two lists were shared with the WATH for validation. Following consultations with the Ghana Export Promotion Authority and the various trade associations dealing with the WATH value chain products to identify other similar firms who did not receive WATH assistance, seven firms were added to the comparison group.22 Thus, the final list of treatment and comparison groups represented nearly the entire population for both groups of firms based in Accra, not only a sample of those firms, plus some that had never been in contact with the Trade Hub. 21 Source: WATH Final Report, FY 2008–2011 22 As mentioned in Section IV, the evaluation team endeavored to enlarge the comparison group by surveying firms with a comparable profile that had never been in contact with the West Africa Trade Hub, and were willing to be included in the evaluation survey; in the event only seven such firms could be identified and surveyed. 26 Africa Trade Hubs Evaluation Report Number of Firms Interviewed and Interview Process The total list consisted of 204 firms. About 38 percent of the firms (77 firms) could not be interviewed for the following reasons: incorrect contact information; the firm did not respond to repeated phone calls and e-mail messages; the firm went out of business; the firm was not willing to participate in the survey; and the main contact was out of the country. The final distribution between the two groups was 84 for the treatment group and 44 for the comparison group—for a total of about 62 percent of the initial list of firms identified. A letter of introduction was sent to all firms to be interviewed before the interviews were scheduled. Thorough training was provided to the enumerators and their supervisors, and a pilot survey was conducted with eight firms before finalizing the survey questionnaire. The full survey was conducted using the final questionnaire (see Annex III) over a five-week period in March and April 2013. Baseline and Endline Data In the absence of complete baseline data from secondary sources, particularly for the early years (2002– 2007), surveyed firms were asked to provide data on sales, exports and full- and part-time staff in the year before they started receiving WATH assistance. For the comparison group, surveyed firms were asked to provide data on sales, exports, full- and part-time staff in 2001, or the year that the firm first began operations if it was after 2001. Both categories of firms were asked to provide information on their sales and exports in the past three years (2010–2012) and the number of current employees. Other Data In addition to reviewing project documents and monitoring and evaluation data, a series of interviews was conducted with selected project staff, beneficiaries and development partners such as the World Bank and the African Cashew Initiative. Qualitative information collected through interviews was a critical supplement to the survey data. Although the views gathered during those interviews could not be generalized, they were critical in understanding project context and the mechanisms through which beneficiaries had been assisted. 2. West Africa Regional Export Trends Trends in West African exports to the United States in sectors that have received support from the West Africa Trade Hub are shown in Figure 4.23 Nontraditional export sectors (i.e., non-petroleum and non-mineral) increased from $838 million in 2001 to $2.7 billion in 2011. Noteworthy in this respect is that the apparel and textiles sector has played a relatively insignificant role in exports to the United States, whereas food and food ingredients, as well as manufactured and other goods (including handicrafts) have all seen substantial increases during the past decade. 23 Source: USITC data and KSA analysis. Africa Trade Hubs Evaluation Report 27 Figure 4: West African Exports to the United States in USAID-Supported Sectors 3. Impact Evaluation Findings Sales and Exports As shown in Table 4 and Figure 5, sales increased from baseline to endline for both groups of firms, but the increase was nearly twice as large for the treatment relative to the comparison group. The difference was substantially higher in export performance. While exports nearly tripled for assisted firms, they decreased by about 70 percent for unassisted firms. Similarly, while exports’ share of total sales more than doubled for assisted firms, it declined by about 70 percent for the unassisted firms. $- $200 $400 $600 $800 $1,000 $1,200 $1,400 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 West African Exports to the United States in Sectors Supported by USAID (in $ millions) Apparel and Textiles Food and Food Ingredients Manuf. Goods (other than food and apparel) Other Finding 1: USAID-assisted firms have performed better than unassisted firms, both in total sales and exports. 28 Africa Trade Hubs Evaluation Report Table 4: Average Sales and Exports at Baseline and Endline for Treatment and Comparison Firms24 Total Sales and Exports Treatment Firms Comparison Firms Sales Baseline ($‘000) $19,193 $3,805 Endline ($‘000) $59,903 $8,083 Change (%) 212% 112% Exports Baseline ($‘000) $16,751 $828 Endline ($‘000) $48,534 $256 Change (%) 190% -69% Exports as a Share of Total Sales Baseline (%) 87% 22% Endline (%) 81% 3% Change (%) –7% –87% Figure 5: Percent Change in Sales and Exports in Assisted vs. Non-Assisted Firms 24 Figures rounded to the nearest decimal. Source: Evaluation Survey -100 0 100 200 300 Sales Exports Percent Change in Sales and Exports in Assisted vs. Unassisted Firms Assisted Firms Unassisted Africa Trade Hubs Evaluation Report 29 Table 5 shows that the performance of exports was better for assisted firms than for unassisted firms. For the value chains in which exports actually decreased (apparel, textiles, cashew and shea), the decrease was smaller for assisted firms than for unassisted firms. In those sectors in which exports increased (specialty foods), the increase was greater for assisted firms than for unassisted firms. Table 5 also shows that most of the decline in exports occurred in apparel and textiles (65 percent and 66 percent, respectively), reflecting the trend in the country’s overall exports in the two value chains. Table 5: Exports at Baseline and Endline for Treatment and Comparison Firms, by Value Chain ($’000)25 Value Chain Treatment Firms Comparison Firms Baseline Endline % Change Baseline Endline % Change Apparel 2,349 818 –65% 1,586 16 –99% Cashew 284,800 256,900 –10% 25,228 0.5 –98% General 18 144 663% 89, 149 67% Home décor and fashion accessories 823 5,500 568% 30 197 541% Shea 16,589 15,225 –8% 1,567 3 –99% Specialty Foods 682,634 2,582,631 278% 4 817 20,300% Textiles 389,473 133,976 –66% 5,275 1,383 –74% These findings conform to data from the International Trade Commission, which show that Ghana’s apparel and textile exports to the United States declined from $8 million in 2004 to an average of $1.5 million in 2010–2011. As shown in Table 6, apparel and textiles declined following their peak in 2006, just after the end of the Multi-Fibre Arrangement regime that previously had offered African exporters preferential access to the U.S. market. For many food categories in the food and food ingredients sector (including specialty foods), exports for both treatment and comparison firms increased fairly dramatically. However, for both cashew and shea processors, USAID assistance to respective industry “alliances” seems to have 25 Figures rounded to the nearest decimal. Source: Evaluation Survey Finding 2: Export performance for USAID-assisted firms was better than for unassisted firms, in all value chains. 30 Africa Trade Hubs Evaluation Report Table 6: Ghanaian Exports (in US$ million) to the United States by Sector Ghana 2001 ‘02 ‘03 ‘04 ‘05 ‘06 ‘07 ‘08 ‘09 ‘10 ‘11 Apparel and Textiles 0 0 4 8 5 9 8 1 0 1 2 Food and Food Ingredients 55 27 10 22 33 73 58 34 97 191 281 Manufactured Goods (other than food and apparel) 0 2 1 0 1 1 0 5 1 5 20 Minerals, Metals and Ores (All) 31 20 3 5 9 6 1 11 9 24 6 Oil and Gas 56 25 29 57 58 49 80 125 2 22 441 Other 37 39 33 44 49 50 49 44 20 27 27 Ghana Total 179 113 80 136 155 188 196 220 129 270 777 been only successful in allowing the treatment firms to survive, while the exports of comparison firms collapsed entirely. Similar firm level survey results are found in the number of export market destinations for the two value chains (see Table 7). The number of export market destinations for the two sectors declined for both the treatment and comparison groups. The decline for the treatment group was larger than that of the comparison group (more than 31 percent versus 12 percent for apparel, and nearly 19 percent versus 15 percent for textiles). Export Diversification The evaluation team measured export diversification by assessing the number of export markets as well as by the value of exports to those markets. As shown in Table 7, the average number of export markets for assisted firms increased for three of the value chains (cashew, shea and specialty foods), remained unchanged for one (fish) and decreased for four, including apparel and textiles, where the decrease reflected the overall trend of declining exports since 2007 (see Table 6). In contrast, the average number of export markets for unassisted firms decreased for all value chains. Finding 3: Export diversification was higher for assisted firms than for unassisted firms in terms of both the average number and size of export market destinations. Africa Trade Hubs Evaluation Report 31 Table 7: Average Number of Export Market Destinations at Baseline and Endline for Treatment and Comparison Firms, by Value Chain26 Value Chain Treatment Firms Comparison Firms Baseline Endline % Change Baseline Endline % Change Apparel 1.21 0.83 –31% 1.55 1.37 –12% Cashew 1.87 2.11 13% 1.45 1.18 –19% Fish 3.00 3.00 0.0% 1.45 1.22 –16% General 2.33 1.67 –28% 1.43 1.22 –15% Home décor and fashion accessories 2.07 1.40 –32% 1.40 1.22 –13% Shea 1.86 1.95 0.5% 1.40 1.10 –21% Specialty Foods 1.00 1.36 36% 1.52 1.23 –19% Textiles 2.10 1.70 –19% 1.42 1.21 –15% Table 8 shows that the average size of all export markets increased more rapidly for assisted firms than for unassisted firms. Although exports to Southern Africa were low for both the treatment and comparison firms, the average export to that market was negligible for unassisted firms. It is worth noting that the rate of increase in the average size of the export markets for assisted firms was highest for West Africa (over 700 percent), followed by the U.S. (135 percent) and the EU (85 percent). The U.S. export market was the largest destination for assisted and unassisted firms alike. Figure 6 illustrates these findings. 26 Figures rounded to the nearest decimal. Source: Evaluation Survey 32 Africa Trade Hubs Evaluation Report Table 8: Average Size of Major Export Market, by Destination, at Baseline and Endline for Treatment and Comparison Firms, by Value Chain ($’000)27 Market Destination Treatment Firms Comparison Firms Baseline Endline % Change Baseline Endline % Change U.S. $426,988 $1,001,779 135% $631,824 $1,127,000 78% EU $523,459 $970,944 85% $5,413 $353 –93% West Africa $53,175 $440,088 728% $612,000 $15,000 –98% Southern Africa $75,143 $103,634 38% -- -- -- East Africa $140 $156 11% -- -- -- Figure 6: Export Growth by Market Destination 27 Figures rounded to the nearest decimal. Source: Evaluation Survey -200 -100 0 100 200 300 400 500 600 700 800 Assisted Firms Unassisted Firms Export Growth by Market Destination West Africa U.S. E.U. Africa Trade Hubs Evaluation Report 33 Table 9: Average Number of Workers at Baseline and Endline for Treatment and Comparison Firms28 Item Treatment Firms Comparison Firms Average Number of Workers Baseline Endline % Change Baseline Endline % Change Total 53 48 –11% 65 70 8% Women 27 20 –27% 23 27 18% Job Creation Table 9 shows that while the average number of workers decreased for assisted firms (by 11 percent overall and 27 percent for women), it increased for unassisted firms (by 8 percent overall and 18 percent for women) between the baseline and endline periods. Job gains for assisted relative to unassisted firms in certain value chains (cashew, fish and shea) were more than offset by higher job losses in apparel and textiles (Table 10). At baseline, the average number of workers for apparel and textiles was 79 in the treatment group (versus 9 for the comparison group). The magnitude of the decline in employment in the two sectors was also higher for the treatment group than for the comparison group (46 percent versus 31 percent). As apparent in Table 11, this decline was more substantial for women (over 45 percent for both apparel and textiles) because more women are active in apparel and textiles production. It should, however, be noted that on average job creation increased more for assisted firms (34 percent) than for unassisted firms (8 percent) when apparel and textiles are excluded from the analysis. This pattern may be explained by the fact that the apparel and textile industry in West Africa (and throughout the continent) has encountered significant difficulties since 2005. Between 2005 and 2011, the value of global apparel and textile exports rose by nearly 50 percent. However, the top 10 developing-country suppliers now account for nearly 60 percent, with China accounting for 37 percent of that share in 2011.29 When the World Trade Organization agreement on Textiles and Clothing expired in January 2005, global apparel and textile trade was no longer subject to quantitative restrictions. Similar trade policies have also affected value chain dynamics. For instance, Chile, Panama, Egypt and Turkey have signed free trade and trade promotion agreements or partnership and cooperation agreements with the U.S. and the EU. 28 Source: Evaluation Survey 29 World Trade Organization. 2013. “Aid for Trade and Value Chains in Textiles and Apparel.” Geneva, Switzerland. Finding 4: On average, job creation decreased for assisted firms and increased for unassisted firms. 34 Africa Trade Hubs Evaluation Report Table 10: Average Number of Total Workers at Baseline and Endline for Treatment and Comparison Firms, by Value Chain30 Value Chain Treatment Firms Comparison Firms Baseline Endline % Change Baseline Endline % Change Apparel 120 73 –39% 20 17 –16% Cashew 20 23 17% 450 458 2% Fish 8 24 220% 42 47 11% General 21 15 –26% 12 19 57% Home décor and fashion accessories 24 31 31% 36 62 71% Shea 94 139 47% 15 17 10 % Specialty Foods 26 26 0% 13 9 –27% Textiles 120 73 –39% 20 17 –16% These new market access conditions and their impact on global market competition have been accompanied by sharp export declines from Africa despite continued AGOA support. Constraints to exports from Africa include availability and cost of key services such as transportation, skilled labor and a stable business climate. For instance, it has been shown that as firms grow larger and more productive, their labor costs increase faster in Africa than elsewhere. 31 For instance, a firm level study has demonstrated that production costs in the garment industry are significantly higher in Kenya than in Bangladesh.32 30 Figures rounded to the nearest decimal. Source: Evaluation Survey 31 Gelb, Alan, Christian Meyer, and Vijaya Ramachandran (2013), “Does Poor Mean Cheap? A Comparative Look at Africa’s Industrial Labor Costs.” CGD Working Paper 325: Center for Global Development. 32 Fukunishi, Takahiro; Mayumi Murayama; Tatsufumi Yamagata and Akio Nishiura (2006), “Industrialization and Poverty Alleviation: Pro-Poor Industrialization Strategies Revisited.” United Nations Industrial Development Organization, Vienna. Africa Trade Hubs Evaluation Report 35 Table 11: Average Number of Women Workers at Baseline and Endline for Treatment and Comparison Firms, by Value Chain33 Value Chain Treatment Firms Comparison Firms Baseline Endline % Change Baseline Endline % Change Apparel 79 43 –46% 9 6 –31% Cashew 7 9 27% 125 133 7% Fish 1 12 1050% 26 30 14% General 8 6 –20% 9 11 20% Home décor and fashion accessories 7 15 120% 31 55 79% Shea 15 18 18% 4 5 15% Specialty Foods 16 15 –2% 4 3 –38% Textiles 79 43 –46% 9 6 –31% Additional Findings In addition to asking firms questions about key Trade Hub project outcomes, the WATH survey also asked firm level respondents about their perceptions of major constraints to export growth and types of assistance received (from the WATH and other donors). Interestingly, the most commonly cited challenges to export growth were economic shocks and lack of access to investment and trade financing. In addition, when asked what type of assistance received from outside entities has most impacted firm sales and exports in recent years, more than 50 percent of firms reported that trade shows and business-to￾business forums and financing had the most impact. More detail on these findings and those related to Trade Hub and donor intervention best practices and firm-reported constraints to continued sales, export and employment growth are provided in Annex V. 4. Conclusions – West Africa Trade Hub Impact Evaluation The comparison between the assisted and unassisted firms yielded several major conclusions. First, USAID-assisted firms have performed better than unassisted firms, both in total sales and exports. Second, export performance for USAID-assisted firms was better than for unassisted firms in all value chains. Third, export diversification was greater for assisted firms than for unassisted firms in terms of both the average number and size of export market destinations. Finally, assisted firms, which were heavily invested in the apparel and textiles sectors, tended to shed employment on average in greater numbers in response to declines in those sectors, despite superior export and sales performance across all sectors. However, statistical tests and regression analysis did not yield any statistically significant 33 Figures rounded to the nearest decimal. Source: Evaluation Survey 36 Africa Trade Hubs Evaluation Report differences in outcomes between the treatment and comparison groups (see Annex II for more details on the tests conducted).34 EAST AND CENTRAL AFRICA TRADE HUB PERFORMANCE EVALUATION 1. Methodology The evaluation team used a mixed-method performance evaluation that incorporated a quantitative phone survey of beneficiary firms with qualitative KIIs of a sample of beneficiary firms in Kenya and Ethiopia as well as KIIs with Trade Hub staff, key partners and associations and a desktop review of Trade Hub documents and secondary data and reports. The evaluation team selected a mixed-method design to increase the rigor of results, allowing for triangulation of findings as well as qualitative methods that seek to answer the “how” and “why” behind quantitative results. The evaluation team attempted to collect data on key evaluation outcomes through phone calls to all 96 firms supported by the ECATH since 2009.35 Evaluators were successful in reaching 66 of those firms (69 percent), five (7.5 percent) of which had gone out of business and were not interested in talking with the evaluation team, leaving 61 firms (64 percent) in the sample. The final sample includes: • 36 Textile/Apparel Firms (68 percent of all the ECATH-supported apparel firms under the second iteration of the Trade Hub) • 5 Floriculture Firms (50 percent of all the ECATH-supported floriculture firms under the second iteration of the Trade Hub) • 5 Specialty Food Firms (42 percent of all the ECATH-supported specialty food firms under the second iteration of the Trade Hub) • 15 Home Décor/Fashion Accessory Firms (71 percent of all the ECATH-supported home décor firms under the second iteration of the Trade Hub) The evaluation team collected sales, export and employment statistics on firms from 2002 (or the year the firm began operations, if later than 2002) and the year before the firm began working with the ECATH, which it compared to 2012 sales, exports and employment statistics provided by the ECATH for each of the firms.36 Evaluators took the first two data points (2002 to the year before the firm began working with the Trade Hub) to represent Period 1, which evaluators used to look at firm sales-, export- and employment-growth trends prior to the point when firms began receiving assistance from 34 Just because a finding is not statistically significant does not mean that it does not provide valuable information; it just means that the sample size was too small to allow for generalizability between that finding in the sample and what is true of the entire population (i.e., the entire intervention group). 35 Many of these firms were also supported by the previous iteration of the Trade Hub. The evaluation team was unable to contact firms that were supported under the previous iteration of the Trade Hub but that were no longer receiving ECATH support because the ECATH did not have records for these firms. While evaluators were able to obtain a list of the firms, data collectors found it difficult to obtain contact information for the firms and also worried about the extent of recall bias that may be present in asking firms to recall exports from 10 years ago. 36 The evaluation team data collectors spot-checked these numbers with firms and found them to be consistently accurate. Africa Trade Hubs Evaluation Report 37 the Trade Hub. Period 1 was compared with Period 2, which evaluators derived by analyzing growth trends between the second two data points (the year the firm began working with the ECATH and 2012), the period of time that represents the period that the firm was working with the Trade Hub. The evaluation team used this methodology to compare trends in key outcome variables for firms before they began working with the Trade Hub to trends in key outcomes after the firms began working with the Trade Hub. The purpose of this comparison was to see if there were any shifts in average annual growth of firm sales, exports or employment after Trade Hub assistance. In addition to the quantitative survey, evaluators also met with 11 firms (7 in Ethiopia and 4 in Kenya37) to gain additional insight into causes of firm level results as well as to what firms attribute these changes in their outcomes. For this part of the study, evaluators sought to reach as many firms as possible within a limited period of time, which meant that the firms had to be located in the capital cities of Addis Ababa and Nairobi. Additionally, evaluators sought to visit firms from each of the sectors as well as to visit some firms that have been more successful and others that have been less successful. As such, the evaluation team purposefully selected firms through consultation with ECATH staff. The evaluation team reached the following types of firms for in-person interviews: • 6 Textile/Apparel Firms • 3 Home Décor and Fashion Accessory Firms • 1 Specialty Food Firm • 1 Leather/Footwear Firm The evaluation team used a semi-structured KII process for the firm level interviews, using the data collection instrument found in Annex III. Next, the evaluation team conducted semi-structured KIIs with ECATH staff, staff from key partner organizations in Ethiopia and Kenya and association-level staff (a complete list of KIIs and ECATH staff interviews is found in Annex IV). Evaluators developed a separate data collection instrument for these interviews, which can be found in Annex III. Finally, evaluators reviewed the ECATH PMP as well as its annual reports, work plans, quarterly reports, success stories and one-off reports, as well as past evaluation reports conducted related to the ECATH or trade capacity building (TCB) by the U.S. government in Africa. A list of documents reviewed can be found in Annex IV. 37 The evaluation team did not have the resources to travel to other countries in the region, and Ethiopia and Kenya are two of the top three countries housing the ECATH-supported firms. 38 Africa Trade Hubs Evaluation Report 2. East and Central Africa Regional Export Trends Trends in East and Central African exports to the United States in sectors that have received support from the ECATH are shown in Figure 7.38 Nontraditional export sectors (i.e., non-petroleum and non￾mineral) increased from $232 million in 2001 to $607 million in 2011. Noteworthy in this respect is that the apparel and textiles sector, which declined following derogation of the Multi-Fibre Arrangement in 2005, and then remained fairly stagnant for several years, appeared to be back on an upward trend starting in 2011. Of the other sectors, food and food ingredients appeared to be the most dynamic. Figure 7: East and Central African Exports to the United States in USAID-Supported Sectors 38 Source: USITC data and KSA analysis $- $50 $100 $150 $200 $250 $300 $350 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 East and Central African Exports to the United States in Sectors Supported by USAID (in $ millions) Apparel and Textiles Food and Food Ingredients Manuf. Goods (other than food and apparel) Other Africa Trade Hubs Evaluation Report 39 3. Findings Firm Level Effects (Sales) The evaluation team found that more firms experienced positive annual sales growth in Period 2 than in Period 1. Specifically, in Period 1: • 20 of the 55 firms (37 percent) that started operations at least one year before beginning to work with the Trade Hub saw positive sales growth between 2002 (or the year the firm began operations) and the year before the firm began working with the Trade Hub.39 • 28 of the 55 firms (51 percent) saw zero growth in these years.40 • 7 of the 55 firms (13 percent) saw negative growth in these years. In Period 2: • 45 of the 60 firms (75 percent) with valid sales figures for the year before the firm began working with the Trade Hub and for 201241 saw an increase in their sales after beginning work with the Trade Hub.42 • 8 of the 60 (13 percent) reported negative sales growth.43 • 7 of the 60 (12 percent) reported no change at all.44 While this positive finding is encouraging, it is possible that even though a firm continued to experience positive sales growth, its annual percentage growth could decrease in Period 2. As such, the evaluation team also analyzed firms’ CAGR and found that the average CAGR in sales among all firms sampled increased by 36 percentage points between Period 1 and Period 2 (this number drops to 14 percentage points if one outlier firm with a significant jump in sales is excluded from the analysis), suggesting that sales were likely to increase after Trade Hub assistance.45 This could be subject to bias if firms had more 39 The average number of years in this period for these firms was 6.2. 40 The average number of years in this period for these firms was 5.9. 41 One firm was excluded because it reported two different figures for sales in 2012. 42 The average number of years in this period for these firms was 4.6. 43 The average number of years in this period for these firms was 6. 44 The average number of years in this period for these firms was 4.7. 45 Thirteen firms (21 percent) are not included in this analysis because their compounded annual growth rates could not be calculated for one or both periods—either because 2002 (or the year the firm first opened for business) was the same as the year before the firm began working with the Trade Hub (five instances) or because the firms had zero sales in some of the Finding 1: More EATH-supported firms experienced positive annual sales growth after working with the ECATH than they did before working with the ECATH. Finding 2: The compounded annual growth rate of sales among the ECATH-supported firms increased following the beginning of the ECATH intervention. 40 Africa Trade Hubs Evaluation Report years in Period 2 than Period 1, thus giving them more time to accumulate positive results. However, the average firm had more years in Period 1 than in Period 2 (5 versus 4.7, respectively), so this does not appear to be an issue. Figure 8 displays the ECATH firms’ CAGR for sales in Period 1 and 2. The great spike in the CAGR results for floriculture is due to one outlier firm that experienced a CAGR for sales of more than 5,000 percent in Period 2. Since there are only three other floriculture firms (two of which actually experienced negative CAGRs for Period 2), this outlier drives the results. It is also important to note that the specialty foods numbers are based on data from only three firms, though there does not appear to be a significant outlier. Figure 8: ECATH Firms’ CAGR for Sales in Periods 1 and 2 Figure 9 presents data on mean absolute annual sales growth. The evaluation team was able to include all 61 of the sample firms for this analysis. However, a few firms again drive results due to their size. For instance, textile and apparel firms appear to have received a great boost in annual sales in Period 2 years (eight instances). Closer examination of these firms reveals that, if anything, exclusion of these firms may be underestimating the differences in sales between Period 1 and Period 2, as almost all (11 of the 13 firms—85 percent— excluded from the analysis) had mean annual growth rates in Period 2 that were significantly higher than their mean annual growth rates in Period 1. In other words, the graph in Figure 8 may underestimate the contribution of the Trade Hub to firm compounded annual sales growth. Africa Trade Hubs Evaluation Report 41 compared to Period 1, but much of this jump is due to four very large firms that experienced sales increases in the range of tens of millions of dollars. Most other textile and apparel firms, by comparison, experienced sales increases in the ten- to hundred-thousand-dollar range. Specialty food and floriculture firm numbers were also largely driven by dramatic changes among a few firms. Data was only collected for five firms within each of those sectors, and of those five, three experienced increases in their mean absolute average sales growth in Period 2, whereas two experienced decreases (the same was true for both sectors). Furthermore, of the two firms experiencing decreases, one for each sector experienced a decrease in the millions of dollars. Figure 9: ECATH Firms’ Mean Absolute Annual Sales Growth in Period 1 and Period 2 Firm Level Effects (Exports) The evaluation team’s findings on exports largely mirror those of sales. Evaluators found that more firms experienced positive annual export growth in Period 2 than in Period 1. Specifically, in Period 1, the evaluation team found that: 42 Africa Trade Hubs Evaluation Report • 20 of the 55 firms (37 percent) that started operations at least one year before beginning to work with the Trade Hub experienced positive export growth between 2002 (or the year the firm began operations) and the year before the firm began working with the Trade Hub.46 • 28of the 55 (51 percent) saw zero growth in these years.47 • 7 of the 55(13 percent) saw negative growth in these years.48 In Period 2: • 44 of the 60 firms (73 percent) with valid sales/export figures for the year before the firm began working with the Trade Hub and 201249 saw an increase in their exports after beginning work with the Trade Hub.50 • 9 of the 60 (15 percent) reported negative export growth.51 • 7 of the 60 (12 percent) reported no change at all.52 The evaluation team also analyzed firms’ CAGR for exports and found that the average CAGR in exports among all firms sampled increased by 157 percentage points between Period 1 and Period 2 (this number drops to 16 percentage points if one outlier firm with a significant jump in exports is excluded from the analysis). These numbers suggest that ECATH interventions likely contributed to increased exports among firms.53 46 The average number of years in this period for these firms was 6.2. 47 The average number of years in this period for these firms was 5.9. 48 The average number of years in this period for these firms was 5.9. 49 One firm was excluded because it reported two different figures for sales in 2012. 50 The average number of years in this period for these firms was 4.5. 51 The average number of years in this period for these firms was 5.7. 52 The average number of years in this period for these firms was 4.7. 53 Twenty-three firms (38 percent) are not included in this analysis because their CAGR could not be calculated for one or both periods—either because 2002 (or the year the firm first opened for business) was the same as the year before the firm began working with the Trade Hub (five instances) or because the firms had zero exports in some of the years (18 instances). Finding 3: More ECATH￾supported firms experienced positive annual export growth after working with the ECATH than they did before working with the ECATH. Finding 4: The compounded annual growth rate of exports amongst the ECATH-supported firms increased following the beginning of the ECATH intervention. Africa Trade Hubs Evaluation Report 43 Figure 10 displays detail of CAGRs by sector for Periods 1 and 2. As with sales outcomes, the great spike in the CAGR results for floriculture exports in the figure is due to one outlier firm that experienced a CAGR for exports of more than 1,100 percent in Period 2. Since there are only three other floriculture firms analyzed (two of which actually experienced negative CAGRs for Period 2), this outlier drives results. It is also important to note that, as with sales outcomes, the specialty foods numbers are based on data from only three firms, though there does not appear to be a significant outlier. Figure 10: ECATH CAGR for Exports Periods 1 and 254 Finally, Figure 11 presents data on mean absolute annual sales growth. As was also true of the sales outcomes analysis, the evaluation team was able to include all of the sample firms for this analysis. However, outliers have, again, had an even greater effect since some firms are much larger than others. For instance, textile and apparel firms appear to have received a great boost in annual exports in Period 2 compared with Period 1, but much of this jump is due to the same four very large firms discussed above that experienced sales increases in the range of tens of millions of dollars. These firms experienced similar jumps in their exports. Most other textile and apparel firms, by comparison, experienced export increases in the ten- to hundred-thousand-dollar range. Specialty food and floriculture firm numbers were also largely driven by outliers. Data was only collected for five firms 54 Twenty-three firms (38 percent) are not included in this analysis because their CAGRs for exports could not be calculated for one or both periods—either because 2002 (or the year the firm first opened for business) was the same as the year before the firm began working with the Trade Hub (five instances) or because the firms had zero exports in some of the years (18 instances). Closer examination of these firms reveals that, if anything, exclusion of these firms may underestimate the differences in export growth between Period 1 and Period 2, as most firms (17 of the 23 firms—74 percent—of those excluded from the analysis) had mean CAGRs for exports in Period 2 that were significantly higher than their CAGRs for exports in Period 1. In other words, Figure 10 may underestimate the contribution of the Trade Hub to firm CAGR of exports. 44 Africa Trade Hubs Evaluation Report within each of those sectors, and of those five, three floriculture and four specialty food firms experienced increases in their mean absolute average export growth in Period 2, whereas two floriculture and one specialty food firm(s) experienced declines. Of the firms experiencing decreases, one for each sector experienced a decrease in the millions of dollars. Figure 11: ECATH Firms’ Mean Absolute Annual Export Growth in Period 1 and Period 2 Though t-tests showed that the difference in CAGRs for sales and exports between Period 1 and Period 2 is not statistically significant within a 90 percent confidence level, there does appear to be a trend suggesting that firms have done better after ECATH assistance. Further, qualitative interviews with firms in Kenya and Ethiopia showed that only two of the eleven firms interviewed reported that their sales and exports did not increase as a direct result of the ECATH assistance. Additionally, one of those firms reported that it expected to experience increased sales and exports, and, in fact, the Trade Hub had helped it to make deals with a few international buyers. The only reason that firm had not yet experienced an increase in sales and exports was because its owner wanted to make some adjustments in production before beginning to export to its new-found buyers. The second firm that did not attribute increased sales and exports to the ECATH also could not remember having received support from the Trade Hub despite Trade Hub reports that the firm has been receiving assistance from the ECATH since the first iteration of the Trade Hub (going Finding 5: ECATH technical assistance with business planning/access to finance and obtaining certificates was shown to positively contribute to export growth. Finding 6: Firms report needing additional technical assistance to take full advantage of the ECATH support. Finding 7: Firms and key informants report that access to finance and regulatory barriers are the greatest constraints to further increasing their exports. Africa Trade Hubs Evaluation Report 45 as far back as at least 2005) and documented evidence of the firm’s participation in multiple ECATH￾supported trade shows. Further, evidence suggests that this particular firm also received direct product line support and training from the ECATH, which respondents later recalled upon prompting. Upon further investigation, evaluators found that the respondents interviewed from this firm were relatively new to the firm. Thus, the firm’s lack of attribution of increased sales and exports to the ECATH may be a result of lack of information (as well as poor branding—since the firm did remember some of the interventions provided but didn’t know they were done by the ECATH) more than anything else. Triangulating these results, five out of seven firms reported that without the Trade Hub they likely would have eventually been able to achieve the same level of sales and exports, but that the ECATH helped them to get to that point much more quickly. Evaluators also found that certain types of ECATH assistance were more likely to contribute to increased exports than other types of assistance. Specifically, through regression analysis, the evaluation team found that both technical assistance on business planning/access to finance and training on how to obtain certificates of quality positively contributed to firms’ CAGRs for exports after firms began receiving Trade Hub assistance (Period 2). Technical assistance on business planning/access to finance was shown to increase CAGRs by almost 1 percentage point and assistance with obtaining certificates of quality by almost 2 percentage points. Results were statistically significant in both cases. However, as reported in the Project Background section of this report, only 10 and 20, respectively, of the 93 firms for which type-of-assistance information is available have received these types of technical assistance. Other types of assistance were not shown to have a statistically significant effect on the evaluation outcomes of interest. Despite these quantitative results, during qualitative interviews, the majority of firms reported that assistance with trade shows was the most important type of assistance that they had received from the Trade Hub. This may not be reflected in the quantitative results because firms also reported that many firms, sometimes including their own firm, were not able to take full advantage of the trade shows due to a lack of capacity. Several firm owners said they knew of firms that had accepted orders from international buyers and then were unable to deliver because of a lack of capacity to fill the order. Key informants reported that this type of failure on the part of firms hurt not only those firms but also African firms in general, as it made buyers skeptical of the capacity of African firms. Firm owners and other key informants also spoke of firms’ lack of ability to attract potential buyers to their booths at the trade shows or to close deals once the buyers were there. In fact, of the seven firms asked, six (86 percent) reported needing more technical assistance/training (despite having received some ECATH interventions—including some technical assistance, according to ECATH reports—these firms reported that they had not received any training but instead only assistance with attending trade shows and making connections with buyers). This lack of capacity and training provided to firms may be limiting the effectiveness of the ECATH￾supported trade shows and may explain the lack of correlation between participation in trade shows and increased exports. Firms that asked for additional technical assistance from the Trade Hub were looking for assistance with tailoring their products to the international market, ensuring the quality of their products for the international market, building the physical capacity of their firms to fill orders (and to understand how many orders are feasible), benchmarking, conducting market research and liaising and closing deals with potential buyers. Six out of seven firms reported that access to finance was the greatest constraint that they faced in relation to increasing their sales and exports. They also listed several other constraints as important, but they reported that these constraints could be avoided or mitigated through increased access to finance. This finding supports the quantitative finding that those firms that received business planning/access to finance training experienced higher annual export growth rates than did those firms that did not. KIIs 46 Africa Trade Hubs Evaluation Report with other stakeholders (including associations and other donors) in Kenya and Ethiopia also identified regulatory barriers as a factor limiting potential export growth. Export Diversification Very few firms reported diversifying their product lines as a result of the ECATH support. However, five out of seven firms who were asked an open-ended question about the most significant changes occurring at their firm in the last five to ten years (during qualitative KIIs) reported that, while they had not expanded to new product lines, they had increased the types of products they sold in the product line with which they already worked (e.g., if the firm sold shirts before, they might now sell shirts and overalls).55 This was also true of the footwear firm interviewed, and a representative from the Leather Industry Development Institute of Ethiopia (LIDI) said this was true of several other ECATH-supported footwear firms in Ethiopia as well. 55 Other changes reported through this open-ended question included increased sales and exports (five out of seven firms), beginning to export for the first time (four out of seven firms), increased employees (three out of seven firms), and beginning to receive Trade Hub assistance (three out of seven firms). These numbers do not represent the total number of firms that have experienced this phenomenon but, instead, which firms spontaneously reported these changes without specifically being asked about them. Finding 8: While overall the ECATH firms have diversified the number of export markets to which they sell their products on a sectoral basis, export diversification has been mixed. Africa Trade Hubs Evaluation Report 47 Figure 12: ECATH Firm Export Diversification in Periods 1 and 2 Additionally, evaluators found that there was a statistically significant difference in the number of export destinations among firms in Period 2 compared with the number of export destinations among firms in Period 1. Specifically, the evaluation team found that the number of export destination countries increased from an average of 2.2 in the year before firms began working with the Trade Hub to 3.1 in 2012. Finally, as shown in Figure 12, evaluators found that while both apparel/textile firms and home décor/fashion accessory firms experienced increases in the number of destination countries to which they exported between Period 1 and Period 2, floriculture and specialty food firms (which both have small sample sizes), experienced a decrease in export destinations between these two periods. Job Creation Job creation outcomes show the greatest overall increase between Periods 1 and 2, with a statistically significant increase (at the 99 percent confidence level) between Periods 1 and 2. The evaluation team found that more ECATH￾supported firms experienced positive employment growth after working with the ECATH than did before working with the Trade Hub. For Period 1: Finding 9: More ECATH￾supported firms experienced positive employment growth after working with the ECATH than they did before working with the ECATH. Finding 10: Women’s share of employment amongst the ECATH-supported firms has not changed since the Trade Hub began working with firms. 48 Africa Trade Hubs Evaluation Report • 19 of the 56 firms (34 percent) that started operations at least one year before beginning to work with the Trade Hub experienced positive employment growth between 2002 (or the year the firm began operations) and the year before the firm began working with the Trade Hub.56 • 37 of the 56 (56 percent) saw zero or negative growth in these years.57 For Period 2: • 37 of the 60 firms (62 percent) with valid employment figures for the year before the firms began working with the Trade Hub and 2012 experienced an increase in their employment after beginning work with the Trade Hub.58 • 8 of the 60 (13 percent) reported negative employment growth.59 • 15 of the 60 (25 percent) reported no change at all.60 The CAGR in employment among all firms sampled increased by 37 percentage points between Period 1 and Period 2, suggesting that employment was likely to increase after Trade Hub assistance.61 See Figures 13 and 14 for information by sector. 56 The average number of years in this period for these firms was 3.8. 57 The average number of years in this period for these firms was 4.6. 58 One firm was excluded from this analysis because it reported two different figures for employment in 2012. The average number of years in this period for these firms was 4.5. 59 The average number of years in this period for these firms was 5.7. 60 The average number of years in this period for these firms was 4.7. 61 Five firms (8 percent) were not included in this analysis because their compounded annual employment growth rates could not be calculated because 2002 (or the year the firm first opened for business) was the same as the year before the firm began working with the Trade Hub (five instances), which meant that the employment growth rate could not be calculated for that period. Africa Trade Hubs Evaluation Report 49 Figure 13: ECATH Firms’ CAGR for Employment in Periods 1 and 2 Figure 14: ECATH Firms’ Absolute Annual Growth in Employment for Periods 1 and 2 50 Africa Trade Hubs Evaluation Report In addition to looking at general employment trends across ECATH-supported firms, the evaluation team assessed how women’s share of employment has changed since firms began working with the Trade Hub. Evaluators found that while women’s share of employment remains much higher than men’s—at 73 percent of the workforce among firms sampled in the evaluation team’s phone interview—this percentage has remained the same since the year before firms first began working with the Trade Hub. Sectoral Effects Regression and correlation analysis of data do not show any statistically significant relationships between export or employment growth and sector. Qualitative findings also did not identify any sectoral differences in outcomes. Regional Effects Regression and correlation analysis of data were unable to identify any statistically significant relationships between export or employment growth and country. However, the evaluation team found that some countries/regions were more likely to experience an increase in imports from ECATH-supported firms than other countries/regions. Specifically, the evaluation team found that while only 26 of the 61 ECATH-supported firms (43 percent) that responded to the phone survey exported to the U.S. the year before that firm began working with the Trade Hub, 53 of the 61 firms (87 percent) exported to the U.S. in 2012, for a total increase of firms in the sample exporting to the U.S. of 104 percent. This finding appears to demonstrate that the Trade Hub has been successful in contributing towards one of its key outcomes—increasing exports from ECA under AGOA. The United Kingdom was the second most common destination to experience a jump in imports from ECATH firms—with 64 percent more firms exporting to the UK in 2012 than did the year before those firms began working with the Trade Hub. France was third, with 57 percent more firms exporting to the country post-ECATH support than pre-ECATH support. 4. Conclusions – East and Central Africa Trade Hub Performance Evaluation With respect to the evaluation questions, the evaluation team concluded the following. Firm Level Effects Though quantitative data do not show statistically significant differences in sales and export performance from the period before firms began receiving ECATH assistance to the period after they began receiving ECATH assistance, the Trade Hub does appear from both the qualitative and quantitative data to have contributed toward increases in both sales and exports. Additionally, regression analysis of firm survey results showed that training in business planning and access to finance, as well as technical assistance with obtaining certificates, have contributed more to sales and exports outcomes among assisted firms than other types of ECATH assistance. Finally, both quantitative and qualitative evidence support the idea that the level of Trade Hub technical assistance provided could be improved, as could interventions aimed at reducing firm level constraints to exporting. Finding 11: Key outcomes do not appear to vary by sector. Finding 12: Key outcomes do not appear to vary based on the country from which the firm operates. Finding 13: The U.S. is the recipient of exports from more ECATH-supported firms than any other country. It has also experienced the greatest growth in the percentage of firms that send exports to the country than has any other country/region— followed by the UK and France, respectively. Africa Trade Hubs Evaluation Report 51 Export Diversification The ECATH beneficiaries reported a statistically significant increase in the number of export product destinations, though this may not be true among floriculture and specialty food firms. Further, at least some firms have also diversified their export product mix within the sector in which they work. Job Creation The ECATH-assisted firms report a statistically significant increase in the number of jobs in assisted firms from the period before the assistance began to the period following the assistance. However, while women make up a majority of those employed at interviewed firms, they are no more likely to hold a job in one of these firms now than they were prior to ECATH assistance. Sectoral Effects The evaluation team has not found any evidence that ECATH assistance is more effective with regard to some product sectors than others. Regional Effects While evidence does not show that ECATH assistance is more effective in generating jobs, sales and exports in some countries than in others, it is more effective at increasing exports to the United States than to any other region or country, likely because of AGOA. SOUTHERN AFRICA EXPORT TRENDS Trends in Southern African exports to the United States in sectors that have received support from the Southern Africa Trade Hub are shown in Figure 15.62 Nontraditional export sectors (i.e., non-petroleum and non-mineral) increased from $2.9 billion in 2001 to $4.7 billion in 2011. Noteworthy in this respect is that the apparel and textiles sector declined precipitously, from $1.5 billion in 2004, just prior to derogation of the Multi-Fibre Arrangement, to $608 million in 2011. Of the other sectors, manufactured goods (other than food and apparel) was the most dynamic. 62 Source: USITC data and KSA analysis 52 Africa Trade Hubs Evaluation Report Figure 15: Southern Africa Exports to the United States in USAID-Supported Sectors (in $ millions) SUMMARY FINDINGS AND CONCLUSIONS 1. Firm Level Effects Did exports/revenues increase as a result of USAID export assistance? Was there a statistically significant difference in export performance by assisted firms vs. overall regional/global export trends in those sectors/products (i.e., more than in entities that were not assisted)? Is there a statistically significant difference in export/revenue performance between firms that received assistance and those that did not? While not statistically significant,63 responses from firms surveyed in and around Accra show that Trade Hub export promotion assistance appears to have positively affected the sales and export performance of the firms assisted by the WATH, both in the aggregate and in each of the value chains supported. Even in the apparel, textiles, cashew and shea sectors, in which the exports of surveyed firms actually fell on average, the relative success of the treatment group was due to the fact that their exports fell by less 63 As described earlier, just because a finding is not statistically significant does not mean that it does not provide valuable information. $0 $500 $1,000 $1,500 $2,000 $2,500 $3,000 $3,500 Southern Africa Exports to the United States in Sectors Supported by USAID (in $ millions) Apparel and Textiles Food and Food Ingredients Manuf. Goods (other than food and apparel) Other Africa Trade Hubs Evaluation Report 53 on average than did those of the comparison group.64 The question remains, however, about whether the sample, which comprised Accra-only firms, is representative of all firms in West Africa. The findings from the performance evaluation of firms assisted by the ECATH support the idea that USAID-assisted firms performed relatively better in regards to sales, exports, jobs and diversification since they began receiving ECATH support than they did before. ECATH-supported firms performed well despite the fact that they were active predominantly in the apparel and textiles sector, which experienced stagnating exports from the East and Central Africa region during the period under review. 2. Export Diversification Has trade/export assistance delivered by the Trade Hubs helped to diversify trade in assisted sectors/firms? Was there a statistically significant difference in the export destination or export product mix of assisted firms in comparison to overall export trends in this product/sector? The in-depth survey of firms in and around Accra offered concrete (yet not statistically significant) evidence that firms assisted by the USAID Trade Hub tended to be more successful in diversifying export markets than did unassisted firms. The average number of export markets for the assisted firms surveyed increased in one half of the value chains supported by USAID in West Africa, whereas the unassisted firms surveyed tended to lose markets in all value chains. Perhaps more important, the average size of export market destinations increased more rapidly for assisted firms than for unassisted firms, with major gains occurring in exports to the EU and West Africa among firms in the treatment group that were not mirrored among firms in the comparison group.65 The performance evaluation in East and Central Africa also found that Trade Hub–supported firms tended to diversify their exports. Assisted firms in all value chains except floriculture increased their export destinations, on average, between Period 1 (before receiving USAID Trade Hub assistance) and Period 2 (after receiving the assistance).66 3. Job Creation Did the increase in exports/revenues lead to the creation of jobs in assisted firms? Is there evidence that this job creation exceeded overall job creation in the sector (i.e., exceed the number created in firms in the same area/industry that were not assisted)? Was there a statistically significant improvement in job creation in assisted firms from the period before the assistance began to the period following the assistance? 64 See Annex II for the results of the statistical analyses. Lack of significance of the statistical tests may result from the relatively small sample size and error terms in the samples collected. It may also be the case that “spillover” effects from USAID assistance, and the benefits of other donor assistance, may have closed the gap between treatment and comparison group firms to such an extent that the differences between them were not statistically significant. 65 Comparison of the number and size of exports to major market destinations between assisted firms and regional trends was subsumed within the comparison of treatment and comparison groups. 66 According to KIIs conducted by the evaluation team, the reason floriculture export destination markets suffered may have been that Jomo Kenyatta Airport failed to meet the security requirements to become a source for direct flights from Kenya to the United States. 54 Africa Trade Hubs Evaluation Report Survey results from the WATH IE indicated that employment by treatment group firms actually declined on average, while employment by comparison group firms increased. This counterintuitive result may be explained by the fact that gains in employment in certain value chains were more than offset by job losses in apparel and textiles, labor-intensive value chains in which overall exports to the United States and the number of export markets have both declined significantly in recent years. By way of contrast, the findings from the performance evaluation performed in East and Central Africa suggest that USAID Trade Hub assistance was associated with more rapid job creation. Whereas 34 percent of the firms reported increasing employment in the period before starting to work with the Trade Hub, 62 percent increased employment in the period after the assistance began. Another interesting result from the ECATH firm level survey was that women’s share of employment has remained much higher than men’s throughout the course of the project—at 73 percent of the workforce among firms sampled from the year before firms first began working with the Trade Hub to 2012. However, the WATH survey found that the women’s share of employment among assisted firms declined by 27 percent after working with the Trade Hub. This is most likely explained by the decline in employment in the apparel and textiles sector—which employs an overwhelming predominance of women. 4. Expenditures What is the relationship between USAID Trade Hub expenditures on export assistance and export/revenue and job creation performance (ratios of expended costs to exports/revenues/jobs generated)? To what extent can changes in export performance or job creation be attributed to USAID Trade Hub expenditures? The evaluation team faced data limitations in answering these questions. Trade Hubs were only required to collect information on attributed exports—a measure that is highly subjective and prone to bias— with the methodology being to ask clients to report on how much of their exports they thought were attributable to USAID. This could have had the effect of either over- or under-estimating the impact of USAID support, and variation in the concept may also occur by value chain, depending on the type of support received and the clients’ definition of “attributed.” With this in mind, the evaluation team compared the Trade Hubs’ reporting of attributed increases in exports derived from firm level export promotion support to figures they supplied on expenditures related to support, on a cumulative basis, for the years in which data was available. The results, reported in Table 12, reflect somewhat implausible divergences in the amount of sales attributed to the amount of firm level support provided in each region. Africa Trade Hubs Evaluation Report 55 Table 12: AGOA-Related Exports Attributed to USAID Trade Hubs’ Export Promotion Assistance67 West Africa, 2010–201268 East and Central Africa, 2009–201269 Southern Africa, 2011–201270 Attributed Exports $205,929,881 $95,690,075 $1,038,775 Export Promotion Expenditures $27,090,788 $17,645,724 $1,768,543 Attributed Exports per USAID $ $7.6071 $5.42 $0.59 The comparatively low figures for Southern Africa appear to be consistent with their reporting that they had achieved only 6 percent of their targeted export levels in FY 2011– FY 2012, and indeed the Southern Africa Trade Hub has been instructed by USAID to phase out firm level assistance. However, the comparatively high figures for West Africa may be related to an unclear definition of the concept of “attributed exports” across value chains, as discussed in the next section. 5. Sectoral Effects Is trade/export assistance delivered by the Trade Hubs relatively more effective within specific product sectors (e.g., traditional agriculture products, nontraditional agriculture products, manufactured goods, services, etc.)? Only the WATH implementing partner was able to provide information on both expenditure estimates and attributions of exports to firm level assistance by value chain. The cumulative results from FY 2010 – FY 2012, presented in Table 13, show that attributable increases in exports per USAID dollar spent vary dramatically by value chain.72 Given these attributions, one dollar spent on firm level assistance in the shea sector produces ten times more in exports than in the handicrafts (i.e., “home decor and fashion accessories”) sector. The greatest export attributions per dollar spent were in the cashew sector (at more than 100 to 1). 67 Source: Export promotion expenditures and attributed export figures provided by Trade Hub implementing partners, by fiscal year. 68 Excluding “Fish and Seafood” value chain, support to which was discontinued by USAID. 69 Cumulative AGOA-related attributed exports figure drawn from COMPETE FY2012 annual report, p. 6. 70 Cumulative attributed exports figure drawn from SATH PMP FY 2012, p. 14. 71 Evaluation Team revised this number from the figures submitted by the Trade Hub to reflect the proper calculation (original value was $17.03). 72 Note that USAID assistance to the fish and seafood sector was terminated after a determination was made that the fishery off the Atlantic coast could not be certified as ecologically sustainable. 56 Africa Trade Hubs Evaluation Report Table 13: West African Trade Hub Attributed Exports and Export Promotion Expenditures by Value Chain, FY 2010–201273 Shea (USD) Cashew (USD) Fish & Seafood (USD) Specialty Foods (USD) Apparel (USD) Home Décor & Fashion Accessories74 (USD) Expenditures $2,640,899 $1,435,537 $414,785 $1,733,829 $1,865,391 $4,003,827 Attributed Exports $36,382,567 $146,569,445 $0.59 $8,060,860 $9,521,426 $5,395,583 Attributed Exports per USAID Dollar $13.78 $102.10 TERMINATED $4.65 $5.10 $1.35 Given that the broader impact of USAID Trade Hub assistance on sectors and the region as a whole (outside of just direct beneficiary firm impacts) is part of USAID’s development hypothesis, it would be useful for the Trade Hubs’ implementing partners, or perhaps trade associations, to measure whether such “scaling up” is occurring over time. Regarding expenditures by value chain, Table 13 demonstrates that firm level assistance in West Africa appears to have been concentrated heavily in the cashew value chain, which accounted for more than 70 percent of total expenditures. Additionally, expenditure levels tracked well with value chain results in terms of attributable exports per USAID dollar spent. According to these figures, the WATH appears to be spending USAID resources cost-effectively, with more spending in those sectors where the most exports per USAID dollar were being realized. 6. Regional Effects Has trade/export assistance delivered by the Trade Hubs been relatively more effective in generating exports/revenues/jobs in specific countries/regions or more effective in generating exports to certain destination markets over others (e.g., traditional agriculture products, nontraditional agriculture products, manufactured goods, services, etc.)? Evaluators were unable to determine whether the WATH was more effective at generating project outcomes in some countries than in others, due to a lack of data from the Trade Hub on clients’ domestic sales, exports and employment by country, and the limited resources of this evaluation (which limited the firm level survey to firms in Accra). In the future, tracking this information will be helpful to gauge the extent to which support to one country or another is relatively more cost-effective, as well as to identify and take advantage of new targets of opportunity as they arise. Additionally, this information would be beneficial for measuring spillovers across the region. 73 Source: West African Trade Hub estimates 74 i.e., “handicrafts” Africa Trade Hubs Evaluation Report 57 Evidence for the ECATH shows that assistance was just as effective in generating jobs, sales and exports in the two countries surveyed. As the charts in the previous sections presenting regional export trends suggest, exports to the United States in sectors supported by the Trade Hubs increased in all three regions (and more so than exports to any other region). The most dramatic increases appear to be in the food and food ingredients sector in both WA and ECA, and in manufactured goods other than apparel in Southern Africa. RECOMMENDATIONS USAID should require future Trade Hub implementing partners to submit a detailed PMP supported by a clear development hypothesis and results framework at the project’s outset that includes outcome-level indicators in addition to input- and output-level indicators and also includes indicators aimed at measuring whether or not theory-of￾change assumptions hold true (as described in Annex VI). Such a PMP is important for informing USAID and implementing partner mid-course management decisions related to project efficiency and effectiveness, as well as for allowing for more rigorous future evaluations of the Trade Hubs as described in Annex VI. This PMP should be based on a clearly defined development hypothesis and results framework and should include indicators that track the following (see Annex VI for more details): • Firm total annual sales at baseline and then every year of project implementation. • Firm total annual exports at baseline and then every year of project implementation. • Firm total annual employment at baseline and then every year of project implementation. • Firm total annual women’s share of employment at baseline and then every year of project implementation. • Firm export destination countries, buyers and the share of exports sent to each of these countries and buyers at baseline and then every year of project implementation. • Firm product types at baseline and then every year following project implementation. • Total project expenditures (including labor expenses) by individual activity as well as a list of firms participating in that activity for every year of project implementation. • A list of other donor organization and NGO assistance provided to the firms by year at baseline and then every year following project implementation. The Trade Hub could collect these indicators directly, or the Hub could require associations to report on these results for each of the firms that they assist as a result of the project. If the latter is done, USAID will need the associations to be careful to differentiate between activities that they would have provided to firms before Trade Hub assistance and those provided only because of Trade Hubs assistance. Data should be disaggregated by value chain and country. USAID should require the Trade Hubs to monitor both surviving and failing firms for results reporting purposes. This should include follow up with site visits and qualitative surveys to both types of firms to enrich USAID and Trade Hub knowledge of what works and does not in firm level export promotion assistance. When firms go out of business, implementing partners should make a concerted effort to interview the business owner to understand the reason behind the firm’s closure, as well as to collect the last data available for each of the indicators outlined in the first recommendation above (and in Annex VI). 58 Africa Trade Hubs Evaluation Report Given the multitude of potential methodological biases and restricted sample size, any future IE of Trade Hub programming should be designed alongside the project. This is particularly critical for development of an approach to measure spillovers, which is an important element of the Trade Hubs’ program design, yet one that can lead to biased evaluation results if unaccounted for. If USAID wishes to conduct an IE in the future, it should plan for that IE by requiring implementing partners or an independent evaluation firm (preferred under USAID’s 2011 evaluation policy)75 to designate a control group that is as similar to the treatment group as possible prior to the project intervention. USAID should also require implementing partners to track key indicators (as outlined in the PMP recommendation above and in Annex VI) for both the treatment and control groups over time. If USAID wishes to not withhold assistance from any subset of firms, it can require the Trade Hubs to stagger the intake of promising firms into their export promotion projects so that there is a two-year lag during which the performance of unassisted firms may be compared with that of assisted firms, prior to their being incorporated into firm level assistance activities.76 Collection of longitudinal data on client exports and employment should be required of Trade Hubs’ clients so that cost-effectiveness of support to different value chains can be documented. Cost-effectiveness of alternative interventions and implementation approaches should also be documented to help identify interventions that use resources most efficiently. Cost-effectiveness may be defined as the achievement of maximum provision of goods or services from given inputs. Cost￾effectiveness analysis could not be performed in this evaluation due to lack of detailed expenditure data. This shortcoming suggests that evidence from an evaluation should extend to include activity-by-activity cost data, including the cost of staff time associated with each activity and some allocation of fixed costs across activities. This information, when paired with project outcomes, will allow stakeholders to assess which alternative is most effective in reaching a particular goal. This is described in more detail in Annex VI of this report. USAID should maintain a centralized database of all Trade Hub reports, including RFPs, work plans, annual and quarterly reports and PMP data in order to ensure institutional memory for future implementing partner and evaluation implementing partners, as expanded upon in Annex VI. In line with its policy of encouraging “collaborating, learning and adapting (CLA),” USAID should periodically assess whether the value chains that the Trade Hubs are supporting actually are those with the greatest potential to generate exports and employment, and if needed, guide the implementing partners to make midstream corrections accordingly. In particular, the experience of firms supported in the textiles and apparel sector is an instructive case study. When the Multi-Fibre Arrangement expired in January 2005, global apparel and textile trade was no longer subject to quantitative restrictions, with the effect of centralizing production in countries outside sub-Saharan Africa. However, USAID did not shift its priorities toward other sectors—such as manufacturing— which have been quite dynamic. A greater CLA-type approach should be implemented in the future. 75 If USAID chooses to contract for an IE through an independent evaluation contractor, as required under USAID’s evaluation policy, it should contract with this firm at the same time that it contracts with the Trade Hub implementing partner firms—or at least several months before any Trade Hub assistance begins. 76 In a comprehensive evaluation of trade capacity building activities, evaluators identified a two-year lag between receipt of the assistance and achievement of large-scale impact (see footnote 4). Africa Trade Hubs Evaluation Report 59 USAID should consider requiring the Trade Hubs to continue market-linkage work with firms either directly or through associations. Quantitative evidence from both the WATH and ECATH show that firms report that this type of assistance has had the most impact on increasing exports. Qualitative interviews demonstrated that the best method for assisting firms in this regard is through a step-down process, meaning firms are supported more heavily (financially and technically) when they first begin working with the Trade Hub (or association) and then, slowly, support is lessened until firms are supporting themselves entirely. USAID should consider requiring the Trade Hubs to expand efforts to reduce firm constraints to exporting by facilitating firm level access to finance and reducing regulatory barriers to exports. Results of regression analysis from the ECATH show that firms that have received support with accessing finance were more likely to have increased exports than firms that did not. In addition, access to finance and regulatory barriers were highlighted by firms in both ECA and WA as the main barriers to firms’ export growth. 60 Africa Trade Hubs Evaluation Report ANNEXES ANNEX I: EVALUATION SCOPE OF WORK ANNEX II: WEST AFRICA EVALUATION METHODS AND LIMITATIONS ANNEX III: DATA COLLECTION INSTRUMENTS ANNEX IV: SOURCES OF INFORMATION ANNEX V: ADDITIONAL LESSONS LEARNED FROM WATH SURVEY ANNEX VI: DATA MONITORING AND EVALUATION FOR FUTURE TRADE HUBS Africa Trade Hubs Evaluation Report 61 ANNEX I: EVALUATION SCOPE OF WORK SCOPE OF WORK AFRICA TRADE HUBS EVALUATION C.1. PURPOSE AND SCOPE OF THE EVALUATION A. Purpose: The purpose of the evaluation is to examine the impact of USAID’s Trade Hubs in effectively promoting international and regional exports from assisted sectors/firms. Although the Trade Hubs have engaged in a variety of Trade Capacity Building (TCB) activities, this evaluation will focus solely on the success of direct export assistance to sectors/firms. Through data collection and analysis, it will examine the effectiveness and impact associated with USAID export assistance. The evaluation findings will be used to support the implementation of the African Competitiveness and Trade Expansion Initiative, the next generation of Trade Hubs, a commitment announced by Secretary Clinton and USTR Ron Kirk at the June 2011 African Growth Opportunity Act (AGOA) Forum. In addition, the findings will be directly relevant to efforts under the Feed the Future Initiative to promote food security and regional trade. B. Scope: Evaluations of trade and export projects have been conducted in the past, typically analyzing at the national level overall changes in trade or export activity as a result of project activities. This project seeks instead to assess the effects of intervention at the disaggregated firm level. The results of the evaluation should yield an understanding of how Trade Hub activities affect the actions and outcomes of individual firms and how those effects vary by demographic, economic and geographic characteristics, as well as the types of interventions received. While each of the three African Trade Hubs has approached trade and export promotion in distinctive ways, this evaluation must integrate these unique approaches into a holistic analytical framework. As a result, the impact evaluation portion of this analysis will focus primarily on the activities of the West Africa Trade Hub and the East & Central Africa Trade Hub because their activities have focused heavily on direct assistance to firms for the purpose of improving export performance. While the projects undertaken by the Southern Africa Trade Hub have clearly impacted individual firms, assistance from the Trade Hub has focused on industry-level interventions such that the ability to compare directly to the other Trade Hubs would be limited and lacking in scientific validity. However, a preliminary step in the analysis involves a macro-level benchmarking exercise that examines trends in export performance by sectors, and the experiences and benchmarking of all three Trade Hubs will be included. C.2. BACKGROUND Three implementing agencies, Carana Corporation, Chemonics International and AECOM, have been providing technical assistance in export promotion activities to over 4,175 businesses77 (FY 2007-2010) 77Estimates are approximately 2,742 in West Africa; 509 for East/Central Africa; 924 Southern Africa. 62 Africa Trade Hubs Evaluation Report across the African continent. This assistance was provided via three sub-regional Trade Hubs; West Africa (WA), East & Central Africa (ECA), and Southern Africa (SA). Assisted firms have been exporting goods in a dozen sectors including, but not limited to, Wood Products, Cashew, Livestock, Shea, Tourism, Home Décor & Fashion Accessories, Furniture, Fish & Seafood, Specialty Food, Cut Flowers, and Apparel. From their launch, the Trade Hubs were primarily conceived as vehicles for firm level support to help countries maximize opportunities provided by the AGOA. Broader business environment activities to enhance regional competitiveness were part of the programming of all three Hubs, but to different degrees. The first generation of Trade Hubs was created to implement the objectives of the four-year, $70 million Trade for African Development and Enterprise (TRADE) Initiative (FY2002-2005). Trade Hub project design reflected TRADE objectives through activities in two technical areas: (1) trade facilitation and trade capacity building; and (2) AGOA education/export development; as well as cross￾cutting projects (e.g., gender, environment, HIV/AIDS prevention, and anticorruption). In 2005, the African Global Competitiveness Initiative (AGCI) was announced at the AGOA Ministerial Forum. AGCI was to expand upon the successes of TRADE and an additional $200 million of U.S. assistance was pledged to fund the second generation of Trade Hubs’ efforts (FY2006-2010) on an expanded list of policy objectives: • Promote U.S.-African business linkages • Enhance the competitiveness of African products and services • Expand the role that trade can play in African poverty reduction strategies • Improve the delivery of public services supporting trade • Build African capacity for trade policy formulation and implementation • Strengthen the enabling environment for African businesses. The USAID-funded Trade Hubs have contributed substantially to improving the enabling environment for U.S. two-way trade with Sub-Saharan Africa (SSA); empowered African small and medium enterprises (SME) to increase exports to U.S., regional and international markets; and facilitated regional economic integration. Implementation has conformed overall to Presidential initiative objectives and shown the creativity, flexibility and ingenuity to overcome difficult operating environments and uncertain funding levels. A more detailed description of the Trade Hubs’ activities can be found here78. Given a renewed commitment to the ongoing work of the Trade Hubs, and the Administration’s focus on increasing regional trade under the Feed the Future Initiative, findings from this evaluation will directly inform several priority projects as well as the already sizeable portfolio of ongoing USAID trade capacity building projects ($510 million in FY2010). In addition, the evaluation will be directly responsive to repeated reports from the U.S. Government Accountability Office describing weaknesses in USAID’s monitoring and evaluation of trade capacity building projects. 2.http://egateg.usaid.gov/sites/default/files/African%20Trade%20Hub%20Best%20Practices%20Review%20(USAID,2010).pdf Africa Trade Hubs Evaluation Report 63 C.3. TRADE HUB ASSISTANCE PROGRAM A. Goal: The USAID Trade Hubs’ overall goal is to help African companies compete in international markets through direct technical assistance and export consulting and by addressing barriers to trade that individual companies cannot overcome on their own, such as inefficient road transport and lack of access to finance. The proposed evaluation focuses only on export promotion activities. B. Development Hypothesis: As the Global Competitiveness Trade Hubs provide increased technical assistance focused on export promotion, these activities will result in (1) increased exports, (2) robust job creation, and (3) enhanced productivity in the trade sectors of the respective countries. C. Implementation Structure: Since 2002, three sub-regional Global Competitiveness Hubs, or West Africa (WA), East & Central Africa (ECA) and Southern Africa (SA) Trade Hubs, have been the primary implementers and regional focal points of USAID-sponsored trade capacity building (TCB) assistance. The implementing organizations are Carana Corporation (WA), Chemonics International (ECA), and AECOM (SA) while the beneficiaries have included over 4,175 firms across the African Continent exporting goods internationally from a dozen different sectors. Period of Funding: FY2002 - 2010 Funding: $270 million Region: West Africa (WA), East Central Africa (ECA), Southern Africa (SA) Contractors: Carana (WA), Chemonics (ECA), and AECOM (SA) Websites: www.watradehub.com, www.competeafrica.org, www.satradehub.org • West Africa: From 2003-2010, the USAID West Africa Trade Hub staff provided technical assistance in export promotion activities to over 2,742 businesses. Additionally, the Trade Hub had established relationships with over 1,000 professional buyers. Firms receiving technical assistance were exporting goods in the following sectors: Specialty Foods, Cashews, Shea, Home Décor, Apparel, and Fish & Seafood. In total dollar value for FY 2003-2011 the USAID West Africa Trade Hub provided approximately $23 million in export promotion assistance. • East/Central Africa: From 2007 to 2010, the East/Central Africa Hub assisted 509 firms in the following sectors: Textiles/Apparel, Specialty Foods, Cut Flowers, Home Décor/Fashion Accessories. In total dollar value for FY 2009-2010, the USAID East/Central Africa Trade Hub (Compete) provided approximately $700,000 in export promotion technical assistance. • Southern Africa: From 2007-2010, the Southern Africa Trade Hub provided export promotion assistance to over 924 firms. These companies provide services in the following sectors: Agriculture, Specialty Food, Tourism, Apparel and Livestock. The total dollar value of export promotion technical assistance provided to the companies during this time period was approximately $5 million. 64 Africa Trade Hubs Evaluation Report C.4. EVALUATION A. Objective: This evaluation will assess the effectiveness of Trade Hubs’ export assistance projects, embedded within the Trade Hubs’ broader mission, and further, will inform project design, reporting parameters, and evaluation methodologies for future projects. B. Key Questions: Supported by relevant data and evidence, disaggregated by product (using Harmonized System Codes - HS Codes) and country to the extent feasible79, the evaluator will answer the following questions using quantitative and qualitative methods: 1. Firm Level Effects: Did exports/revenues increase as a result of USAID export assistance? Was there a statistically significant difference in export performance by assisted firms vs. overall regional/global export trends in those sectors/products (i.e. more than in entities that were not assisted)? Is there a statistically significant difference in export/revenue performance between firms that received assistance and those that did not? 2. Export Diversification: Has trade/export assistance delivered by the Trade Hubs helped to diversify trade in assisted sectors/firms? Was there a statistically significant difference in the export destination or export product mix of assisted firms in comparison to overall export trends in this product/sector? 3. Job Creation: Did the increase in exports/revenues lead to the creation of jobs in assisted firms? Is there evidence that this job creation exceeded overall job creation in the sector (i.e. exceed the number created in firms in the same area/industry that were not assisted)? Was there a statistically significant improvement in job creation in assisted firms from the period before the assistance began to the period following the assistance? 4. Expenditures: What is the relationship between USAID Trade Hub expenditures on export assistance and export/revenue and job creation performance (ratios of expended costs to exports/revenues/jobs generated)? To what extent can changes in export performance or job creation be attributed to USAID Trade Hub expenditures? 5. Sectoral Effects: Is trade/export assistance delivered by the Trade Hubs relatively more effective within specific product sectors (e.g. traditional agriculture products, nontraditional agriculture products, manufactured goods, services, etc.)? 6. Regional Effects: Has trade/export assistance delivered by the Trade Hubs been relatively more effective in generating exports/revenues/jobs in specific countries/regions or more effective in generating exports to certain destination markets over others (e.g. traditional agriculture products, nontraditional agriculture products, manufactured goods, services, etc.)? 79 Feasibility will depend largely on the amount and types of data collected by the Trade Hubs throughout project implementation as well as to what extent HS codes can be mapped to the specific sectors in which the Trade Hubs work. Africa Trade Hubs Evaluation Report 65 C. This evaluation will include both impact (IE) and performance evaluation (PE) components. SI will use an IE to answer evaluation Questions 1-3 for the West Africa Trade Hub. However, for the East/Central Africa Trade Hub, in which a reliable comparison group is not available, these questions will be answered through a rigorous mixed-method performance evaluation, which will include quantitative analysis of changes in major outcomes for beneficiary firms as well as several qualitative interviews with beneficiary firms. To the extent possible, SI will also seek to conduct a PE to identify any firm level effects resulting from Southern Africa Trade Hub activities. The SI team will rely on performance evaluation methods to answer Questions 4 through 6 for all three Trade Hubs. This will include quantitative analysis of expenditures by value chain by country for Question 4. The results from this analysis will be combined with findings from the firm level IE and PE analysis conducted for Questions 1 through 3. As such, SI will only be able to assess “attribution” of changes in export performance and job creation for West Africa Trade Hub expenditures (and only for firms in and around Accra). For the Eastern and Southern Africa Trade Hubs, on the other hand, SI will (to the extent possible) provide findings on the “contribution” of expenditures to changes in export performance and job creation outcomes. SI will rely on HS Code data (in correlation with data collection for Questions 1 through 3) to answer Questions 5 and 6. Finally, all evaluation questions will be supported by qualitative interviews with firms and other key informants to determine the how and why behind any quantitative findings. This evaluation is intended to give context and depth to existing macroeconomic information on Trade Hub activities by focusing on the decisions and outcomes of individual firms from across the spectrum of relevant productive sectors. To the greatest extent possible, the evaluator will capture detailed financial and export information at the firm level both to reconstruct a baseline (the year each firm began working with the Trade Hub or the year the firm began operations) as well as an end line for all three African Trade Hubs. However, the team will only be able to attempt to isolate the effects of the Trade Hub in West Africa (in and around Accra, only)80 due to the lack of available data on potential comparison firms in Eastern and Southern Africa. In order to gain a broad macro-level baseline perspective, a first step in the evaluation will be a trend analysis of African export activity using the Harmonized System (HS) Codes for the products most commonly exported by firms receiving Trade Hub assistance. This data is readily available from domestic and international sources, and further, is amenable to triangulation and verification using mirror import data, especially from the US and EU. To the extent possible, the HS Code trend analysis will integrate national, regional and global trends in order to ascertain broad baseline performance in each of the relevant sectors and products. The trend analysis will not only give context to the focused firm level analysis, but may also identify specific constraints and opportunities for expansion. However, there may be limitations to the extent that specific 80 There is not enough available data from the East Africa Trade Hub to pursue an evaluation design that would isolate the effects of Trade Hub technical assistance. A suitable comparison group could not be identified. 66 Africa Trade Hubs Evaluation Report Trade Hub sectors can be assessed using HS Code data since not all sectors have their own specific HS Code. As such, SI will only use HS Code data for Trade Hub sectors with direct codes. The firm level analysis will be based on data such as total assistance provided for export promotion by sector/firm, value of exports to external markets by sector/firm, numbers of jobs created due to exports by sector/firm, and sales figures collected from Trade Hub records, third party sources and firms themselves. However, as noted above, the analytical structure of this evaluation may be limited by the availability of control data as no baseline was put in place at the project’s inception. Most firm level data will be gathered through evaluation team visits to the field (with the exception of data from the Southern Africa Trade Hub), which will include setting up a survey and training enumerators to roll out that survey in Accra; conducting individual firm interviews in Accra, Addis Ababa, and Nairobi; conducting phone surveys with firms in East Africa; and gathering Trade Hub database data from both West and East Africa Trade Hubs. Additionally, other data limitations that may affect analysis include: (1) lack of data from the early years (2002-2007), (2) lack of comparable data collection instruments across the hubs (for example, one hub collected self-reported data in a questionnaire directly from each individual firm while another reported results by working through one intermediary), (3) limited data collection among assisted firms on the outcome variables upon which this evaluation focuses, (4) potentially low survey response rates or refusal by firms to answer questions related to their sales and exports, (5) the possibility that identified comparison firms in Accra are not similar enough to beneficiary firms in Accra to allow for the use of a quasi-experimental design, (6) the limited scope of field data collection (which will focus only on Accra in West Africa and mainly on Addis Ababa and Nairobi in East Africa) . In order to mitigate the potential limitations of the existing data, findings on firm level results will largely be focused in the areas of direct data collection – Accra, Addis Ababa, and Nairobi – but the team will expand the analysis to other areas as possible. Additionally, before completing the firm level IE analysis in West Africa, the evaluation team will compare and attempt to match beneficiary and potential comparison firms using a set of biographical characteristics (such as whether or not he firm exported, the general level of sales, years of existence as a firm, registration status, etc.) at baseline. However, should the team find that the beneficiary and comparison firms are too different from one another at that point, they may make a decision not to directly compare the two groups. Finally, to mitigate low survey response issues, the team will follow up with firms on multiple occasions and will provide confidentiality disclaimers to help respondents feel more comfortable with providing necessary information. The evaluators may choose to refine the research design, and will conduct the evaluation and deliver a report on the results with actionable recommendations for future trade and export promotion projects. These recommendations will add to, rather than simply reiterate, previous evaluation work conducted by USAID and other donors. To the extent feasible, the team will utilize local consultants/firms in both of the regions to collect primary data from a representative sample of assisted and non-assisted firms. The team will have experience in evaluating economic growth and/or trade capacity building projects, particularly those in Sub-Saharan Africa. The team will have demonstrated skills conducting surveys, focus groups, economic analysis/cost-benefit analysis and advanced statistical analysis. They will consult project documents, deliverables and related documentation. The team will compare methodologies employed by the project with standard economic tools employed to measure TCB impacts in the context of international assistance projects. Africa Trade Hubs Evaluation Report 67 C.5. TASKS, DELIVERABLES AND TIMELINE The estimated period of performance for this task order is a total of twelve months, commencing on the date the contract is signed. A. Project Phases 1. Project Planning, Research Instrument Development and Desk Research This project requires substantial on-the-ground work to collect primary data from firms, the Trade Hubs, and third party entities. Phase 1 thus involves logistical planning to identify local vendors to administer surveys and conduct interviews, as well as methodological planning. The evaluator will construct a detailed methodological plan including: a comprehensive list of quantitative variables and qualitative data necessary to carry out the evaluation, a scientifically sound theoretical and statistical model through which to analyze data, and a valid survey and data collection instrument to obtain the necessary data. If possible, Phase 1 will also include the data collection and trend analysis of African export performance by relevant HS Code.81 2. Baseline Data Collection and Tabulation In order for the final analysis to reach scientifically valid conclusions, baseline data on treatment groups must be compiled in order to construct accurately the profile of comparison group firms. The quasi-experimental design thus requires the evaluator to compile the list of firms that received, via Trade Hub activities, technical assistance for the purpose of export promotion, and to tabulate their relevant characteristics in order to target relevant comparison group firms. It is expected that the evaluator will work closely with USAID Trade Hubs to compile preliminary data on individual firms, sectors of production, firm size, demographic characteristics of the entrepreneur, and any other readily available information that may account for relevant differences in performance between treatment and comparison group firms. This data will inform the number and types of control group firms to be surveyed in order to generate a correctly stratified, representative82 and scientifically valid dataset from which to draw conclusions. Phase 2 will also include gathering for each relevant country and/or sector information on events external to any individual firm that may have impacted economic performance, such as war, irregular changes in governance, adverse weather conditions, etc. 3. Firm Selection and Primary Data Collection Building on the results of Phase 2, the evaluator will work with local vendors to identify the list of treatment and comparison group firms to be surveyed and interviewed, based on analysis of the tabulations from Phase 2, in order to construct a representative and 81 This may depend on knowledge about necessary HS Codes, which may have to be gathered from the field. 82 The database for West Africa will only be representative of firms in and around Accra. 68 Africa Trade Hubs Evaluation Report scientifically valid dataset, if possible.83 In Phase 3, the evaluator will work with local vendors to carry out the primary data collection from treatment and comparison group firms, utilizing the instrument developed in Phase 1. Quantitative data collected will, to the extent possible, be stored in a standard spreadsheet format, easily readable by major statistical analysis packages. Qualitative data will be organized by topic and be stored in a keyword searchable format. 4. Data Analysis, Report Writing and Revisions The evaluator in Phase 4 will employ the statistical methodology elaborated in Phase 1 to analyze the collected data. The usual tests for robustness and appropriateness of statistical techniques will be conducted. Qualitative data will be integrated into the analysis to give richness and dimension to quantitative findings or explain anomalous statistical results. The final report will be a thorough analysis of export performance by firms receiving Trade Hub technical assistance versus relevant control groups,84 written to be broadly comprehensible and used as a relevant document in planning future Trade Hub and technical assistance projects. B. Work Plan and Evaluation Design: A Work Plan and Evaluation Design for the evaluation shall be completed by the lead evaluator within three weeks of the award of the contract and presented to the Contracting Officer Representative (COR). The evaluation design will include a detailed evaluation design matrix (including the key questions, the methods and data sources used to address each question), draft questionnaires and other data collection instruments, and known limitations to the evaluation design. The final design requires COR approval. The work plan will include the anticipated schedule and logistical arrangements and delineate the roles and responsibilities of members of the evaluation team. C. Reports and Deliverables In addition to the requirements for submission of reports in Section F.3-Reports and Deliverables or Outputs of the basic IQC and in accordance with AIDAR clause 752.242-70, Periodic Progress Reports, the Contractor shall submit reports, deliverables or outputs as further described below to the COR. All reports and other deliverables shall be in the English language, unless otherwise specified by the COR. • Oral Briefings - The evaluation team will meet with USAID upon arrival in country for data collection. The team will also provide an oral briefing of its findings and recommendations to the USAID senior management and economic growth office prior to departure. • A draft final report (for USAID acceptance/review) 83 See note 6 above. 84 Firm level data analysis will only be conducted in West Africa for firms in and around Accra, and, in East Africa, no comparison group will be used. Africa Trade Hubs Evaluation Report 69 • A final report incorporating edits (Not To Exceed (NTE) 50 pages excluding appendices) • The annexes to the report shall include: 1) The Evaluation Scope of Work; 2) Any “statements of differences” regarding significant unresolved difference of opinion by funders, implementers, and/or members of the evaluation team; 3) All tools used in conducting the evaluation, such as questionnaires, checklists, survey instruments, and discussion guides; 4) Sources of information, properly identified and listed; and 5) Disclosure of conflicts of interest forms for all evaluation team members, either attesting to a lack of conflict of interest or describing existing conflict of interest. • All data collected as part of the evaluation (logically organized in excel or other mutually agreed format) • An executive summary of no more than 5 pages highlighting key findings of the evaluation along with specific recommendations to address these weaknesses in future programming. • A one page (front and back) fact sheet about the evaluation and findings/recommendations • A power point presentation (NTE 25 slides) covering the key findings/recommendations from the evaluation. The implementing partner will deliver this presentation to USAID following completion of the evaluation. • A standard Monitoring and Evaluation template for collecting baseline data and tracking export promotion assistance activities that could be adopted by all the Trade Hubs to inform a future evaluation of export promotion projects. This would include a list of all data elements that should be captured to justify exports/sales/jobs generated, the most plausible method of collecting that data (survey techniques, etc.), frequency of data collection, baseline data required to inform a future impact evaluation, and an evaluation methodology/timeline to assess impact. D. An acceptable report will meet the following requirements as per USAID policy (evaluators should refer to the USAID Evaluation Policy)85. • The evaluation report should represent a thoughtful, well-researched and well organized effort to objectively evaluate what worked, what did not, and why. • The evaluation report should address all evaluation questions included in the Scope of Work. 85 http://www.usaid.gov/evaluation/USAIDEvaluationPolicy.pdf 70 Africa Trade Hubs Evaluation Report • The evaluation report should include this Scope of Work as an Annex. All modifications to the Scope of Work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline shall be agreed upon in writing by USAID. • Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex to the final report. • Evaluation findings will assess outcomes and impacts using gender disaggregated data. • Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). • Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. • Findings should be specific, concise and supported by strong quantitative or qualitative evidence. • Sources of information need to be properly identified and listed in an Annex, including a list of all individuals interviewed (interviewers must receive the permission of the individual). • Recommendations need to be supported by a specific set of findings. • Recommendations should be action-oriented, practical and specific, with defined responsibility for the action. Africa Trade Hubs Evaluation Report 71 ANNEX II: WEST AFRICA EVALUATION METHODS AND LIMITATIONS Evaluators interviewed a sample of firms in Accra, Ghana, using a convenience sample of firms that received trade assistance and a complementary convenience sample of firms that applied for trade assistance but were not selected. Specifically, all firms that received assistance and for which contact information was available were contacted for participation. The total sample size was 127 firms, of which 83 received trade assistance in some capacity from the West Africa Trade Hub; the remainder did not. These firms participated in both designated Africa Trade Hubs value chains and in non-designated value chains. The list of value chains designated and interviewed included: • Apparel • Cashews • Fish and seafood • Home décor and fashion accessories, including furniture • Shea and cosmetics • Specialty foods • Textiles Value chains in West Africa are defined according to criteria that are relevant to project objectives. The universe of firms that might potentially participate in trade assistance projects might thus not know without consulting project officers whether they belong in the value chain to be assisted. The absence of a clear value chain participation criterion impedes the definition of the universe of firms from which our sample is drawn, making it difficult to characterize the universe of firms that our results might represent statistically. A variety of tests to determine whether the firm belongs to an export value chain all have limitations. The direct exporter criterion does not adequately characterize the value chain. The evaluators might ask firms whether they directly export any of the commodities or products in the Trade Hubs trade assistance scope of work. A firm might very well participate in an export value chain without directly exporting products that belong in the value chain, such as agricultural commodities, handicrafts, and specialty foods. The wholesalers and exporters that directly export products might have a large network of suppliers and buyers that the exporter considers vendors and thus part of the value chain. An indirect export criterion will not solve this problem. Firms may not know the ultimate destination of their wares when they hit the market. Distributors may purchase an array of products and qualities variously bound for domestic and international markets without explaining those destinations to the primary producers. The export commodity criterion fails in two specific ways. First, the universe of firms that produce a ubiquitous good such as cashews would vastly overstate the number of firms participating in the export value chain, since a majority of these firms and a majority of the tonnage produced never enters the markets where export-supply vendors make purchases. Since the product is essentially for local consumption, there is no reason to treat these domestic producers as if they are part of the population of firms that might potentially be helped by trade assistance. The second way in which the export commodity criterion fails is that firms often do not categorize their products in ways that are consistent with customs criteria. A single exporter might include a wide variety of textiles, apparel, furniture, wood carvings and other woven products that span a large variety 72 Africa Trade Hubs Evaluation Report of export commodities. Specialty foods could similarly span a large number of agricultural products and processed foods. Firms may have difficulty answering whether their primary line of business is exactly equal to a value chain mentioned, or a subset of that value chain, or a larger sector that includes the value chain in question, or one that partially overlaps with the value chain in question. Depending on the means of production and distribution, breaking out sales for the value chain in question versus the total firm sales may be difficult to do. Breaking out sector-specific shares of costs and employment may be even more difficult. Two important consequences arise from this problem. The first is to what extent the results of the evaluation generalize. The second is to what extent national statistics on economic output, international trade and employment can be used as effective benchmarks for the performance of firms in this study. It is difficult to call the sample anything other than a convenience sample, given the lack of clarity surrounding which of the population of firms might have received trade assistance based on the project objectives. Additionally, no well-specified counterfactual exists to inform the development hypothesis. The policy relevance of the evaluation’s statistical findings would be greatest if there were a statistical basis for claiming that, based on the sample of firms that did receive trade assistance, the likely effect of trade assistance on firms that did not receive it would have been some related quantity. However, what that effect would have been from the convenience sample cannot be determined. To illustrate the above: sectoral effects were extremely important to firm sales, exports, profitability and employment, and industry was crucial to the selection of our firms. Therefore, disentangling the effects of industry from trade assistance itself will be difficult. It is sometimes possible to use bootstrapping to establish that the distribution of sampled firms converges to the distribution of non￾sampled firms for key characteristics such as firm size, employment and productivity. However, without clear definitions of the relevant population of firms, a bootstrapping approach is not possible. The second consequence is to what extent national statistics on economic output, international trade and employment can be used as effective benchmarks for the performance of firms in this study. National statistical agencies estimate employment, output, productivity and other key quantities related to input-output analysis for the economy as a whole and for specific industry sectors. Assuming that a random sample existed and that the performance of sectors varied somewhat during the study window, it could be possible to benchmark firm performance to more relevant industry measures instead. For example, the end of the multi-fiber arrangement in 2005 had detrimental effects on many incumbents in the global apparel and textile industries. Low-cost producers suffering from trade quotas found that they gained new market share, which ultimately came at the expense of existing firms exposed to harsh price competition. As a benchmark for output, employment and productivity, then, the sectoral statistics are preferable for the national statistics if our sample is explicitly sectoral (which it is). However, the differences in industry definitions between our sample and the economic statistics make such benchmarks unreliable. The original list of value chains to be investigated in the West Africa Trade Hubs project included several anomalies. Several value chains to be investigated in the project were not represented at all in the dataset. Those absent value chains included cut flowers, livestock, tourism and wood products. Several value chains could not be distinguished on the basis of survey data obtained. Those commingled value chains included home décor and fashion accessories, furniture, and the new category “handicrafts.” Finally a number of firms responded that their value chain was “general” exports, essentially putting them in a separate category that was irrelevant to the evaluated value chains. Africa Trade Hubs Evaluation Report 73 Absence of a Control Group The most significant limitation of the convenience sample is the absence of any control group in the study. It was thus determined that the terminology “comparison group” more accurately reflected the counterfactual. The West African comparison group consists of firms that applied for trade assistance and failed to receive it. The USAID selection criterion for inclusion in the Africa Trade Hubs trade assistance project is an evaluation of export readiness. In short, both explicit and implicit criteria for treated firms should predispose them toward superior performance in exports, with uncertain effects on employment (suspect a positive correlation) and productivity (suspect a positive correlation, even net of weak positive effects on employment). Comparison group firms were overwhelmingly composed of failed applicants to the WATH project. Therefore, in terms of propensity to receive trade assistance, the firms were largely equivalent on the basis of a willingness to participate. This could result from any combination of aptitude, skill, motivation, political connections, ambition and ability to garner competitive intelligence. However, the WATH explicitly rejected firms not deemed export-ready, suggesting crucial gaps in the firms’ ability to produce inventory of sufficient quality for export. Therefore, the difference in performance on key variables such as output, exports, employment and productivity between treatment and comparison firms would have been expected to differ dramatically, whether or not the trade assistance was provided at all. As a result, the accurate interpretation of findings is How did the treated firms fare relative to their untreated counterparts? rather than How did trade assistance produce improved outcomes among candidates for trade assistance? No amount of statistical machinery can overcome this basic consequence of the research design. Given this circumstance and the wide variability in firm size, growth trajectories and productivity rises (and falls), the evaluation statistician recommended nonparametric approaches to characterizing differences in subgroups. Multivariate regression in most forms seeks to measure the marginal impact of a unit of treatment in creating an outcome of interest. Given that differences in output, exports, employment and productivity do not fundamentally represent marginal outcomes attributable to changes in inputs (such as trade assistance), those marginal effects are misleading and often entirely spurious. The differences in track records among different groups, either in aggregate or as a percentage change, are the concrete results that the design and evaluation allow for. There is no additional benefit to analyzing the expected or predicted value of exports, sales, employment or productivity based on measures of participation in trade assistance, because the research design cannot identify effects attributable to trade assistance. Development Hypothesis Some inconsistency in approach weakened our development hypothesis. In the canonical regression, the trade assistance variable is to be a measure X of assistance received, and a set of outcomes Y that reflect the efficacy of that assistance according to project objectives. Controlling for other variables Z that could also produce variance in Y, firms with more benefits X should produce more favorable results Y than firms with less X. The marginal effect of (1) inclusion in a training project, (2) access to concessional finance, (3) introductions with buyers in export markets, (4) logistical assistance, (5) improved infrastructure or any other type of trade assistance should produce greater improvement of outcomes over the window of the study than for comparable non-treated firms. From simple univariate OLS to fancier techniques such as difference-in-differences analysis, fixed and random effects, propensity score matching, and two-stage least squares, the aim of econometrics as a 74 Africa Trade Hubs Evaluation Report whole is to measure by how much treated firms exceed their untreated counterparts, at least in expectation. The team evaluated on a regular basis whether spillovers could be evidence of the efficacy of trade assistance. By the team’s definition, any transfer of capacity through replication of techniques, acquisition of human resources, piggybacking on marketing efforts and mimicking of financial strategies was both part of the project’s objective and a secondary development hypothesis. Carried to its extreme, perfect diffusion of techniques, financing, marketing, logistics, training and human resources would create identical gains in performance across all firms within the radius of diffusion. Depending on the model to be evaluated, an evaluation could look for such effects within an urban area, a national economy, a regional value chain or any other sensible way of hypothesizing that economic and financial techniques become relevant, feasible and worthy of replication. This is essentially a free parameter in the study design. This introduces a perverse problem for econometrics. The team hypothesized that the treatment firm’s difference in Y from the untreated firms is the statistical basis for estimating the direct effects of trade assistance. Then, it was assumed that the shared difference in performance is the basis for estimating the indirect effects of trade assistance. By this definition, it is statistically impossible to distinguish between a trade assistance project with zero direct effects and infinitely fast diffusion across treated and untreated firms. In other words, it is assumed that the trade assistance is effective by definition, and the team is looking to estimate whether its benefit tends to accrue mostly to directly participating firms or their competitors. With these two development hypotheses in mind, no pair of firms in the dataset can gauge the effect of trade assistance in its direct and indirect forms together, especially if sectors and national economies differ for reasons other than the presence of trade assistance. Limitations of the Dataset The data had four principal shortcomings. The first problem with the data was the erroneous treatment of zeros in the dataset. Due to the flexibility of question logic in the questionnaire, it was normal that a large number of questions would be appropriately left blank if the question did not apply to the individual in question. Refusals to answer specific questions were properly coded with 99 and 999 entries. However, zeros were uniformly left blank, and when queried, the enumerators replied that in all cases, blank cells referred to zeros and not missing data. Eighty-five percent of the cells in the raw data in Excel format were blank. The vast majority of these should have been missing data and not zeros. This presents a significant management problem, as most statistical measures of interest required differentiation between zeros and missing observations. Averages and (with that, many missing data points) even medians are sensitive to rampant misreporting of data. To the extent that the team wished to analyze averages of the entire group or specific cohorts (e.g., by value chain, by trade assistance participation, etc.), the addition of a large number of spurious zeros into the dataset would affect those cohorts. As the cohorts are not of equal size, the zeros would directly bias cohort averages. Therefore, while firms with zeros in both baseline and endline data points were excluded, some significant gaps existed in the ability to distinguish firms in the cases (1 vs. 2 and 3 vs. 4) in Table 14. Africa Trade Hubs Evaluation Report 75 Table 14: Inferential Problems That Result from Missing Data Identifying properly coded missing data from question refusals was trivial; however, the absence of 99 and 999 codings, which are typical of SPSS for most question refusals, rendered this strategy moot. Forty-four cells in the entire dataset used these codings for question refusals, equivalent to 0.04% (1/25 of one percent) of the total number of possible data points, and 0.26% (one quarter of one percent) of all the clearly non-missing data provided by the enumerators. Many of the blank cells correctly reflect questions not asked and data not collected, while others reflect improper data entry. The second shortcoming of the dataset was the poorly structured factor variables. The value chains and years of firms’ participation should have had a 1:1 correspondence with the master list of value chains for which assistance was provided, but this was not the case. A large number of firms reported doing business in handicrafts, which might relate to any of the following value chains: home décor and fashion accessories, furniture, apparel and even wood products. Some degree of judgment had to be used in coding the firms’ value chain participation into discrete categories. Due to the small size of the dataset, statistical approaches to validation and cross-checking were unnecessary, but nonetheless the data reflect judgments of three independent individuals: the survey enumerator, the survey respondent, and the evaluation team. The third shortcoming of the dataset was the poor documentation of key variables, such as the specific types of sales that executives were meant to report during the interviews. Several firms reported that baseline sales were lower than baseline exports. This could either indicate a data entry problem, conflicting data or a misinterpretation that exports need not be a subset of sales. Documentation for the survey enumerator indicated that sales and exports should come from the same value chain in the same year, but that exports should refer to the portion of sales ultimately destined for export. The fourth and final shortcoming of the data was endline-only measurement. Since no baseline data were collected in the actual baseline year, the baseline data were reconstructed retroactively through the recollections and records of surviving firms. This introduced multiple biases, including survivorship bias, low incidence and recall bias of 10-year-old baseline data. CASES DEPENDENT VARIABLE BASELINE YEAR DEPENDENT VARIABLE ENDLINE YEAR CORRECT INFERENCE CASE 1 Zero Nonzero Market entry; or new employer CASE 2 Missing Nonzero No inference CASE 3 Nonzero Zero Market exit; or mass layoffs CASE 4 Nonzero Missing No inference 76 Africa Trade Hubs Evaluation Report Variables Calculated Baseline Year The default baseline and endline years were 2001 and 2011. However, not all firms could report during this window for a variety of reasons. For treated firms, the baseline year of reporting was the year prior to joining the Africa Trade Hubs project. A firm that received first assistance in 2006 would report baseline data in 2005, so that the baseline and endline would exactly encompass the number of years of trade assistance received, presuming no attrition from the project. In a multivariate regression setting, which was thought to be the methodology at the outset of the analysis, this would be the simplest way to structure a difference￾in-differences regression. In fact, only a tenth of the treated firms joined in the first year of the project, and approximately half joined halfway through the 10-year period studied. For untreated firms, the default baseline year was the year 2001, which was assumed to be the year prior to enrollment for most treated firms in the sample. Untreated firms that were formed after the baseline year reported their first year of operations as the baseline year. Endline Year Firms reported three years of endline data: 2010, 2011 and 2012. Employment, exports, sales and productivity were averaged over that three-year period and reported as if they were 2011 annual data. Sales Firms reported total sales during each of the baseline and endline years in local currency. Where reported sales were lower than exports, it was understood as an error of interpretation such that sales meant non-export sales. Therefore, exports were added to sales to estimate true sales for that period. For each firm, the evaluation team calculated the period growth in sales, the annualized growth in sales and the percentage change in sales. Firms with zero exports in the baseline year were not suitable for inclusion in regressions using percentage changes as the dependent variable. Exports Exports were valued in local currency and reported in baseline and endline years, separately for each value chain. For each firm, the evaluation team calculated the period growth in exports, the annualized growth in exports and the percentage change in exports. Firms with zero exports in the baseline year were not suitable for inclusion in regressions using percentage changes as the dependent variable. Employment Employment was reported in part-time and full-time employee head counts by sex for both baseline and endline years. From these raw data, the evaluation team estimated full-time equivalents as full-time employees, plus half the head count of part-time employees. Dataset questions did not address the time contributions of full- and part-time employees. Finally, using separate full time employment (FTE) tallies for male and female workers in baseline and endline years, the share of women’s employment at baseline and endline was calculated, consistent with the language in the scope of work. For each firm the evaluation team calculated the period growth in employment, the annualized growth in employment, and the percentage change in employment. Firms with zero exports in the baseline year were not suitable for inclusion in regressions using percentage changes as the dependent variable. Africa Trade Hubs Evaluation Report 77 Female Ownership Firms reported the head counts of male and female ownership. Female ownership was treated as a percentage of ownership by head count as opposed to equity stake. Types of Assistance The data collection instrument checked more than 20 different types of trade assistance received. Firms answered whether they received any WATH assistance in the following areas, and whether managers thought those assistance projects had a relatively strong effect on the firm’s export performance: • Financing • Trade shows • Connecting to buyers • Connecting to similar firms to reach buyers • Product design and development • Packaging, labeling and branding • Marketing and promotional materials • Practical business information • Logistical issues, such as visas and legal paperwork • Partnering • Food safety standards training • Traceability systems training • Certification scheme training • Sustainability and legality issue training • Technical training for factory personnel • Cost and pricing strategy training • Training on AGOA benefits • Training on access to finance • Training on customs documentation • ICT training • Other (free text) National Benchmarks ATH Sectors The list of sectors assisted is generated here from product documents: apparel, cashews, cut flowers, fashion design, fish and seafood, furniture, home décor and fashion accessories, livestock, shea and cosmetics, specialty foods, textiles, tourism and wood products. Data Sources The International Trade Centre (ITC) aggregates and standardizes nationally reported trade data according to HS codes that correspond to a commodity. HS codes can be specified from broadest to narrowest with 2-, 4-, 6-, or 10-digit precision. These codes are hierarchical, meaning that the code 010203 would fall under commodity code 01 at the broadest level of detail, sub-commodity 02 within 78 Africa Trade Hubs Evaluation Report that category and eventually sub-sub-category 03 within that. Subcategories are numbered sequentially beginning from 01. Data Coverage The table below indicates the best available match for desired data. The WATH sectors are not equally covered by ITC partner trade data. Some sectors reflect a single two-digit HS code; others a single four￾digit code; others multiple four-digit codes. Some appear to have no obvious analogs in the merchandise trade data. Five of the assisted sectors have good ITC data to reflect performance. Others are too broad, too narrow, or simply inadequate and therefore not reported. The sectors with adequate data appear to be Apparel, Cashews, Cut Flowers, Fish and Seafood and Furniture. While the actual effects of the WATH project might be too small to show up at a national level in apparel, for example, the symmetry between the proposed industry sectors treated and the national level statistical data reported is promising. Several sectors might be feasible with further specification: Livestock, Shea and Cosmetics, Textiles and Wood Products. These statistical categories are much broader than the sectors described by the WATH project materials. Finally, some categories do not lend themselves well to comparison with UN Comtrade data sources, such as trade in services and end-user descriptions of product functions (as opposed to product type and details of manufacture). It is unlikely that any statistical data exist on Home Décor and Fashion Design. Tourism will require different statistical sources, not reported in the same manner as UN Comtrade. Specialty Foods may be a product category that is meaningful in the private sector, but it does not appear to be recognized in trade statistics. Africa Trade Hubs Evaluation Report 79 ATH sector HS Code Description Coverage Apparel 61 62 Articles of apparel and clothing accessories, knitted or crocheted Articles of apparel and clothing accessories, not knitted or crocheted Adequate Cashews 080131 080132 Cashew nuts, in shell Cashew nuts, shelled Adequate Cut flowers 0603 Cut flowers of a kind suitable for bouquets Adequate Fashion design NA Inadequate Fish & seafood 03 Fish and seafood Adequate Furniture 94 Furniture; bedding, mattresses, cushions and similar stuffed furnishing Adequate Home décor 6302 6303 6304 Bed linen, table linen, toilet linen and kitchen linen Curtains, blinds, curtain or blind valences Other furnishing articles, excluding those of 94.04 Too narrow Livestock 01 Live animals Too broad Shea & cosmetics 33 Essential oils and resinoids; perfumery or cosmetic preparations Too broad Specialty foods NA Recommend we do not use “Misc food preparations” Inadequate Textiles 63 Other made textile articles, sets, worn clothing, etc. Too broad Tourism NA Inadequate Wood products 44 Wood and articles of wood; wood charcoal Too broad World Development Indicators (WDI) The WDI dataset is the primary open access repository of internationally recognized statistics on macroeconomic performance, such as trends in GDP growth, labor force participation and trade statistics over time. WDI GDP statistics and labor force participation were used over the evaluation window. Inferential Statistics One can see that the treatment and comparison groups have different aggregate output, employment, exports and productivity. Similarly, the averages of output, employment, exports and productivity differ between treatment and comparison groups. The same differences are apparent between groups identified by primary value chain participation. 80 Africa Trade Hubs Evaluation Report Several independent methods exist to evaluate whether the differences among these aggregates and averages differ more than expected if treatment and comparison groups were drawn from the same distribution. One method is regression analysis, which, for aforementioned reasons, was avoided where possible. A second method is using a family of nonparametric tests, which tests whether means and variances of firm sales, firm output, firm employment and firm productivity differ enough to suggest that they were drawn from different distributions. The null hypothesis for this family of tests is that the distributions are identical, and the alternative hypothesis is that the two groups have systematically different means and variances. In the bivariate setting, the Wilcoxon or Mann-Whitney test evaluates whether two samples have different means, but the nonparametric specification relaxes the assumption of normality among the samples. This accommodation is crucially important because firm size is generally not normally distributed, and specifically not so in the evaluation sample. This test was used for the treatment and comparison group comparisons, wherein each group constituted a subsample; the test returned a positive response whenever the null hypothesis (identical means for treatment and comparison groups) could be rejected with 95% confidence. In the multivariate setting, each value chain subsample constituted an independent group. A broadly similar nonparametric test, the Kruskal-Wallis test, returned positive results whenever at least one of the value chains systematically differed (e.g., in employment, sales, exports and productivity) from the remainder of the groups. That is, a positive Kruskal-Wallis test rejects the null hypothesis that value chains all experienced approximately the same changes in outcomes during the sample window, within the 95% confidence interval. The nonparametric test is appropriate in that it makes no assumptions about the normality of the underlying distributions. Twelve firms claimed participation in more than one value chain. Some analysis looked at the firm level, judging each firm by its total employment and total annual sales. In these analyses, firms were identified only by the primary value chain, as determined by maximum exports when more than one value chain was reported. Analysis of exports looked specifically at the firm-sector dyad, meaning that a firm with more than one export sector would have two entries in the dataset. A firm with no export sectors would not appear in the firm-sector dyad dataset at all. Multivariate Regression Regression analysis was broadly inconclusive. With a small sample size, using firm fixed effects specifications and anything more than a univariate specification (e.g., whether a firm received trade assistance at all or how many years of trade assistance were received) rapidly exhausted the power of the dataset. To account for the zero initial exports problem, the evaluation statistician selected a two-stage regression model that explained market baseline and endline zeros with a logit specification, and then a log-linear model to account for observed percentage changes. The results were broadly inconclusive for all two-stage regression models of that form. This two-stage regression model gave maximal inferential power from the limited number of observations available. For example, the evaluation team estimated a first-stage logit model explaining whether treated or comparison firms were significantly more likely to enter the export market following the baseline year. This provided some information about whether trade assistance was useful in encouraging new exporters. Then, a second-stage logarithmic-linear model was estimated to investigate whether the annual rate of change in export growth differed significantly among treatment and comparison groups, and if so by how much. Africa Trade Hubs Evaluation Report 81 Despite clear differences among treatment and comparison groups that were validated by nonparametric testing, the specificity of the effects was insufficient to precisely estimate nonzero marginal coefficients of trade assistance on the outcomes of interest. That finding was consistent across many different specifications of outcomes (exports, sales, employment and productivity) and many different specifications of trade assistance (binary assistance variable vs. a factor variable of types of assistance potentially received). Statistical Analysis Firm Entry and Exit The frequency tables below analyze whether firms entered and left the market in large numbers. The tables show that: • The number of new entrants is very small • The number of exits is greater than the number of new entrants • Trade assistance has no apparent effect on these outcomes An additional 12 observations are excluded from this analysis, which restricts firms to the primary line of exports, even though a substantial minority did in fact export in more than one value chain. Frequency Tables for Market Entry and Exit > # There was no apparent influence of trade assistance on market entry. > table.expstart newexports tradeassist FALSE TRUE Control 42 2 Treatment 81 2 > # There was no apparent influence of trade on market exit. > table.expcease ceaseexports tradeassist FALSE TRUE Control 39 5 Treatment 76 7 > # There was no apparent influence of trade assistance on new sellers. > table.entrant entrants 82 Africa Trade Hubs Evaluation Report tradeassist FALSE TRUE Control 34 10 Treatment 57 26 Regressions for Market Entry and Exit Null hypothesis: Treated and comparison firms entered/left the market at about the same rate. Alternative hypothesis: Treated firms entered/left the market at different rates from comparison firms. Logistic regressions compute the odds ratio that an outcome of interest will occur for a given individual, all else being equal. The logistic regression provides no evidence to reject the null hypothesis. Univariate Trade Assistance Measures These regressions ask whether trade assistance created measurable effects on sales, exports, employment and productivity for treated firms. Across the board, the regression evidence is null. Trade assistance had no observable effect by the measure of OLS regression. Dependent variables are absolute changes in sales; absolute changes in exports; absolute changes in employment; and absolute changes in productivity over the baseline to endline period. The regressions are univariate, in that they only ask whether trade assistance changed; no other right￾hand side variables are used. The null hypothesis is always that treatment and comparison firms have identical average changes over the period from baseline to endline, and the alternative hypothesis is that they differ. Sales Sales are annual, firm-wide sales reported in cedis. Trade assistance is a binary variable taking the values “Treatment” for Africa Trade Hubs participants and “Control” for other firms. # Do sales rise with treatment? > lm01 <- lm(sales.tot.end - sales.tot.base ~ tradeassist, data=ghana.7) > summary(lm01) Call: lm(formula = sales.tot.end - sales.tot.base ~ tradeassist, data = ghana.7) Residuals: Min 1Q Median 3Q Max -380800352 -5241174 -5185600 864531 539152159 Africa Trade Hubs Evaluation Report 83 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) -866315 11323257 -0.077 0.939 tradeassistTreatment 6107489 14006646 0.436 0.664 Residual standard error: 75110000 on 125 degrees of freedom Multiple R-squared: 0.001519, Adjusted R-squared: -0.006469 F-statistic: 0.1901 on 1 and 125 DF, p-value: 0.6636 Employment Employment is full-time equivalent employees, equal to the sum of the full-time employee head count and half the part-time employee head count. There are no significant results. Exports Exports are the value in cedis of the primary export value chain of the firm, reported in cedis. This specification excludes some exports due to some firms’ participation in multiple value chains. Frequency tables reported elsewhere give complete results for all value chains. These results would be unlikely to change with the additional exports from excluded value chains. There are no significant results. Productivity Expcap2 is the ratio of endline productivity to baseline productivity. Values over 1 indicate a rise in productivity. Values under 1 indicate a fall in productivity. In a more careful econometric exercise, we would typically work with the log of productivity rises and falls to ensure sensible prediction of the equations. The tradeoff is that with logarithmic specifications, interpretation of coefficients is less intuitive. Since the regression design already has fatal flaws, it makes little difference whether we use the optimal functional form. There are no significant results. 84 Africa Trade Hubs Evaluation Report Bivariate Specification86 Firm size makes a difference largely in the context of regression to the mean. The bigger they are, the harder they fall. Similarly, a bit of growth from a very large firm can swamp robust growth on a percentage basis from a large number of small firms. The results from this regression are designed to differentiate between large and small firms’ contributions to growth, while discerning the effect of trade assistance. The most significant findings have to do with the track record of large firms between baseline and endline. No significant results are found for trade assistance. Our measure of firm size cuts baseline full-time equivalent employment into three bins: small firms of under 20 employees, medium firms of 20 to 99 employees and large firms. Sales and Firm Size Large firms suffered greater falls in growth than other firms. Employment and Firm Size This regression is highly subject to autocorrelation. Large firms fell more in terms of employment as well as sales. Exports and Firm Size Large firms saw an apparent rise in exports despite the massive fall in sales. Productivity and Firm Size No meaningful trends differentiated large and small firms with regard to productivity growth. Type of Assistance Firms could receive several types of assistance, and in fact often received many types in addition to Africa Trade Hubs Assistance. • Meeting buyers • State-owned banks • Government agencies • Private firms • USAID trade assistance • Multilateral bodies • Other non-governmental organizations • Friends and family Each of these types of assistance is reported in addition to participation in Africa Trade Hubs, which retains its familiar “Treatment” and “Comparison” values. 86 The team also conducted regressions using multivariate specifications including controlling for baseline year but did not find any additional results. Africa Trade Hubs Evaluation Report 85 Sales and Type of Assistance Firms with big rises in sales tended not to receive multilateral assistance. The prototypical firm in this specification answers “Yes” to every type of aid except Africa Trade Hubs. Coefficients reflect the marginal contribution of the WATH assistance, but conversely the absence of other type of assistance relative to the prototypical firm. The coefficient on the WATH happens to be negative, but neither especially large nor near the range of significance. Employment and Type of Assistance Controlling for other types of assistance received, the regression indicates a correlation between falling employment and participation in the WATH. But it would not reject the null hypothesis that no correlation existed between the WATH participation and changes in employment. The prototypical firm in this specification answers “Yes” to every type of aid except WATH. Coefficients reflect the marginal contribution of the WATH assistance, but conversely the absence of other type of assistance relative to the prototypical firm. The coefficient on the WATH happens to be negative, but neither especially large nor near the range of significance. Exports and Type of Assistance Controlling for other types of assistance received, the regression indicates a correlation between falling exports and participation in the WATH. But it would not reject the null hypothesis that no correlation existed between the WATH participation and changes in exports. The prototypical firm in this specification answers “Yes” to every type of aid except Africa Trade Hubs. Coefficients reflect the marginal contribution of the WATH assistance, but conversely the absence of other type of assistance relative to the prototypical firm. The coefficient on the WATH happens to be negative, but neither especially large nor near the range of significance. Productivity and Type of Assistance Controlling for other types of assistance received, the regression indicates a correlation between rising export productivity and participation in the WATH. But it would not reject the null hypothesis that no correlation existed between the WATH participation and such changes. The prototypical firm in this specification answers “Yes” to every type of aid except Africa Trade Hubs. Coefficients reflect the marginal contribution of the WATH assistance, but conversely the absence of other type of assistance relative to the prototypical firm. The coefficient on the WATH happens to be negative, but neither especially large nor near the range of significance. Baseline and Founding Years Did the baseline years and founding years for firms appear to come from the same distribution for treatment and comparison groups? Null: No difference between means of the treatment and comparison groups. Alternative (founding): One group was founded systematically before the other. Alternative (baseline): One group began the study systematically before the other. Baseline Years The baseline years are systematically earlier for comparison firms. 86 Africa Trade Hubs Evaluation Report Founding Years There are differences between the founding years, but the statistical significance is marginal at best. Africa Trade Hubs Evaluation Report 87 ANNEX III: DATA COLLECTION INSTRUMENTS Trade Hub Staff Interview Guide A. Objective: This evaluation will assess the effectiveness of Trade Hubs’ export assistance projects, embedded within the Trade Hubs’ broader mission, and further, will inform project design, reporting parameters, and evaluation methodologies for future projects. B. Key Questions: 1. Can you please describe your job, and your unit within the Trade Hub project? 2. How has the work you have been involved in contributed to export promotion assistance through the Trade Hub project? 3. What have been the most remarkable results achieved in the export promotion assistance area? • What were the most successful value chains or sectors, and why? • What were the most dynamic destination markets, and why? • What were the elements of Trade Hub export promotion assistance that proved most effective in achieving those results, and why? 4. Were problems encountered that impeded effective project implementation in the export promotion assistance area? • If so, can you please describe what they were and which elements of Trade Hub export promotion assistance were impeded? • What measures did the project take to deal with these impediments, and were they successful? • What are there possible lessons learned that might be taken into account in design / implementation of future export promotion assistance activities? 5. Taking into account the level-of-effort or other project expenditures on various Trade Hub export promotion assistance elements, • Which would you say were the most cost-effective in producing tangible and sustainable results, and why? • Which would you say were least cost-effective, and why? 6. To what extent do you think Trade Hub firm level assistance has resulted in spillover effects (also impacting non-beneficiary firms in targeted value chains)? What evidence of these spillover effects have you seen? Or, what has prevented much spillover? 88 Africa Trade Hubs Evaluation Report Value-Chain Firm Interview Guide A. Objective: This evaluation will assess the effectiveness of Trade Hubs’ export assistance projects, embedded within the Trade Hubs’ broader mission, and further, will inform project design, reporting parameters, and evaluation methodologies for future projects. B. Key Questions 1. Background • Before we being talking about the Trade Hub, I am curious what major changes have occurred in your firm in the past five to ten years? What led to these changes, and what was the result? • When did the firm begin working with the Trade Hub (only for those not survey)? And why? • What types of assistance has the firm received from the Trade Hub? • What other support has the firm received from outside entities, and how has this effected the firm’s operations? 2. Export increases. • Did your sales and exports increase as a result of the export promotion assistance of the Trade Hub? • If so, what was the most significant element of the assistance in terms of achieving that increase? 3. Export Diversification. • Did you begin to produce additional product lines for export as a result of assistance of the Trade Hub? o If so, which have been the most successful? o What was the most significant element of the assistance in terms of achieving that success? • Did you begin to sell your product to additional countries as a result of assistance of the Trade Hub? o İf so, in which destination markets have you experienced the most rapid growth? o If so, what was the most significant element of the assistance in terms of achieving that diversification of export destination? 4. Employment and productivity. • If your sales and exports increased as a result of Trade Hub assistance, have you added new jobs to cope with the increased demand? If not, why not? o What proportion of these have gone to women? • Would you say that employee productivity has increased as a result of Trade Hub assistance? o If so, what was the reason? Africa Trade Hubs Evaluation Report 89 5. Competition. • Who is your competition? Has your competition increased among Ghanaian firms producing the same products for export? o If so, has it been a significant increase? Why or why not? o What, in your opinion, has led to this increase/lack of increase? • Who are your input suppliers? o Are there more of them competing to sell to you than before? o What, in your opinion, has led to this increase/lack of increase? • How about transporters? And buyers? o Are there more firms competing to transport or purchase your product? o What, in your opinion, has led to this increase/lack of increase? 6. Changes • Where do you think your firm would be today if it hadn’t received assistance from the Trade Hub? • What, if anything, could the Trade Hub improve upon, or how else might it better support your firm in increasing exports? • What major constraints prevent your firm from increasing sales and exports today? 90 Africa Trade Hubs Evaluation Report Key Informant Interview Guide A. Objective: This evaluation will assess the effectiveness of Trade Hubs’ export assistance projects, embedded within the Trade Hubs’ broader mission, and further, will inform project design, reporting parameters, and evaluation methodologies for future projects. B. Key Questions 1. Background and Overview. • What is your organization’s purpose? What are its major activities? • İn what way has your organization coordinated with or benefitted from the Trade Hub? • What is your overall view of the Trade Hub? • How has the Trade Hub impacted exports in the region? What about employment? • What does the Trade Hub do particularly well? • What could the Trade Hub improve upon? • How, if at all, would you recommend the Trade Hub adjust its programming in its next iteration? • How, if at all, does the Trade Hub address gender in its programming? 2. Export increases. • Have exports in the [Value Chain] value chain (or the “Trade Hub value chains” for interviews with other donor organizations) increased significantly during the past decade? o If so, what has led to these increases? If not, why not? o If so, has the Trade Hub contributed in a significant way to that increase through its export promotion assistance to firms within the value chain? o If so, what was the most significant element of the Trade Hub export promotion assistance in terms of achieving the increase? 3. Export Diversification. • Have producers in the [Value Chain] value chain begun to produce additional product lines during the past decade? o If so, which ones? o If so, what has led to these increases? If not, why not? o If so, has the Trade Hub contributed in a significant way to that increase through its export promotion assistance to firms within the value chain? o If so, what was the most significant element of the Trade Hub export promotion assistance in terms of achieving the increase? • Did producers in the [Value Chain] value chain begin to sell their product to additional countries during the past decade? o If so, which ones? o If so, what has led to these increases? If not, why not? o İf so, in which destination markets has the value chain experienced the most rapid growth? Africa Trade Hubs Evaluation Report 91 o If so, what was the most significant element of the Trade Hub export promotion assistance in terms of achieving that diversification of export destination? 4. Competition. • Has competition increased during the past decade among Ghanaian firms producing products in the [Value Chain] value chain? o If so, has it been a significant increase? o If so, what has led to this increase? If not, why not? • Has the number of input suppliers upstream in the [Value Chain] value chain increased during the past decade? o If so, has it been a significant increase? o If so, what has led to this increase? If not, why not? • Has the number of transporters of [Value Chain] value chain products increased during the past decade? o If so, has it been a significant increase? o If so, what has led to this increase? If not, why not? • How about buyers of [Value Chain] value chain products? o Has it been a significant increase? o If so, what has led to this increase? If not, why not? 5. Other Donors/NGOs • What other donor organizations/NGOs does your organization work with? • What do these other organizations do? • Is there any overlap in what they organizations do and what the Trade Hubs do? • What, if anything, could the Trade Hub learn from these other organizations? • What, if anything, could the Trade Hub teach these other organizations? 92 Africa Trade Hubs Evaluation Report West Africa Trade Hub Firm Level Questionnaire Africa Trade Hubs Evaluation Report 93 94 Africa Trade Hubs Evaluation Report Africa Trade Hubs Evaluation Report 95 96 Africa Trade Hubs Evaluation Report Africa Trade Hubs Evaluation Report 97 98 Africa Trade Hubs Evaluation Report Africa Trade Hubs Evaluation Report 99 100 Africa Trade Hubs Evaluation Report Africa Trade Hubs Evaluation Report 101 ANNEX IV: SOURCES OF INFORMATION Documents Reviewed Bearing Point, Inc. 2008. Appendix C – Business and Gender Details of Hub Client Companies (Extracted from a larger Bearing Point report). Carana Corporation. 2011. Exports, Employment and Incomes in West Africa. West Africa Trade Hub Technical Report NO. 39. Chemonics International, Inc.. 2009. Competitiveness and Trade Expansion (COMPETE) Project: First Year Work Plan (FY 09). Chemonics International, Inc. 2011. African Growth Opportunities Act (AGOA) in Action: Stories from COMPETE Partners and Beneficiaries. Chemonics International, Inc. 2013. Competitiveness and Trade Expansion (COMPETE) Project: 2012 Annual Progress Report (FY 12). Chemonics International, Inc. 2013. USAID East Africa Trade Hub 2013 Fact Sheet. Chemonics International, Inc.. 2013. Competitiveness and Trade Expansion (COMPETE) Project: Fifth Year Work Plan (FY 13). Development Alternatives, Inc. and the Nathan Group. 2011. African Trade Hub Best Practices Review: Building on Successes and Lessons Learned for the Next Generation of Trade Hubs. Fukunishi, Takahiro; Mayumi Murayama; Tatsufumi Yamagata and Akio Nishiura (2006), “Industrialization and Poverty Alleviation: Pro-Poor Industrialization Strategies Revisited,” United Nations Industrial Development Organization, Vienna. Gelb, Alan, Christian Meyer, and Vijaya Ramachandran (2013), “Does Poor Mean Cheap? A Comparative Look at Africa’s Industrial Labor Costs.” CGD Working Paper 325: Center for Global Development. Hageboeck, Molly. 2010. From Aid to Trade: Delivering Results; A Cross-Country Evaluation of USAID Trade Capacity Building. Levine, Ruth. 2011. USAID Evaluation Policy, USAID, 1300 Pennsylvania Avenue, NW, Washington, D.C. United States Agency for International Development (USAID). 2012. Request for Proposals No. SOL￾624-12-000005 – Trade Hub and African Partners Network (Trade Hub Network) Project. West Africa Trade Hub. 2012. FY 2012 Annual Report. World Trade Organization. 2013. Aid for Trade and Value Chains in Textiles and Apparel. Geneva, Switzerland. 102 Africa Trade Hubs Evaluation Report Persons Contacted Firm/Key Informant Organization Contact Name Position Type of Firm/ Organization Country African Cotton & Textiles Industries Federation (ACTIF) Rajeev Arora Association Kenya AGOA-Plus (Volunteers for Economic Growth - VEGA) Teklu Kidane and Teddy Wossenyeleh Other Donor Project (also supported by EATH) Ethiopia Almeda Amanuel Girmay Apparel/Textiles Ethiopia Alpha Knits Hiran Bid Apparel/Textiles Kenya Bezamar Hailegiorgis Demissie Specialty Foods Ethiopia Center for African Women Economic Empowerment (CAWEE) Nigest Haile Women's NGO Ethiopia EATH Trade Hubs Staff JC Mazinge Implementing Partner Regional EATH Trade Hubs Staff Finn Holm-Olsen Implementing Partner Regional Ethiopian Exporter's Association Hailegiorgis Demissie Association Ethiopia Ethiopian Leather Industry Development Institute (LIDI) Birkenesh Gonfa Government Org. Ethiopia Ethiopian Textile and Garment Manufacturing Association (ETGAMA) Teodros Worku Association Ethiopia Ethiopian Textile Industry Development Institute (TIDI) Yared Mesfin Government Org. Ethiopia Fikirte Addis Fikerte Addis Apparel/Textiles Ethiopia Flip Flop Recycling Julie Church Home Décor/ Handicrafts Kenya Global Apparel Narain Shahdadpuri Apparel/Textiles Kenya Katchy Kollections Jennifer Mulli Home Décor/ Handicrafts Kenya Kenyan Export Processing Zone Authority John Akara Authority Kenya Mafi Designs Mahlet Afework Apparel/Textiles Ethiopia Muya Ethiopia Sara Abera and Jacques Dubois Home Décor/ Handicrafts Ethiopia Novastar Mohamed Umer Apparel/Textiles Ethiopia Tikur Abbay Abebe Teklu Leather Ethiopia Trademark Scott Allen Other Donor Project Kenya Africa Trade Hubs Evaluation Report 103 Firm/Key Informant Organization Contact Name Position Type of Firm/ Organization Country USAID/East Africa Matt Rees Client Regional USAID/Ethiopia Lazarre Potier Client Ethiopia USAID/West Africa Brinton Bohling Chief, Trade and Investment Office Client Ghana USAID/West Africa Evelyn Rekia Ayivor Project Management Specialist, Trade and Investment Office Client Ghana West Africa Trade Hub Sara Angleka Deputy Director Client Ghana West Africa Trade Hub Elane Bellezza Trade Hub Home Décor and Fashion Accessories Advisor Ghana West Africa Trade Hub Abou Fall AGOA Services Manager Ghana West Africa Trade Hub Makhtar Thiam Dakar Hub Director Senegal Global Shea Alliance Joseph Funt Managing Director Ghana Africa Cashew Alliance Christian Dahm Managing Director Ghana Africa Cashew Alliance Sunil Dahiya Business Advisor Ghana African Cotton & Textiles Industries Federation (ACTIF) Rajeev Arora Kenya Government of Ghana Gerald Nyarko￾Mensah Director of Agriculture, Ministry of Trade and Industry Ghana Ghana Export Promotion Authority Maxwell Osei-Kusi Director Ghana Ghana Export Promotion Authority Alexander Dadzawa Head, Agriculture, Marketing and Promotion Division Ghana African Cashew Initiative Mary Adzanyo Director, Private Sector Development Ghana World Bank Business Development Support Fund Eric Agyare Deputy Team Leader Ghana 104 Africa Trade Hubs Evaluation Report Databases ECA Trade Hub Companies Assisted to Export Institution/ Company Country Type of Assistance Provided Results Sector Contact Person Email Address Tel 1. TECHNICAL ASSISTANCE 2. INWARD BUYER'S VISIT 3. TRADE SHOW 4.OTHER INCLUDING MATCHMAKING ADDIS GARMENT Ethiopia 123 Coveralls Buyer Apparel Ms. Giuliana Zuccato addisgsc@ethionet.et +251 11 371 5216 ALMEDA Ethiopia 123 Several Million USD Repeated Business Apparel M. Kidane Mariam almeda-3@ethionet.et +251 347 711 483 AYKA ADDIS Ethiopia 12 Buyers Visit Apparel M. Zvika Shamir zvishamir@gmail.com +972 523 957 145 ETGAMA Ethiopia 4 Tech Assistance Apparel M. Teodros etgama@ethionet.et +251 11 442 3978 GMM Ethiopia 1 New US Client Apparel Ms. Genet Kebede info@gmmgar.com +251 11 419 7374 MAA GARMENT Ethiopia 123 Significant Technical Improvement in Knitting Apparel M. Fassil Tadesse midroc.finance@ethionet.et +251 11 552 0575 MULAT GARMENT Ethiopia 13 New US Clients Apparel M. Abera Mulat mulata@ethionet.et +251 11 440 3615 NAZRETH GARMENT Ethiopia 34 Matchmaking Apparel M. Kassaye Mekuria dametto@ethionet.et +251 11 551 6880 NOVASTAR GARMENT Ethiopia 123 Superior Uniform Apparel M. Mohamed Umer mumer@novastargarment.com +251 11 445 0222 VISION TEXTILE Ethiopia 4 Matchmaking Apparel Ms. Ozturk birgul@visiontextile.net +251 910 460 667 Africa Trade Hubs Evaluation Report 105 ECA Trade Hub Companies Assisted to Export Institution/ Company Country Type of Assistance Provided Results Sector Contact Person Email Address Tel KOMBOLCH A Ethiopia 24 Buyers Visit Apparel M. Mustofa Jemal kte@ethionet.et +251 33 551 0202 FIKIRTE ADDIS Ethiopia 14 Fashion Show Apparel Ms. Fikirte fikirtea@yefikirdesign.com ??? MAFI DESIGNS Ethiopia 14 Fashion Show Apparel Ms. Mahlet Afework (Mafi) mafi.eth@gmail.com ??? AFRICA APPAREL Kenya 2 Buyers Visits Apparel M. Pankaj Shaudhari pankaj@aaepz.com +254 202 359 531 ALPHA KNITS LTD Kenya 4 Matchmaking/ Trade Show Apparel M.Hiran Bid info@alphaknits.com +254 72 76 00 63 ASHTON APPAREL LTD Kenya 14 Matchmaking Apparel M. Amiji amiji@atraco.ae +971 48 81 2686 FINE SPINNERS LTD Kenya 12 Buyers Visits Apparel M. Jas Bedi jasbedi@finespinners.com +254 20 556 144 BONK Kenya 1 Introduction to US Market Apparel M. Fady Rostom fady@bonk.co.ke +254 722 868 348 EPZ Kenya 34 Buyers Visits Apparel Ms. Margaret Waithaka mwaithaka@epzakenya.com +254 722 332 234 GARMENT LABELS LTD Kenya 14 Matchmaking Apparel M. Patrick Kariuki pkariuki@garmentlabels.co.ke +254 722 709 922 GLOBAL APPAREL Kenya 13 UPS Business Apparel M. Narain narain@swiftkenya.com +254 45 22 720 KAPRIC APPARELS LTD Kenya 3 Work in Progress Apparel M. Thomas Puthor thomas@shaasafari.com +254 723 022 331 UNITED Kenya 123 US Repeated Business Apparel M. Pankaj Bedi penkajbedi@unitedaryen.net +254 286 1950 106 Africa Trade Hubs Evaluation Report ECA Trade Hub Companies Assisted to Export Institution/ Company Country Type of Assistance Provided Results Sector Contact Person Email Address Tel ARYAN VIVA AFRICA Kenya 4 Matchmaking Apparel Ms. Viva Bedi viva@bedi.com +254 51 22 12 320 KONDAKIS Kenya 234 Trade Show/ Matchmaking/ Buyer Mission Apparel Ms. Nike Kondakis nike@kondakis.biz +254 737 115 507 Ki2 Kenya 34 Trade Show/ Matchmaking Apparel Ms. Rachel info@ki2fashion.com +254 722 689 491 CANDYTEX Mauritius 24 Matchmaking Apparel M. Anill Joyram anilljoyram@candytex.mu +230 254 3444 CENTURY KNITTING Mauritius 14 Preparation to the US Market Apparel M. Raj Pothiah century@intnet.mu +230 240 8640 CHAMPION INDUSTRIES LTD Mauritius 14 Preparation to the US Market Apparel M. Robert Ng N/A +230 233 4210 CIEL GROUP Mauritius 234 US Repeated Orders Apparel M. Michel Mayer Mmayer@cdl-fabrics.com +230 602 3800 ESQUEL Mauritius 4 Matchmaking Apparel M. Hemraj Ramnial ramh@esquel.com +230 401 9850 DALLAS GARMENT Mauritius 2 Trade Show Apparel M. Ramjen Cader Allee allee.dalas@intnet.mu +230 454 3283 DENIM DE L'ILE LTD Mauritius 4 Matchmaking Apparel M. Vigier de Latour mauricev@ddi.mu +230 412 5190 ENTERPRISE MAURITIUS Mauritius 4 MOU Apparel M. Dev Chamroo dev.chamroo@em.intnet.mu +230 212 9760 FIREMOUNT TEXTILES LTD Mauritius 123 Trade Show Apparel M. Soma Sekharan soma.s@firemount.mu +230 283 2431 Africa Trade Hubs Evaluation Report 107 ECA Trade Hub Companies Assisted to Export Institution/ Company Country Type of Assistance Provided Results Sector Contact Person Email Address Tel FIT-U GARMENT LTD Mauritius 2 Trade Show Apparel Ms. Jenny Pidial fit.u@intnet.mu +230 433 3653 FLOREAL KNITWEAR Mauritius 2 Trade Show Apparel M. Joel Couve jcouve@floreal.intnet.mu +230 686 39 95 HIMAVAN GARMENTS Mauritius 1 Preparation to the US Market Apparel M. V. Ancharaz himvkc@yahoo.com +230 418 4400 JACK TELLOR INTERNATIO NAL Mauritius 12 Trade Show Apparel M. Abdul Ravate tellor@intnet.mu +230 286 2163 KARINA Mauritius 12 Trade Show Apparel Ms. Joyce Lo joyce.lo@karina￾international.com +230 686 8102 MAXIWEAR LTD Mauritius 1 Preparation to the US Market Apparel M. Brij Mohabirsingh maxiwear@intnet.mu +230 264 9640 MAZAVA Mauritius 1 Matchmaking Apparel Ms. Ning Chaoming molly@mapwear.biz +230 779 0041 PALMAR Mauritius 12 US Repeated Orders Apparel M. Guillaume Heller gheller@palmar.intnet.mu +230 401 7033 PROMINTEX CO. LTD Mauritius 1 Preparation to the US Market Apparel M. Sam Somna dsomna@yahoo.com +230 637 9325 RT KNITS Mauritius 12 Trade Show Apparel M. Jean Li Wan Po jean.l@rtknits.com +230 401 6888 SHIVANI Mauritius 13 Matchmaking Apparel M. Subhas Ramchurn subhasramchurn @shivanimanufacturing.com +230 601 8524 ST MALO EXPORTS LTD Mauritius 12 US Repeated Orders Apparel M. Robert Koon smgroup@intnet.mu +230 247 2264 108 Africa Trade Hubs Evaluation Report ECA Trade Hub Companies Assisted to Export Institution/ Company Country Type of Assistance Provided Results Sector Contact Person Email Address Tel STAR KNITWEAR GROUP Mauritius 12 US Repeated Orders Apparel M. Ahmed Parkar reception @starknitweargroupltd.com +230 412 7418 TARA KNITWEAR LTD Mauritius 1 Preparation to the US Market Apparel Ms. Nancy Wong Mi nancywongmin @taragroup.intnet.mu +230 212 3715 TEXTO Mauritius 1 Matchmaking Apparel Ms. Lili Chung Lili.chung@textolimitee.com +230 241 9232 WORLD KNITS LTD Mauritius 14 Matchmaking Apparel M. Adrien Noel wkl@wknits.com +230 403 8552 A TO Z Tanzania 3 Superior Uniform Apparel SUNFLAG Tanzania 4 Unknown Apparel M. Binu Sahadevan binu@sunflag-tz.com +255 27 250 6303 KIBOTRADE Tanzania 34 Apparel M. Bo Raahauge Rasmussen raahauge@gmail.com +255 22 2861787 M. Iver Rosenkrantz iver@urudiamonds.com +255 755 800 428 PHENIX Uganda 12 US Repeated Orders Apparel M. Mitsuo Tamada gm.phenix@bushnet,net +256 41 434 4227 SOUTHERN RANGE NYANZA LTD Uganda 12 US Repeated Orders Apparel M. Richard Mubiru legal@picfare.com +256 41 230 416 Ethiopia Horticulture Producers and Exporters Ethiopia 34 Trade Show Floriculture Solomon Sebehatu Gebreyesus menaflow@ethionet.et Africa Trade Hubs Evaluation Report 109 ECA Trade Hub Companies Assisted to Export Institution/ Company Country Type of Assistance Provided Results Sector Contact Person Email Address Tel Association Golden Rose Agro Farms Ltd Ethiopia 34 Trade Show Floriculture Alim Abdulhamid Jamal Shamji gomba@ethionet.et Jobera Flowers Ethiopia 34 Trade Show Floriculture Michael Asres Michael@joberaflowers.com Menagesha Flowers Ethiopia 34 Trade Show Floriculture Solomon Sebhatu menaflow@ethionet.et East African Growers Kenya 34 Trade Show Floriculture Peeush Mahajan peeush@eaga.co.ke KNET Flowers Kenya 34 Trade Show Floriculture Mike King’ori mike@k-netflowers.com Valentine Flower Growers Co Ltd Kenya 34 Trade Show Floriculture Joseph Kamau joseph@valentine-flowers.com Suera Flowers Kenya 34 Trade Show Floriculture Susan Wangui Mureithi md@suerafarm.sgc.co.ke Agrex Ltd Mauritius 34 Trade Show Floriculture Alban Doger de Speville agrex@intnet.mu Association of Producers & Exporters of Horticultural Products of Mauritius Mauritius 34 Trade Show Floriculture Raifa Bundhun apexhom@intnet.mu World Mauritius 34 Trade Show Floriculture Jean Felix jfelix@worldtropicals.mu 110 Africa Trade Hubs Evaluation Report ECA Trade Hub Companies Assisted to Export Institution/ Company Country Type of Assistance Provided Results Sector Contact Person Email Address Tel Tropicals Mount Meru Flowers Ltd Tanzania 34 Trade Show Floriculture Heikki Niskala office@mount-meru￾flowers.com Tanzania Horticulture Association Tanzania 34 Trade Show Floriculture Jacqueline Mkindi taha@habari.co.tz Frigoken Kenya 134 Preparation for US Market/Trade Show Specialty Food S. Dharmarajan pfil@peptang.com Ten Senses Africa Kenya 134 Preparation for US Market/Trade Show Specialty Food Dora Waruiru dwaruiru@ncck.org Browns Cheese Kenya 134 Preparation for US Market/Trade Show Specialty Food Andrew and Delia Stirling stirling@landmarktcn.com Premier Foods Kenya 134 Preparation for US Market/Trade Show Specialty Food S. Dharmarajan pfil@peptang.com Bezamar Ethiopia 134 Preparation for US Market/Trade Show Specialty Food Haile Demissie bezamar@ethionet.et Ecopia Ethiopia 134 Preparation for US Market/Trade Show Specialty Food Labourdonnais Mauritius 13 Technical Assistance/ Trade Show Specialty Food Reaz Gunga processing@ddl.mu Bourbon Vanilla Mauritius 13 Technical Assistance/ Trade Show Specialty Food David Barbera info@vaynilla.mu Sundaram Mauritius 1 Technical Assistance Specialty Food yoginee@sundaramspices.com Sukpak Mauritius 1 Technical Assistance Specialty Food sukpak@intnet.mu Africa Trade Hubs Evaluation Report 111 ECA Trade Hub Companies Assisted to Export Institution/ Company Country Type of Assistance Provided Results Sector Contact Person Email Address Tel NOGAMU Uganda 134 Preparation for US Market/Trade Show Specialty Food Charity Namuwoza cnamuwoza@nogamu.org.ug Nile Teas Uganda 134 Preparation for US Market/Trade Show Specialty Food Sammy Handmade Ethiopia 1234 Home Decor/Fashio n Accessories Sammy Abdella samabdella@yahoo.com Muya Ethiopia Ethiopia 234 Home Decor/Fashio n Accessories Sarah Abera and Constantine Mavrikios muyaethiopia@ethionet.et FOMI Ethiopia 24 Home Decor/ Fashion Accessories Afomia Tesfaye sales.fomi@gmail.com Menby Designs Ethiopia 24 Home Decor/ Fashion Accessories CAWEE Ethiopia 124 Grantee/Inward Buyers Mission Home Decor/ Fashion Accessories Nigest Haile cawee@ethionet.et Undugu Society Kenya 1234 Trade Show/ Matchmaking/ Inward Buyer Mission Home Decor/ Fashion Accessories Fred Masinde fred.masinde @undugufairtrade.co.ke 112 Africa Trade Hubs Evaluation Report ECA Trade Hub Companies Assisted to Export Institution/ Company Country Type of Assistance Provided Results Sector Contact Person Email Address Tel Mohazo Kenya 1234 Trade Show/ Matchmaking/ Inward Buyer Mission Home Decor/ Fashion Accessories Zohra Baraka zohra@mohazo.co.ke 254 720 307 675 Leakey Collection Kenya 14 Inward Buyer Mission/ Matchmaking Home Decor/ Fashion Accessories Katy Leakey leakeycollection@uuplus .com Katchy Kollections Kenya 14 Inward Buyer Mission/ Matchmaking Home Decor/ Fashion Accessories Jennifer Mulli info@katchy-kollections.com 254 736 751 740 Kenana Knitters Kenya 3 Trade Show Home Decor/ Fashion Accessories Patty Nightingale kenana.knitters@gmail.com African Heritage Kenya 234 Home Decor/ Fashion Accessories Makena Mwiraria makena@africanheritage .co.ke UniquEco Kenya 24 Home Decor/ Fashion Accessories Julie Church julie@uniqueco-designs.com Gahaya Links Rwanda 1234 Trade Show/ Matchmaking/ Inward Buyer Mission Home Decor/ Fashion Accessories Janet and Joy Nkubana gahayalinks@yahoo.com Africa Trade Hubs Evaluation Report 113 ECA Trade Hub Companies Assisted to Export Institution/ Company Country Type of Assistance Provided Results Sector Contact Person Email Address Tel Indego Africa Rwanda 134 Trade Show/ Matchmaking Home Decor/ Fashion Accessories Sarah Manion severalmeerkats@gmail.com Marvelous Batik Tanzania 1234 Trade Show/ Matchmaking/ Inward Buyer Mission Home Decor/ Fashion Accessories Flotea Massawe marvbatik@yahoo.com Footloose Tanzania 1234 Trade Show/ Matchmaking/ Inward Buyer Mission Home Decor/ Fashion Accessories Doreen Mashika Tanzania 1234 Trade Show/ Matchmaking/ Inward Buyer Mission Home Decor/ Fashion Accessories Doreen Mashika doreenmashika@hotmail .com Wrap up Africa Uganda 1234 Trade Show/ Matchmaking/ Inward Buyer Mission Home Decor/ Fashion Accessories Letha Sandison lethasandison@yahoo.com Uganda Crafts Ltd. Uganda 1234 Trade Show/ Matchmaking/ Inward Buyer Mission Home Decor/ Fashion Accessories Betty Kinene ugandacrafts2000ltd@yahoo.co m One Mango Tree Uganda 1234 Trade Show/ Matchmaking/ Inward Buyer Mission Home Decor/ Fashion Accessories Halle Butvin halle@onemangotree.com 114 Africa Trade Hubs Evaluation Report ECA Trade Hub Companies Assisted to Export Institution/ Company Country Type of Assistance Provided Results Sector Contact Person Email Address Tel Invisible Children Uganda 234 Trade Show/ Matchmaking/ Inward Buyer Mission Home Decor/ Fashion Accessories Sseko Designs Uganda 1234 Trade Show/ Matchmaking/ Inward Buyer Mission Home Decor/ Fashion Accessories Brent Boutwell brent@ssekodesigns.com Africa Trade Hubs Evaluation Report 115 ECA Trade Hubs Survey Database Number Year when Firm was established/started operations Year when Firm first started working with the ECA Trade Hub Countries Firm exported to BEFORE the year it started to work with Trade Hub Sales by the Firm BEFORE the year it started to work with Trade Hub Exports by the Firm BEFORE the year it started to work with Trade Hub Number of workers employed by the Firm BEFORE the year Firm started to work with Trade Hub % of FEMALE employees BEFORE the year Firm started to work with Trade Hub % of MALE employees BEFORE the year Firm started to work with Trade Hub Base Year Countries Exported to at Baseline Total Sales in Dollars (Base Year) % Export (Base Year) Total Number of Employees (Base Year) % of Male Employees (Base Year) % of Female Employees (Base Year) 1 1980 2009 Italy 800,000 100% 300 85% 15% 2002 None 800,000 0% 300 20% 80% 2 2000 2009 None 5,000,000 0% 5,000 80% 20% 2002 None 0 0% 2,000 25% 75% 3 2009 2011 Germany 12,000,000 100% 2,000 90% 10% 2009 Germany; Turkey 12,000,000 100% 2,000 20% 80% 4 2004 2009 Italy 250,000 50% 650 80% 20% 2004 Italy 250,000 50% 650 20% 80% 5 2005 2009 Germany, Italy, UK, USA 10,000,000 50% 1500 80% 20% 2005 Germany; USA 1,000,000 40% 350 15% 85% 6 2000 2011 USA 500,000 100% 120 90% 10% 2002 None 250,000 0% 150 30% 70% 7 1980 2009 None 0 0% 500 85% 15% 2002 None 1,000,000 0% 500 30% 70% 8 2005 2010 USA 2,000,000 100% 400 85% 15% 2005 USA <2000000 90% 220 5% 95% 9 2000 2009 Italy; USA 500,000 100% 120 85% 15% 2002 Italy; USA 500,000 100% 120 15% 85% 10 1986 2011 None 1,500,000 0% 500 50% 50% 2002 None 2,500,000 0% 800 30% 70% 11 2008 2009 None 10,000 0% 1 100% 0% 2008 None 10,000 0% 1 0% 100 % 12 2009 2009 None 20,000 0% 1 100% 0% 2009 None 20,000 0% 1 0% 100 % 116 Africa Trade Hubs Evaluation Report ECA Trade Hubs Survey Database Number Year when Firm was established/started operations Year when Firm first started working with the ECA Trade Hub Countries Firm exported to BEFORE the year it started to work with Trade Hub Sales by the Firm BEFORE the year it started to work with Trade Hub Exports by the Firm BEFORE the year it started to work with Trade Hub Number of workers employed by the Firm BEFORE the year Firm started to work with Trade Hub % of FEMALE employees BEFORE the year Firm started to work with Trade Hub % of MALE employees BEFORE the year Firm started to work with Trade Hub Base Year Countries Exported to at Baseline Total Sales in Dollars (Base Year) % Export (Base Year) Total Number of Employees (Base Year) % of Male Employees (Base Year) % of Female Employees (Base Year) 13 2007 2009 USA 7,776,117 100% 888 66% 34% 2007 USA 7,776,117 100% 888 34% 66% 14 2003 2007 UK, COMESA 3,614,457 40% 500 50% 50% 2003 US; UK <5000000 50% 650 25% 75% 15 2009 2009 None 15,000 0% 3 100% 0% 2009 None 15,000 0% 3 0% 100 % 16 2001 2009 USA <4000000 0 100% 1,100 80% 20% 2002 USA <40000000 100% 1,100 20% 80% 17 2002 2010 USA 11,000,000 100% 1,000 90% 10% 2002 USA 11,000,000 100% 1,000 10% 90% 18 2002 2003 UK, Spain, Uganda, Tanzania, USA 14,000,000 50% 500 80% 20% 2002 UK, Spain, Uganda, Tanzania, USA 14,000,000 50% 500 20% 80% 19 2002 2011 None 0 0% 1 100% 0% 2002 None 0 0% 1 0% 100 % 20 2002 2009 USA 1,500,000 100% 200 50% 50% 2002 USA 1,500,000 100% 200 50% 50% 21 2000 2009 None 1,000,000 0% 100 50% 50% 2002 None 1,000,000 0% 60 50% 50% Africa Trade Hubs Evaluation Report 117 ECA Trade Hubs Survey Database Number Year when Firm was established/started operations Year when Firm first started working with the ECA Trade Hub Countries Firm exported to BEFORE the year it started to work with Trade Hub Sales by the Firm BEFORE the year it started to work with Trade Hub Exports by the Firm BEFORE the year it started to work with Trade Hub Number of workers employed by the Firm BEFORE the year Firm started to work with Trade Hub % of FEMALE employees BEFORE the year Firm started to work with Trade Hub % of MALE employees BEFORE the year Firm started to work with Trade Hub Base Year Countries Exported to at Baseline Total Sales in Dollars (Base Year) % Export (Base Year) Total Number of Employees (Base Year) % of Male Employees (Base Year) % of Female Employees (Base Year) 22 1980 2011 UK, France, Spain 200,000,00 0 100% 2,500 70% 30% 2002 France, Germany, UK, Italy, Spain, USA 200,000,000 100% 2,500 30% 70% 23 1985 2011 USA 200,000,00 0 100% 12,000 70% 30% 2002 USA 200,000,000 100% 6,000 30% 70% 24 2000 2009 None 300,000 0% 70 70% 30% 2002 None 500,000 0% 100 30% 70% 25 2000 2009 UK, France 20,000,000 100% 1,500 50% 50% 2002 France, UK, USA 10,000,000 100% 600 50% 50% 26 2000 2009 UK, USA 10,000,000 100% 800 50% 50% 2002 UK, Germany, USA 10,000,000 100% 500 50% 50% 27 1995 2005 France, UK, Holland 1,000,000 100% 200 85% 15% 2002 UK, USA 1,000,000 100% 200 15% 85% 28 1980 2009 UK, France, USA 50,000,000 100% 6,500 70% 30% 2002 France, Germany, UK, Italy, Spain, USA 30,000,000 100% 5,000 30% 70% 29 1985 2009 UK, France 400,000 100% 80 50% 50% 2002 France 500,000 100% 80 50% 50% 118 Africa Trade Hubs Evaluation Report ECA Trade Hubs Survey Database Number Year when Firm was established/started operations Year when Firm first started working with the ECA Trade Hub Countries Firm exported to BEFORE the year it started to work with Trade Hub Sales by the Firm BEFORE the year it started to work with Trade Hub Exports by the Firm BEFORE the year it started to work with Trade Hub Number of workers employed by the Firm BEFORE the year Firm started to work with Trade Hub % of FEMALE employees BEFORE the year Firm started to work with Trade Hub % of MALE employees BEFORE the year Firm started to work with Trade Hub Base Year Countries Exported to at Baseline Total Sales in Dollars (Base Year) % Export (Base Year) Total Number of Employees (Base Year) % of Male Employees (Base Year) % of Female Employees (Base Year) 30 2005 2009 France, UK, Canada 4,000,000 100% 1,000 85% 15% 2005 UK 2,000,000 100% 50 10% 90% 31 2004 2009 France, UK 10,000,000 100% 1,500 70% 30% 2004 France, Spain 5,000,000 100% 1,200 30% 70% 32 2000 2009 UK, Belgium 10,000,000 100% 500 70% 30% 2002 France, Belgium, UK, South Africa, USA 8,000,000 100% 300 30% 70% 33 1980 2009 France 15,000,000 100% 400 75% 25% 2002 France 10,000,000 100% 400 25% 75% 34 1985 2009 UK, Italy, France 4,000,000 100% 2,500 80% 20% 2002 UK, Germany, USA 20,000,000 100% 2,000 20% 80% 35 2000 2009 France, Italy, UK 5,000,000 100% 800 80% 20% 2002 UK 5,000,000 100% 800 20% 80% 36 2000 2009 France, UK, USA 5,000,000 100% 1,000 80% 20% 2002 France, Spain, Germany, USA 6,000,000 100% 500 30% 70% 37 2000 2009 UK, France, Spain 10,000,000 100% 2,200 70% 30% 2002 France, UK, USA 8,000,000 100% 1,500 30% 70% Africa Trade Hubs Evaluation Report 119 ECA Trade Hubs Survey Database Number Year when Firm was established/started operations Year when Firm first started working with the ECA Trade Hub Countries Firm exported to BEFORE the year it started to work with Trade Hub Sales by the Firm BEFORE the year it started to work with Trade Hub Exports by the Firm BEFORE the year it started to work with Trade Hub Number of workers employed by the Firm BEFORE the year Firm started to work with Trade Hub % of FEMALE employees BEFORE the year Firm started to work with Trade Hub % of MALE employees BEFORE the year Firm started to work with Trade Hub Base Year Countries Exported to at Baseline Total Sales in Dollars (Base Year) % Export (Base Year) Total Number of Employees (Base Year) % of Male Employees (Base Year) % of Female Employees (Base Year) 38 1996 2007 USA, Germany, UK, Dubai, South Africa 30,000,000 97% 300 67% 33% 2002 UK 5,000,000 95% 20 70% 30% 39 1998 2003 USA, Japan, Holland, Italy, Sweden, Germany, France, Australia, Dubai, Reunion 500,000 95% 10 50% 50% 2002 USA, Japan, Holland, Italy, Sweden, Germany, France, Australia, Dubai, Reunion 500,000 95% 10 50% 50% 40 1996 2003 Israel, Holland, UK, Germany 6,024,096 100% 200 80% 20% 2002 Israel, Holland, UK, Germany 6,024,096 100% 250 20% 80% 120 Africa Trade Hubs Evaluation Report ECA Trade Hubs Survey Database Number Year when Firm was established/started operations Year when Firm first started working with the ECA Trade Hub Countries Firm exported to BEFORE the year it started to work with Trade Hub Sales by the Firm BEFORE the year it started to work with Trade Hub Exports by the Firm BEFORE the year it started to work with Trade Hub Number of workers employed by the Firm BEFORE the year Firm started to work with Trade Hub % of FEMALE employees BEFORE the year Firm started to work with Trade Hub % of MALE employees BEFORE the year Firm started to work with Trade Hub Base Year Countries Exported to at Baseline Total Sales in Dollars (Base Year) % Export (Base Year) Total Number of Employees (Base Year) % of Male Employees (Base Year) % of Female Employees (Base Year) 41 1985 2010 USA, Canada, Italy, France, Dubai, Australia, Hong Kong, Japan 500,000 80% 30 83% 17% 2002 None 0 0% 25 8% 92% 42 1987 2011 USA, France, Germany, Japan, Australia, Reunion, Island, Seychelles 20,000 4% 9 44% 56% 2002 Seychelles, Madagascar, USA 4,000 5% 1 100% 0% 43 2005 2009 None 25,000 0% 20 75% 25% 2005 None 0 0% 48 33% 67% 44 1979 2010 Tanzania, South Sudan, Uganda, Rwanda 1,700,000 7% 45 18% 82% 2002 Tanzania, South Sudan, Uganda, Rwanda 1,700,000 7% 45 82% 18% Africa Trade Hubs Evaluation Report 121 ECA Trade Hubs Survey Database Number Year when Firm was established/started operations Year when Firm first started working with the ECA Trade Hub Countries Firm exported to BEFORE the year it started to work with Trade Hub Sales by the Firm BEFORE the year it started to work with Trade Hub Exports by the Firm BEFORE the year it started to work with Trade Hub Number of workers employed by the Firm BEFORE the year Firm started to work with Trade Hub % of FEMALE employees BEFORE the year Firm started to work with Trade Hub % of MALE employees BEFORE the year Firm started to work with Trade Hub Base Year Countries Exported to at Baseline Total Sales in Dollars (Base Year) % Export (Base Year) Total Number of Employees (Base Year) % of Male Employees (Base Year) % of Female Employees (Base Year) 45 2001 2010 Germany, Switzerland, Netherlands, France, Denmark, Austria, Italy 36,000,000 40% 2 50% 50% 2002 UK 4,000,000 80% 2 1% 1% 46 2005 2009 None 0 0 4 90% 10% 2005 None 0 0% 4 90% 10% 47 1994 2009 USA, Canada, New Zealand, UK, Germany 500,000 100% 800 40% 60% 2002 UK, USA 80,000 40% 705 60% 40% 48 1987 2008 USA, Italy, Germany, Greece 150,000 90% 6 67% 33% 2002 US, UK 14,940,000 90% 510 25% 75% 49 2006 2011 Lebanon, South Africa 9,000 5% 15 80% 20% 2006 US 1,800,000 80% 54 1% 99% 50 2002 2003 None 1,900 0% 13 100% 0% 2002 None 1,900 0% 13 0% 100 % 122 Africa Trade Hubs Evaluation Report ECA Trade Hubs Survey Database Number Year when Firm was established/started operations Year when Firm first started working with the ECA Trade Hub Countries Firm exported to BEFORE the year it started to work with Trade Hub Sales by the Firm BEFORE the year it started to work with Trade Hub Exports by the Firm BEFORE the year it started to work with Trade Hub Number of workers employed by the Firm BEFORE the year Firm started to work with Trade Hub % of FEMALE employees BEFORE the year Firm started to work with Trade Hub % of MALE employees BEFORE the year Firm started to work with Trade Hub Base Year Countries Exported to at Baseline Total Sales in Dollars (Base Year) % Export (Base Year) Total Number of Employees (Base Year) % of Male Employees (Base Year) % of Female Employees (Base Year) 51 2003 2003 USA, France, Holland, UK 600,000 90% 500 90% 10% 2003 USA; France 600,000 90% 500 90% 10% 52 2004 2012 USA, Australia, Switzerland, Sweden 150,000 70% 35 40% 60% 2004 US 0 0% 47 17% 83% 53 1994 2004 USA 575000 95% 24 12% 88% 2002 US 200,000 95% 24 12% 88% 54 1992 2003 None 1,200 0% 1 100% 0% 2002 None 1,200 0% 1 0% 100 % 55 1999 2003 None 0 0% 5 60% 40% 2002 None 0 0% 5 40% 60% 56 2006 2009 None 12,000 0% 5 80% 20% 2006 None 12,000 0% 5 20% 80% 57 1983 2008 USA, Canada, Japan, South Africa 50,000 70% 355 85% 15% 2002 None 500 0% 25 24% 76% Africa Trade Hubs Evaluation Report 123 ECA Trade Hubs Survey Database Number Year when Firm was established/started operations Year when Firm first started working with the ECA Trade Hub Countries Firm exported to BEFORE the year it started to work with Trade Hub Sales by the Firm BEFORE the year it started to work with Trade Hub Exports by the Firm BEFORE the year it started to work with Trade Hub Number of workers employed by the Firm BEFORE the year Firm started to work with Trade Hub % of FEMALE employees BEFORE the year Firm started to work with Trade Hub % of MALE employees BEFORE the year Firm started to work with Trade Hub Base Year Countries Exported to at Baseline Total Sales in Dollars (Base Year) % Export (Base Year) Total Number of Employees (Base Year) % of Male Employees (Base Year) % of Female Employees (Base Year) 58 2009 2011 USA 39,000 98% 4 100% 0% 2009 USA 39,000 100% 4 0% 100 % 59 2002 2005 Switzerland, Australia 350,000 15% 6 70% 30% 2002 Switzerland; Australia; UK 20,000 15% 6 30% 70% 60 2005 2005 UK,USA 18,000 40 12 60% 40% 2005 UK,USA 18,000 40% 12 40% 60% 61 2005 2005 USA, France, Australia 945,280 82% 189 60% 40% 2005 USA, France, Australia 945,280 82% 189 60% 40% 124 Africa Trade Hubs Evaluation Report Number Total Sales in Dollars, Last 3 Years Total Sales in Dollars, Previous Year Total Sales in Dollars, Previous Year -1 Total Sales in Dollars, Previous Year -2 % Exported (Most Recent Year) Countries Exported to in 2012 Total No. of Employees (Current) % of Male Employees % of Female Employees Was Applicant Determined by Project to be 'Export Ready'? (Y/N) Was Product Design Technical Assistance/Training Provided? (Y/N) Was Business Planning/Access to Finance Technical Assistance/Training Provided? (Y/N) Was Market Linkages/Marketing Technical Assistance/Training Provided? (Y/N) Was Logistics Technical Assistance/Training Provided? (Y/N) Was Obtaining Certifications Technical Assistance/Training Provided? (Y/N) Was Environmental Issues/Compliance Technical Assistance/Training Provided? (Y/N) Was Other Technical Assistance/Training Provided? (Y/N) Old Hub? (Y/N) 1 3,000,000 1,200,000 1,000,000 800,000 90% USA, Germany 300 15% 85% Y Y N Y N Y N Y N 2 32,000,000 10,000,000 12,000,000 10,000,000 30% USA, Germany, UK, Turkey 5,000 20% 80% Y Y Y Y Y Y Y Y Y 3 132,000,000 70,000,000 50,000,000 12,000,000 100% Germany, Turkey, USA 5,000 10% 90% Y N N Y N Y Y Y N 4 2,800,000 1,000,000 1,000,000 800,000 100% Italy, USA 150 10% 90% Y Y Y Y N Y Y Y N 5 47,000,000 20,000,000 15,000,000 12,000,000 50% Germany, Italy, UK, USA 1,800 20% 80% Y Y N Y N Y Y Y Y 6 1,500,000 500,000 500,000 500,000 90% USA 120 10% 90% Y Y N Y N Y Y Y N 7 7,000,000 2,400,000 2,200,000 2,400,000 50% Germany, USA 500 15% 85% Y Y N Y N Y Y Y N 8 6,000,000 2,000,000 2,000,000 2,000,000 100% USA; Italy; Japan 640 15% 85% Y Y N Y N Y Y Y Y 9 2,200,000 800,000 800,000 600,000 100% Italy, USA 150 15% 85% Y Y N Y N Y Y Y N Africa Trade Hubs Evaluation Report 125 Number Total Sales in Dollars, Last 3 Years Total Sales in Dollars, Previous Year Total Sales in Dollars, Previous Year -1 Total Sales in Dollars, Previous Year -2 % Exported (Most Recent Year) Countries Exported to in 2012 Total No. of Employees (Current) % of Male Employees % of Female Employees Was Applicant Determined by Project to be 'Export Ready'? (Y/N) Was Product Design Technical Assistance/Training Provided? (Y/N) Was Business Planning/Access to Finance Technical Assistance/Training Provided? (Y/N) Was Market Linkages/Marketing Technical Assistance/Training Provided? (Y/N) Was Logistics Technical Assistance/Training Provided? (Y/N) Was Obtaining Certifications Technical Assistance/Training Provided? (Y/N) Was Environmental Issues/Compliance Technical Assistance/Training Provided? (Y/N) Was Other Technical Assistance/Training Provided? (Y/N) Old Hub? (Y/N) 10 5,500,000 2,000,000 2,000,000 1,500,000 10% USA 500 50% 50% Y Y N Y N Y Y Y N 11 200,000 80,000 60,000 60,000 30% USA 10 0% 100 % Y Y N Y N Y Y Y N 12 160,000 60,000 50,000 50,000 25% USA 4 0% 100 % Y Y N Y N Y Y Y N 13 75,000,000 25,000,000 25,000,000 25,000,000 100% UK, USA 3,000 30% 70% Y N N Y N N N Y N 14 15,000,000 5,000,000 5,000,000 5,000,000 50% UK, USA 300 50% 50% Y N N Y N N N Y Y 15 300,000 100,000 100,000 100,000 20% Uganda 3 0% 100 % Y N N Y N N N Y N 16 120,000,000 40,000,000 40,000,000 40,000,000 100% USA 1,200 20% 80% Y N N Y N N N Y N 17 240,000,000 80,000,000 80,000,000 80,000,000 100% USA 2,000 30% 70% Y N N Y N N N Y N 18 45,000,000 15,000,000 15,000,000 15,000,000 50% UK, Spain, Uganda, Tanzania, USA 2,500 30% 70% Y N N Y N N N Y Y 126 Africa Trade Hubs Evaluation Report Number Total Sales in Dollars, Last 3 Years Total Sales in Dollars, Previous Year Total Sales in Dollars, Previous Year -1 Total Sales in Dollars, Previous Year -2 % Exported (Most Recent Year) Countries Exported to in 2012 Total No. of Employees (Current) % of Male Employees % of Female Employees Was Applicant Determined by Project to be 'Export Ready'? (Y/N) Was Product Design Technical Assistance/Training Provided? (Y/N) Was Business Planning/Access to Finance Technical Assistance/Training Provided? (Y/N) Was Market Linkages/Marketing Technical Assistance/Training Provided? (Y/N) Was Logistics Technical Assistance/Training Provided? (Y/N) Was Obtaining Certifications Technical Assistance/Training Provided? (Y/N) Was Environmental Issues/Compliance Technical Assistance/Training Provided? (Y/N) Was Other Technical Assistance/Training Provided? (Y/N) Old Hub? (Y/N) 19 300,000 100,000 100,000 100,000 20% Denmark <10 0% 100 % Y N N Y N N N Y N 20 3,400,000 1,200,000 1,200,000 1,000,000 100% USA 150 70% 30% Y N N Y N N Y Y N 21 3,000,000 1,000,000 1,000,000 1,000,000 sells to other Maur itian comp anies France, UK, USA 100 50% 50% Y N N Y N N N Y N 22 788,000,000 288,000,000 250,000,000 250,000,000 100% France, Germany, UK, Italy, Spain, USA 28,20 0 30% 70% Y N N Y N N Y Y Y 23 753,577,000 253,577,000 250,000,000 250,000,000 100% USA >15, 000 30% 70% Y N N Y N N N Y Y 24 1,500,000 500,000 500,000 500,000 0% Nil 100 30% 70% N N N N N N N Y N 25 90,000,000 30,000,000 30,000,000 30,000,000 100% France, UK, USA 1,900 50% 50% Y N N Y N N N Y N Africa Trade Hubs Evaluation Report 127 Number Total Sales in Dollars, Last 3 Years Total Sales in Dollars, Previous Year Total Sales in Dollars, Previous Year -1 Total Sales in Dollars, Previous Year -2 % Exported (Most Recent Year) Countries Exported to in 2012 Total No. of Employees (Current) % of Male Employees % of Female Employees Was Applicant Determined by Project to be 'Export Ready'? (Y/N) Was Product Design Technical Assistance/Training Provided? (Y/N) Was Business Planning/Access to Finance Technical Assistance/Training Provided? (Y/N) Was Market Linkages/Marketing Technical Assistance/Training Provided? (Y/N) Was Logistics Technical Assistance/Training Provided? (Y/N) Was Obtaining Certifications Technical Assistance/Training Provided? (Y/N) Was Environmental Issues/Compliance Technical Assistance/Training Provided? (Y/N) Was Other Technical Assistance/Training Provided? (Y/N) Old Hub? (Y/N) 26 32,000,000 10,000,000 12,000,000 10,000,000 100% UK, Germany, USA 800 50% 50% Y N N Y N N N Y Y 27 3,000,000 1,000,000 1,000,000 1,000,000 100% UK, USA 200 15% 85% Y N N Y N N N Y N 28 150,000,000 50,000,000 50,000,000 50,000,000 100% France, Germany, UK, Italy, Spain, USA 6,500 30% 70% Y N N Y N N N Y N 29 1,500,000 500,000 500,000 500,000 100% France, USA 80 50% 50% Y Y N Y N N Y Y N 30 16,500,000 6,500,000 5,000,000 5,000,000 100% UK, Canada, USA 1,200 15% 85% Y N N Y N N N Y N 31 31,000,000 11,000,000 10,000,000 10,000,000 100% France, Spain, USA 1,600 30% 70% Y N N Y N N N Y Y 128 Africa Trade Hubs Evaluation Report Number Total Sales in Dollars, Last 3 Years Total Sales in Dollars, Previous Year Total Sales in Dollars, Previous Year -1 Total Sales in Dollars, Previous Year -2 % Exported (Most Recent Year) Countries Exported to in 2012 Total No. of Employees (Current) % of Male Employees % of Female Employees Was Applicant Determined by Project to be 'Export Ready'? (Y/N) Was Product Design Technical Assistance/Training Provided? (Y/N) Was Business Planning/Access to Finance Technical Assistance/Training Provided? (Y/N) Was Market Linkages/Marketing Technical Assistance/Training Provided? (Y/N) Was Logistics Technical Assistance/Training Provided? (Y/N) Was Obtaining Certifications Technical Assistance/Training Provided? (Y/N) Was Environmental Issues/Compliance Technical Assistance/Training Provided? (Y/N) Was Other Technical Assistance/Training Provided? (Y/N) Old Hub? (Y/N) 32 34,000,000 12,000,000 11,000,000 11,000,000 100% France, Belgium, UK, South Africa, USA 500 30% 70% Y N N Y N N N Y N 33 54,000,000 18,000,000 18,000,000 18,000,000 100% France, USA 570 25% 75% Y Y N Y N N N Y N 34 95,000,000 35,000,000 30,000,000 30,000,000 100% UK, Germany, USA 3,000 20% 80% Y N N Y N N N Y Y 35 21,000,000 7,000,000 7,000,000 7,000,000 100% UK, USA 1,200 20% 80% Y N N Y N N N Y N 36 30,000,000 10,000,000 10,000,000 10,000,000 100% France, Spain, Germany, USA 800 30% 70% Y Y Y Y Y Y Y Y N 37 36,000,000 12,000,000 12,000,000 12,000,000 100% France, UK, USA, South Africa 2,200 30% 70% Y N N Y N N N Y N Africa Trade Hubs Evaluation Report 129 Number Total Sales in Dollars, Last 3 Years Total Sales in Dollars, Previous Year Total Sales in Dollars, Previous Year -1 Total Sales in Dollars, Previous Year -2 % Exported (Most Recent Year) Countries Exported to in 2012 Total No. of Employees (Current) % of Male Employees % of Female Employees Was Applicant Determined by Project to be 'Export Ready'? (Y/N) Was Product Design Technical Assistance/Training Provided? (Y/N) Was Business Planning/Access to Finance Technical Assistance/Training Provided? (Y/N) Was Market Linkages/Marketing Technical Assistance/Training Provided? (Y/N) Was Logistics Technical Assistance/Training Provided? (Y/N) Was Obtaining Certifications Technical Assistance/Training Provided? (Y/N) Was Environmental Issues/Compliance Technical Assistance/Training Provided? (Y/N) Was Other Technical Assistance/Training Provided? (Y/N) Old Hub? (Y/N) 38 10,000,000 4,000,000 3,500,000 2,500,000 80% Holland, Japan, UK 300 50% 50% Y N N Y N N N Y N 39 5,500,000 2,600,000 1,400,000 1,500,000 100% USA, Japan, Holland, Italy, Sweden, Germany, France, Australia, Dubai, Reunion 10 50% 50% Y N N Y N N N Y Y 40 3,500,000 1,500,000 1,000,000 1,000,000 100% Israel, Holland, UK, Germany, Russia, Japan, USA 1,000 30% 70% Y N N Y N N N Y Y 130 Africa Trade Hubs Evaluation Report Number Total Sales in Dollars, Last 3 Years Total Sales in Dollars, Previous Year Total Sales in Dollars, Previous Year -1 Total Sales in Dollars, Previous Year -2 % Exported (Most Recent Year) Countries Exported to in 2012 Total No. of Employees (Current) % of Male Employees % of Female Employees Was Applicant Determined by Project to be 'Export Ready'? (Y/N) Was Product Design Technical Assistance/Training Provided? (Y/N) Was Business Planning/Access to Finance Technical Assistance/Training Provided? (Y/N) Was Market Linkages/Marketing Technical Assistance/Training Provided? (Y/N) Was Logistics Technical Assistance/Training Provided? (Y/N) Was Obtaining Certifications Technical Assistance/Training Provided? (Y/N) Was Environmental Issues/Compliance Technical Assistance/Training Provided? (Y/N) Was Other Technical Assistance/Training Provided? (Y/N) Old Hub? (Y/N) 41 3,900,000 1,300,000 1,300,000 1,300,000 70% Netherland s, Japan, Australia, USA 250 40% 60% Y N N Y N N N Y N 42 12,000,000 3,000,000 4,500,000 4,500,000 80% Holland, Japan, Australia 300 30% 70% Y N N N N N N Y N 43 2,500,000 1,000,000 750,000 750,000 100% UK, USA 100 50% 50% Y Y N Y N N Y Y N 44 2,000,000 800,000 700,000 500,000 30% Dubai, India, USA, UK, France, Tanzania, South Sudan, Uganda, Rwanda 25 50% 50% Y Y N Y N N Y Y N Africa Trade Hubs Evaluation Report 131 Number Total Sales in Dollars, Last 3 Years Total Sales in Dollars, Previous Year Total Sales in Dollars, Previous Year -1 Total Sales in Dollars, Previous Year -2 % Exported (Most Recent Year) Countries Exported to in 2012 Total No. of Employees (Current) % of Male Employees % of Female Employees Was Applicant Determined by Project to be 'Export Ready'? (Y/N) Was Product Design Technical Assistance/Training Provided? (Y/N) Was Business Planning/Access to Finance Technical Assistance/Training Provided? (Y/N) Was Market Linkages/Marketing Technical Assistance/Training Provided? (Y/N) Was Logistics Technical Assistance/Training Provided? (Y/N) Was Obtaining Certifications Technical Assistance/Training Provided? (Y/N) Was Environmental Issues/Compliance Technical Assistance/Training Provided? (Y/N) Was Other Technical Assistance/Training Provided? (Y/N) Old Hub? (Y/N) 45 1,600,000 600,000 500,000 500,000 40% UK, USA 25 70% 30% Y Y Y Y N Y N N N 46 2,000,000 800,000 600,000 600,000 30% UK 16 60% 40% Y Y N Y N Y N Y N 47 550,000 200,000 200,000 150,000 40% USA 712 50% 50% Y N N Y N N N Y N 48 557,000 220,000 182,000 155,000 75% US, Japan, Greece, Australia, Italy, UK 260 50% 50% Y Y N Y N N N Y N 48 25,000 15,000 6,000 4,000 20% Japan 320 15% 85% Y Y Y Y N N N Y N 132 Africa Trade Hubs Evaluation Report Number Total Sales in Dollars, Last 3 Years Total Sales in Dollars, Previous Year Total Sales in Dollars, Previous Year -1 Total Sales in Dollars, Previous Year -2 % Exported (Most Recent Year) Countries Exported to in 2012 Total No. of Employees (Current) % of Male Employees % of Female Employees Was Applicant Determined by Project to be 'Export Ready'? (Y/N) Was Product Design Technical Assistance/Training Provided? (Y/N) Was Business Planning/Access to Finance Technical Assistance/Training Provided? (Y/N) Was Market Linkages/Marketing Technical Assistance/Training Provided? (Y/N) Was Logistics Technical Assistance/Training Provided? (Y/N) Was Obtaining Certifications Technical Assistance/Training Provided? (Y/N) Was Environmental Issues/Compliance Technical Assistance/Training Provided? (Y/N) Was Other Technical Assistance/Training Provided? (Y/N) Old Hub? (Y/N) 49 750,000 300,000 250,000 200,000 80% US, UK, Holland, France, Japan, Australia 674 10% 90% Y Y Y Y Y N N Y Y 50 1,400,000 500,000 500,000 400,000 40% US, UK, Japan 46 90% 10% Y Y N Y N N N Y Y 51 700,000 300,000 200,000 200,000 25% UK, France 12 20% 80% Y Y N Y N N N Y N 52 1,835,000 650,000 610,000 575,000 75% US, UK, Japan 4,015 10% 90% Y Y Y Y N N N Y Y 53 475,000 200,000 175,000 100,000 60% US, India, Japan, UK 40 30% 70% Y Y Y Y N N N Y Y Africa Trade Hubs Evaluation Report 133 Number Total Sales in Dollars, Last 3 Years Total Sales in Dollars, Previous Year Total Sales in Dollars, Previous Year -1 Total Sales in Dollars, Previous Year -2 % Exported (Most Recent Year) Countries Exported to in 2012 Total No. of Employees (Current) % of Male Employees % of Female Employees Was Applicant Determined by Project to be 'Export Ready'? (Y/N) Was Product Design Technical Assistance/Training Provided? (Y/N) Was Business Planning/Access to Finance Technical Assistance/Training Provided? (Y/N) Was Market Linkages/Marketing Technical Assistance/Training Provided? (Y/N) Was Logistics Technical Assistance/Training Provided? (Y/N) Was Obtaining Certifications Technical Assistance/Training Provided? (Y/N) Was Environmental Issues/Compliance Technical Assistance/Training Provided? (Y/N) Was Other Technical Assistance/Training Provided? (Y/N) Old Hub? (Y/N) 54 1,075,000 400,000 350,000 325,000 75% USA, Germany, UK, Sweden, Japan, Portugal, Holland 538 30% 70% Y Y N Y N N N Y Y 55 500,000 250,000 150,000 100,000 40% USA, Germany, South Africa, Japan, Switzerland 12 33% 66% Y Y Y Y N N N Y N 56 295,000 120,000 100,000 75,000 65% US, Denmark, Canada, Kenya, South Africa 357 29% 71% Y Y N Y N N N Y N 57 1,105,000 475,000 430,000 200,000 100% USA, UK 24 1% 99% Y N N Y N N N Y N 134 Africa Trade Hubs Evaluation Report Number Total Sales in Dollars, Last 3 Years Total Sales in Dollars, Previous Year Total Sales in Dollars, Previous Year -1 Total Sales in Dollars, Previous Year -2 % Exported (Most Recent Year) Countries Exported to in 2012 Total No. of Employees (Current) % of Male Employees % of Female Employees Was Applicant Determined by Project to be 'Export Ready'? (Y/N) Was Product Design Technical Assistance/Training Provided? (Y/N) Was Business Planning/Access to Finance Technical Assistance/Training Provided? (Y/N) Was Market Linkages/Marketing Technical Assistance/Training Provided? (Y/N) Was Logistics Technical Assistance/Training Provided? (Y/N) Was Obtaining Certifications Technical Assistance/Training Provided? (Y/N) Was Environmental Issues/Compliance Technical Assistance/Training Provided? (Y/N) Was Other Technical Assistance/Training Provided? (Y/N) Old Hub? (Y/N) 58 12,000,000 4,000,000 4,000,000 4,000,000 60% UK, France, USA 200 20% 80% Y Y N Y N Y N Y Y 59 500,000 250,000 150,000 100,000 6500 % US, UK, Italy, Australia 326 40% 60% Y Y N Y N N N Y Y 60 975,000 425,000 350,000 200,000 85% USA, France, Australia 120 80% 20% Y Y N Y N N N Y Y Africa Trade Hubs Evaluation Report 135 ANNEX V: ADDITIONAL LESSONS LEARNED FROM WATH SURVEY The WATH IE survey asked for information beyond that which is reported in response to the evaluation questions. The descriptive information below provides some additional lessons learned about the needs of firms in Accra as well as the general operating context. Constraints to Exports All firms (treated and untreated) were asked to report on the level of challenge (no challenge, little or moderate challenge, severe challenge) they have faced on production and export of their products during the past decade. Figure 16 graphically represents the percentage of firms reporting the level of challenge for each constraint. Economic shocks was the most reported “severe challenge,” while environmental shocks appear to be reported most frequently as “no challenge.” Figure 16: Challenges in Production and Export Table 15 ranks the constraints to export by percentage of firms (high to low) reporting as “severe challenge.” 136 Africa Trade Hubs Evaluation Report Table 15: Reported Constraints to Export Constraint to Export Percentage ranked as “Severe Challenge” Economic shocks 64% Lack of access to investment or trade financing 57% Challenges with domestic infrastructure services 48% Difficulty identifying/marketing to foreign markets 41% Government regulation/corruption 35% International trade barriers 35% Environmental shocks 28% Difficulty meeting buyer requirements 28% Low skill level among workforce 22% Change in Constraints to Exports Firms receiving Trade Hubs assistance were asked whether (and which) challenges had increased or decreased since they began working with the Trade Hub. Figure 17 displays the percentage of firms reporting that a challenge has increased or decreased. International trade barriers, low skill level among workforce and difficulty meeting buyer requirements were the most reported “decreased challenge” since working with the Trade Hubs. Economic shocks, lack of access to investment or trade financing and challenges with domestic infrastructure services are the challenges most frequently reported as “increased.” Figure 17: Change in Constraints to Export—Treated Firms Africa Trade Hubs Evaluation Report 137 Assistance Received All firms were asked what type of assistance they have received from outside entities (see Table 2 in full report for percentage of support from outside entities). Figure 18 displays the type of assistance that treated and untreated firms received, followed by Table 16, which lists, from high to low, the percentage of type of assistance received by all firms (treated and untreated). Figure 18: Type of Assistance Received—Treated vs. Untreated 138 Africa Trade Hubs Evaluation Report Table 16: Reported Type of Assistance Received Type of Assistance Received Percentage of all firms receiving assistance (high to low) Trade show/ business to business forums 69% Financing 65% Connecting with buyers 54% Training for access to AGOA benefits 51% Practical business information 46% Marketing 40% Access to finance 40% ICT 39% Packaging/labeling 37% Customs documentation 36% Connecting with other companies to meet demands 35% Product design 35% Costing and pricing strategy 33% Logistical issues 31% Training for factory personnel 28% Partnering 26% Certification scheme training 26% Traceability systems training 24% Sustainability/legality issues 21% Food safety standards training 16% Other 2% Africa Trade Hubs Evaluation Report 139 Finally, firms were also asked what type of assistance received had the most and least impact on their organization. It should be noted that firms were not limited to marking a single type of assistance as the most or least impactful. Figure 19 displays the percentage of firms (treated vs. untreated) reporting on which type of assistance had the most impact. It appears as though assistance with trade shows and business to business forums and financing were reported most frequently as having the most impact for both treated and untreated firms. Figure 19: Beneficiary-Reported Type of Assistance with Most Impact—Treated vs. Untreated 140 Africa Trade Hubs Evaluation Report ANNEX VI: DATA MONITORING AND EVALUATION FOR FUTURE TRADE HUBS Introduction This document presents recommendations for development of key indicators, data collection methods and sources of data for performance management of the firm level effects of USAID’s Africa Trade Hubs. The narrative includes a discussion of the types of data that should be collected and maintained throughout the life of future Trade Hub projects as well as how that information might be used for any future project evaluations. All of the information and recommendations included in this document are informed by lessons learned during the course of the 2012/2013 IE of the Africa Trade Hubs as well as best practices in performance management and evaluation. In addition to a narrative discussing performance management/evaluation, this document includes a table of potential indicators and a draft firm level data collection instrument, as well as an explanation on how to use the instrument (and revise it as the project develops and changes), for use in collecting information on the impact of export promotion assistance provided by the Trade Hubs.87 Performance Management General Overview Performance monitoring is critical both for helping implementing partners, USAID and other stakeholders to make management decisions about the project along the way as well as for use in evaluations of project results. As with any USAID project, before identifying project indicators for use in a performance monitoring plan (PMP), it is important that the project theory of change and results framework be defined. Indicators should be designed not only to test the desired outcomes and impacts of the project but also to test the theory of change (or development hypothesis) and its corresponding assumptions. These intermediate-level indicators are critical for helping management to understand how and why the project has or has not reached its targeted outcomes/impacts. For instance, the East Africa Trade Hub, run by the COMPETE Project, during the period 2009–2013 set a goal of ECA firms taking advantage of opportunities under AGOA. To reach this goal, the Trade Hub implemented international trade shows, direct buyer missions and technical assistance. A strong PMP should include indicators that help to assess the success of each of these activities, measure the assumptions that must hold true for these activities to lead to the goal of firms taking advantage of AGOA and measure the ultimate outcome of firms taking advantage of AGOA. To make this more concrete, evaluators have provided an example below. Take an intermediate result chain such as that provided in Figure 20: 87 As described in more detail later in this document, this questionnaire could also be used by business associations seeking to maintain monitoring data on the firms with which they work. Africa Trade Hubs Evaluation Report 141 Figure 20: Example Results Chain For this chain, monitoring and evaluation (M&E) staff should have indicators measuring: 1. The success of the technical assistance provided, likely through pre- and post-tests of participants who received training; 2. Whether participants actually implemented what they learned, likely through site visits or tracking of the quality or quantity of marketing materials (depending on what was taught in the training); 3. The implicit assumptions that come between the second- and third-level results—that firms are able to attract new U.S. buyers and that those buyers actually procure products from the firms that attended the training. This might be measured by tracking the number of buyers per firm, where those buyers are located and total sales and exports to buyers; and 4. The total number/percentage of firms who attended the training that are taking advantage of opportunities under AGOA. In addition, depending on whether USAID or its implementing partners want to measure contribution or attribution (discussed later), there is some additional information that the Trade Hub should track, related again to assumptions made in this results chain. For instance, if M&E staff were to find that marketing of firms who attended the training improved or that firms who attended training secured more U.S. buyers, it would not be possible to know that these changes were due explicitly to the Trade Hub training. There may be another donor project also working with firms that is partly or wholly responsible for measured outcomes, or it could be that U.S. firms simply started looking to buy more products from Africa because prices of similar products increased elsewhere in the world. On the other hand, even if the marketing plan were to succeed and be implemented well, M&E staff may find that buyers are not attracted to African firms because limited infrastructure in some countries has pushed prices up too high. For these reasons, it is also helpful for performance monitoring plans to track indicators that measure possible alternative explanations for outcomes or reasons that outcomes might not be reached. The Trade Hub, for example, could keep track of how many other donor projects are assisting the same firms with which the Trade Hub works, as well as what types of assistance those projects are providing. Additionally, the Trade Hub could track world commodity prices and shocks, currency fluctuations, environmental shocks, trade barriers, etc. As illuminated in the above example, it is critical that the performance monitoring plan for the Africa Trade Hubs Project be tailored specifically to the Trade Hubs based on a detailed results chain and clear understanding of the implicit assumptions within that results chain. Firms take advantage of opportunities under AGOA Firms incorporate the knowledge they learned in the training to improve their marketing Technical assistance on marketeting provided to firms 142 Africa Trade Hubs Evaluation Report Recommended Indicators and Data Collection Methods It is not possible for evaluators to provide a full PMP, including all indicators that should be tracked for future iterations of the Africa Trade Hub Project (due to the lack of full results framework for future projects and also because the evaluation upon which this tool is based only looked at firm level assistance by the Trade Hubs—instead of all activities and intermediate results for the Trade Hubs); however, the evaluation team presents below a list of some key indicators that would have made the 2012/2013 IE easier and more useful to USAID if they had been tracked during the course of the past Trade Hubs’ Projects. Firm Level Indicators (all of which can be tracked using the data collection instrument provided below and should be collected before the project begins—at baseline—and then every year that the project is operating, and possibly for a few years after the project ends [in order to pick up lagged effects]): • Product lines (by year): This information is important if USAID wants to be able to track diversification of product lines over time. Tracking results by year allows M&E staff to compare changes with intervention timelines, shocks, etc. to better determine the source of any identified changes. This is the same reason each of the below indicators is also tracked by year. • Sales (by product line and year): This information is critical for measuring whether Trade Hub assistance is leading or contributing to changes in total sales (both domestic and international through exports). Sales should be tracked by product line for each firm to help M&E staff to understand which Trade Hub interventions work better or whether to target specific product lines over others. • Exports (by product line, destination, buyer and year): This information is critical for understanding Trade Hub effects on exports. It should be tracked by product line, destination and buyer to allow M&E staff to understand whether the Trade Hub is having more success in increasing exports of specific products or to specific destinations as well as to track diversification of exports. Tracking at the buyer level allows for a more nuanced understanding of what is happening with exports. If, for instance, exports have increased, it could be due to the same buyers buying more products or new buyers beginning to buy products from a firm. The former is probably a more risky way of expanding than the latter and also might mean that even though exports increased, it is not necessarily due to the Trade Hub linking firms to new buyers. • Employment (by gender and year): This information is critical for understanding how Trade Hub interventions have affected employment at the firm level. This information should be tracked by gender and should also be measured in full-time equivalents or by dividing out part-time from full-time employees. Without this differentiation, employment may appear to have increased, for example, when in reality it has just shifted from mostly full-time to mostly part-time employment. • Type(s) of Trade Hub assistance received (by year): This information is critical if USAID wishes to understand what types of assistance have the most effect on firms as well as for looking at the cost-effectiveness or cost benefit of firm level assistance (or assistance to firms through associations). • Other donor assistance received (by year): This information is critical for ruling out potential causes of changes in outcomes other than that of Trade Hub activities and is best tracked during the course of the project to avoid challenges with recall bias. • Investment and loan amounts (by year): This information is important both for ruling out other potential causes of changes in outcomes and also to consider it as a possible outcome of Trade Hub interventions. Africa Trade Hubs Evaluation Report 143 Project-Level Indicators: • Expenses (by country, value chain, activity and year): It is critical that each of the implementing partners closely track their expenses for the project by country, value chain, activity and year if USAID wishes to conduct cost-benefit or cost-effectiveness analysis. Additionally, to truly get at project costs, implementing partners should track labor costs by country, value chain and activity—or at the very least by activity—and should also determine a way of divvying up fixed and overhead costs (such as buildings and management) among activities, as direct activity costs are only part of the picture. For example, just because direct costs are low does not mean that an activity is cheap, as it could require hours of labor. Cost-benefit and cost-effectiveness analysis help to identify interventions that use resources most efficiently. Cost-benefit analysis allows evaluators to compare expenses with monetary benefits to determine how much money it costs for a given level of income through sales or exports, for instance. Cost-effectiveness analysis is an economic evaluation method that examines both the costs and outcomes of alternative intervention strategies. It compares the cost of an intervention to its effectiveness as measured by a specific outcome. It is critical to note that cost-effectiveness analysis uses a particular outcome measure that must be common among the alternatives being considered. Outcomes can be determined either through impact or performance evaluations. The best method when attempting cost-effectiveness analysis is IE, as it allows outcomes to be attributed directly to project costs whereas performance evaluations only allow evaluators to determine that project costs contributed in some way to outcomes. In the latter, there could be other potential costs or interventions that also affected outcomes. Cost-effectiveness results are generally presented in the form of a cost-effectiveness ratio, which expresses cost per outcome (e.g., $100/job created). Understanding the cost benefit or cost-effectiveness of a project and its various activities is important for determining where to focus resources in future interventions to have the most impact. Since USAID is often faced with the challenges of resource allocation, cost-effectiveness analysis could help to identify the most cost-effective strategies from a set of options that have similar results. Relevant cost￾effectiveness indicators include: cost/a given value of additional domestic sales; cost/a given value of additional exports (e.g., by product line, country and market destination); cost/a given number of additional jobs created (e.g., by value chain and country). National Indicators: Each of the following should be tracked for all firms in each of the Trade Hub target countries if USAID wishes to compare beneficiary firm results with average results by firms nationwide (as is described in the PE section of this report). USAID may also wish to track national-level results in an attempt to understand how the Trade Hubs might be affecting those results. • Sales (by country, value chain and year) • Exports (by country, value chain and year) • Employment (by country, value chain and year) • Foreign investment (by country, value chain and year) Firm Level Data Collection Instrument USAID should require the Trade Hub implementing partners, or an independent M&E contractor, to collect and maintain longitudinal survey data on the indicators discussed in the above section. This information should be maintained for all firms receiving Trade Hub support as well as for any firms identified as a control group (as discussed in the Impact Evaluation section of this report). The survey 144 Africa Trade Hubs Evaluation Report should be administered at baseline (before the Trade Hub intervention) and then through routine periodic telephone or email surveys of firms throughout the life of the project (and potentially for a period of time afterwards to measure lagged project effects). As with any telephone or email survey, follow-up may be necessary through site visits to firms that are unresponsive or partially responsive to survey queries. Implementing partners are likely to get the highest response rates if they require firms to fill out this survey in order for the firms to receive Trade Hub assistance. The survey forms should aim for simplicity in design and implementation to avoid error. A sample data collection survey is included below. However, it is important to note that this instrument is entirely illustrative, and the Trade Hubs or associations with which the Trade Hubs work may wish to add or subtract questions based on the data needs and outcomes that USAID would like to be able to measure. For instance, USAID may wish to track firm-reported constraints to exports and/or the most useful types of assistance received from the Trade Hub or other donors to help in targeting and adapting interventions each year. It is also important to note that prior to implementing the data collection survey, implementing partners or associations should explain the point of the survey and also explain the fact that firm confidentiality will be maintained. This is critical when tracking data on sensitive firm financial information. While prior to the 2012/2013 IE and PE of the Trade Hubs several stakeholders were concerned that firms would be reluctant to provide this data to evaluators, the evaluation team found that as long as firms were made aware of the purpose of the data and also ensured of their confidentiality, firms were very open about their sales, exports, employment, etc. Determining Responsibility for Data Tracking USAID is the primary entity responsible for determining whose responsibility it is to track the above monitoring data and at what level it should be tracked. The evaluation team recommends that USAID require the implementing partner to provide data at the firm level either through direct contact with firms (if interventions are provided at the firm level) or through requiring beneficiary associations to track member-firm results using the same survey instrument (if interventions are provided at the association level). If the latter is determined to be the best course of action, the implementing partner must require associations to also track the project-level indicators described above as well as what types of interventions the association is providing to each of its member firms (and which of those are Trade Hub–supported interventions and which are not). Africa Trade Hubs Evaluation Report 145 USAID TRADE HUBS FIRM DATA COLLECTION SAMPLE APPLICANT INFORMATION Please provide your name and contact information so that you may be contacted for any follow-up information. Full Name (respondent): 10T Title: 10T Email: 10T Phone: 10T REPORTING INFORMATION This form should be filled out each year of Trade Hubs participation This form is filled out as (check box): ☐ Application ☐Follow-Up If Follow-Up, report covers: Year: Choose YearChoose Year Submission Date: 10T COMPANY CONTACT INFORMATION Company Full Name: 10T Country: 10T Email: 10T Website: 10T Company Physical Address: 10T Company Mailing Address: 10T Phone 1: 10T Phone 2: 10T Fax: 10T Managing Director Name: 10T Managing Director Phone: 10T Managing Director Email: 10T COMPANY DETAILS Type of Company: Choose TypeChoose Type Year of First Operation: Choose YearChoose Year Current Number of Male and Female Owners: Female Owners Male Owners Firm’s Primary Sector: Primary Sector If Other, Specify: 10T Firm’s Secondary Sector: Primary Sector If Other, Specify: 10T EMPLOYMENT DETAILS Current Count of Full-Time Employees: Female 10T Male 10T Current Count of Part-Time/Seasonal Employees (employees working fewer than 30 hours per week): Female 10T Male 10T TRADE HUBS AND OTHER ASSISTANCE Please provide information about the type of assistance that your firm has received during the reporting year. If the specific type is not listed, please specify in the “Other” line. Mark if your firm received this assistance from the USAID Trade Hubs Mark if your firm received this assistance from any organization other than the USAID Trade Hubs Assistance Type ☐ ☐ Financing ☐ ☐ Trade Show/ Business-to-business forum ☐ ☐ Connecting with buyers ☐ ☐ Coordination with other companies to meet demands ☐ ☐ Product design ☐ ☐ Packaging and labeling ☐ ☐ Marketing ☐ ☐ Practical business information ☐ ☐ Partnering ☐ ☐ Food safety standard training ☐ ☐ Traceability systems training 146 Africa Trade Hubs Evaluation Report USAID TRADE HUBS FIRM DATA COLLECTION SAMPLE ☐ ☐ Logistical issues ☐ ☐ Certification ☐ ☐ Sustainability and legality issues ☐ ☐ Costing and pricing strategy ☐ ☐ Access to AGOA benefits ☐ ☐ Access to finance ☐ ☐ Customs documentation ☐ ☐ Information and Communication Technology ☐ ☐ Other Please Specify: 10T What organizations, outside of the USAID Trade Hubs have provided any of the assistance from the above question? ☐ International Buyers/Investors ☐ State-Owned Banks/Enterprises ☐ Other Government Agencies ☐ Private Firms ☐ Friends or Relatives ☐ Multilateral Organizations ☐ NGOs ☐ Other PRODUCT LINE SALES Please list all product lines, followed by corresponding total sales (combined domestic and international) by product line during the reporting year. Total Annual Sales: 10T Percentage of Total Sales Exported 10T Product Line Total (domestic and international) product line sales for reporting year 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T PRODUCT LINE EXPORTS Please list all export buyers for the reporting year, as well as their respective destination country and the amount they purchased by product line. If a buyer buys more than one product, it will have to be listed on more than one line. Buyers (list all) Product Line Destination Country Total Product Line Sales to that Buyer for the Reporting Year 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T 10T Investments and Loans Total amount of loans used in reporting year: 10T Total amount of investments made in the firm by outside entities in the reporting year: 10T Africa Trade Hubs Evaluation Report 147 Evaluating the Project Overview This section provides guidance on how the USAID Africa Trade Hubs Project might evaluate the effectiveness of its programming and activities for future iterations of the Trade Hubs. As mentioned above, the existence of the monitoring data described will go a long way toward ensuring the ability of future evaluation teams to conduct an evaluation that yields high-quality, meaningful and useful findings, conclusions and recommendations. It is also important, however, that any future evaluation of the Trade Hubs Project be planned in advance of project implementation. During project design, it is important that USAID, its implementing partners and other project/evaluation stakeholders identify the purpose of any planned evaluations as well as the evaluation questions (which should stem directly from the evaluation purpose). This will allow stakeholders to plan for the resources necessary to conduct the evaluations as well as to ensure both that all necessary monitoring data is collected throughout the life of the project (including baseline information) and that selection of beneficiaries is done in a way that allows evaluators to answer evaluation questions. The latter is described in more detail in the IE section below. The importance of monitoring data is described above. Baseline information is important for all evaluations (both impact and performance evaluations) because it is difficult to apply any value to endline outcomes and impacts without having a reference of what the situation was before the project started. For example, if evaluators find that (using the example above) 60 percent of beneficiary firms have implemented a strong marketing plan, how can they know that this is a good percentage? What if before the project started, 60 percent of firms already had strong marketing plans? To truly understand project impacts, evaluators must have a reference point, and it is important that this information be collected prior to project implementation (rather than through recall at the end of a project, as was the case with the 2012/2013 Trade Hubs IE) because recreating baseline numbers introduces validity issues—including recall and response biases, as described in the body of this evaluation report. Evaluation Purpose and Contribution Versus Attribution According to USAID’s 2011 Evaluation Policy (EP), there are two key evaluation purposes—learning and accountability. Evaluations that are focused on learning usually ask questions about the validity of the development hypothesis (whether implemented activities led to desired results and if so, why, or if not, why not); test an innovative project design to determine whether it produced the desired results; compare types of activities and outcomes to determine which activities are most effective; assess the cost-effectiveness of various activities so that future projects can be focused on the most efficient activities; determine the types of environments where similar projects can be replicated in the future and yield similar results; look at unintended consequences of interventions; and identify reasons why a project either did or did not succeed in reaching its intended targets. As such, learning evaluations are often asking high-level questions and looking for project contribution to or attribution of results. They generally focus on project outputs and impacts. According to the EP, IEs (which answer cause-and-effect questions) are the only way of attributing results (outputs or impacts) to a specific project or program (determining that a result is definitely due to a project or program). However, that does not mean that a performance evaluation (PE) can’t also answer questions of project outcomes or impact. On the contrary, effectively designed PEs can often present high-quality findings and conclusions about project contributions (meaning there could be other factors outside of the project also contributing) and often provide very good information about the nuances of project results and how and why they were or were not achieved. Evaluations that are focused on accountability generally look more at project input- and output-level indicators and questions and provide information on what happened in a project or compare what was planned with what is actually delivered to identify potential gaps between planned and realized outputs. 148 Africa Trade Hubs Evaluation Report Questions in accountability-focused evaluations generally resemble the following: Did the project implement the activities it said it would implement? Did it reach the number of beneficiaries it was intended to reach? and Was funding used efficiently (without waste or abuse)? Accountability evaluations usually take the form of PEs since project inputs and outputs are generally directly linked to project activities and therefore cannot be due to anything but the actual project (meaning attribution is explicit). For instance, using the training example above, accountability evaluation might ask the question of whether training occurred or how many firms received training from the project. It is critical that USAID and other evaluation stakeholders identify the purpose and evaluation questions during project design for the reasons specified in the Overview section above but also so that stakeholders can plan for and implement the appropriate type of evaluation. For example, if stakeholders decide to conduct a learning evaluation with questions that attempt to attribute outcomes and impacts to a specific project (as was the case with the 2012/2013 IE conducted by the evaluation team) in addition to collecting baseline information on a beneficiary (or treatment) group, stakeholders should also identify and collect baseline information on a comparison (or control) group. This is explained in more detail in the IE section below. Impact Evaluation While monitoring data enables program implementers to document beneficiary participation, how fast the program is expanding, how resources are being utilized, whether activities are being implemented as planned and what types of outcomes and impacts beneficiaries are experiencing, a project PMP does not demonstrate whether indicators, targets and achievements are a result of project interventions. In order to attribute outcomes and impacts to a project, stakeholders must rely on IEs, which attempt to rule out all other potential causes for any identified outcomes and impacts. There are two types of IEs—those that utilize experimental designs and those that rely on quasi￾experimental designs. An experimental design, which is the strongest evaluation design according to USAID’s EP, requires that beneficiary selection be done randomly. For the Trade Hubs to use an experimental design, they would need to identify all firms (or associations—depending on the unit of intervention—or treatment) eligible to receive Trade Hubs’ programming and then randomly assign half of those firms or associations to a beneficiary group and half to a comparison group (which would not receive any Trade Hubs support) before the project was implemented. This assures that—assuming an appropriate sample size—the beneficiary (treatment) and comparison (control) groups are identical or as similar as possible. This is important as it allows evaluators to rule out any difference between the groups (rather than the project intervention) as being the cause of any identified results. Quasi￾experimental designs (QEDs), by contrast, simply require a treatment and control group. However, the two groups do not have to be selected through random assignment. Instead, they can be purposefully selected or they can self-select into participation. Quasi-experimental IE designs, though, rely on certain assumptions that do not always hold true, namely that the treatment and control group are very similar or, if they are not similar, that in absence of the intervention being assessed, changes in outcomes measured between the two groups would have been similar. In other words, without the Trade Hub intervention, the two groups would have experienced the same growth in sales, exports and employment. The below sections explore the feasibility of using each of these designs in a future IE of the Trade Hubs Project. Feasibility of Using an Experimental Design. The evaluation team found that it would be difficult for the Trade Hubs to use an experimental design in evaluating the Trade Hubs in the future unless they were to change their methods for selecting beneficiaries. As described in the 2012/2013 IE report, in the past, Trade Hub beneficiary firms have been selected by assessing the firms’ level of “export readiness.” While each of the regional Trade Hubs used different criteria to determine whether a firm was export￾ready, their criteria were similar and attempted to identify firms that were producing quality products, had good capacity and—preferably—had exported before. In other words, the Trade Hubs were looking Africa Trade Hubs Evaluation Report 149 for high performers. The reasons for this, according to the Trade Hubs, were twofold: 1) to ensure the firms had the ability to take advantage of Trade Hub interventions; and 2) because the goal of the Trade Hubs Project is to attract buyers, especially those from the U.S., to purchase African products, and the best way to do so is by promoting good firms that can attract business for other firms. As described above, fully experimental IE designs require that project beneficiaries be randomly selected. Even though the Trade Hubs have selection criteria, it would still be possible for them to randomly select beneficiary and control firms from all firms that meet the selection criteria. However, the evaluation team found that there are not many firms that meet the selection criteria, and, of those that do, not all apply for Trade Hub assistance. Therefore, Trade Hub resources, for the most part, have allowed the Trade Hubs to work with (provide treatment to) all firms that apply for assistance and meet the selection criteria, meaning that it hasn’t been, and likely will not become, possible to randomly select a control group. Feasibility of Using a Quasi-Experimental Design. For future Trade Hubs evaluations it is important to understand how to implement a QED approach (of which there are many types) as well as to know whether the assumptions that QEDs rely upon are likely to hold true. To apply a QED approach to the Trade Hubs in the future, USAID and its implementing partners would work together with an evaluation firm to identify firms or associations that could be used as a comparison group. There are a few ways of doing this—one way would be to designate all those firms that applied to receive Trade Hub assistance but were not selected as beneficiary firms as the comparison group (this is what the evaluation team did in the 2012/2013 IE). Another way would be to identify possible comparison firms through secondary data (data collected by national governments, other donor projects or associations, for instance). Each of these options is considered below. The challenge with designating the comparison group as those that apply for Trade Hub assistance but do not receive it is that unless the firms that are not selected to receive assistance are not selected simply due to limited resources (and not because they do not meet the selection criteria), the assumption that many QEDs rely upon—that treatment and control firms are very similar—does not hold true. This, however, does not negate the use of a QED, it just means that evaluators will need to use a different QED design, likely difference-in-differences. A difference-in-differences design relies on the parallel-trend assumption (that even though the treatment and control groups are different, if it weren’t for the intervention the two groups would still have been likely to experience similar outcome and impact indicator trends). To describe how this assumption works (using the example of the training leading to improved marketing plans, described above), assume that training participants were fundamentally different from non-participants—say, for example, that they were selected to receive the training based on need (because their marketing plans were particularly bad and, thus, those in the control group tend to have much better marketing plans). The differences between the two groups in relation to need are acceptable as long as stakeholders believe that firms from both groups would have developed and improved their marketing plans at the same rate if it weren’t for the Trade Hub. Marketing plans are a difficult example, but if, instead, evaluators were to look at sales and exports, for instance, this becomes much easier to visualize. If, for example, the Trade Hubs select firms based on the value of their exports (selecting those with more exports to be the treatment group and those with fewer exports to be the control group), a difference-in-difference QED is still possible as long as it is reasonable to assume that without the Trade Hubs, both types of firms would have experienced similar percentage changes in their exports, as demonstrated in the table below (which shows that despite different start and end points, the two groups have the same expected percentage increase for the firms from baseline to endline— assuming no intervention). 150 Africa Trade Hubs Evaluation Report Example of the Parallel-Trend Assumption Type of Firm Value of Exports at Baseline Expected Value of Exports at Endline (without the intervention) Percent Change Treatment $10,000 $15,000 50% Control $1,000 $1,500 50% The reason the parallel-trend assumption is important is that evaluators want to be able to say that if the treatment group actually experienced a 75 percent increase in its exports, those extra 25 percentage points were due to the project and not due to the differences between the treatment and control groups. If stakeholders believe that larger firms are likely to grow at a faster pace than smaller firms, this assumption would not hold true and the comparison would be useless. While a difference-in-differences design could be feasible for future IEs of the Trade Hubs, it is important that the types of differences present between treatment and control groups at baseline be closely analyzed to determine if those differences might affect the validity of the parallel-trend assumption. It was not possible for the evaluation team to do this during the 2012/2013 IE of the Trade Hubs until after data were collected (because there was not sufficient baseline information with which to compare firms). However, in the future, this is an important step, and if USAID wishes to conduct a QED of the Trade Hubs Project moving forward, it should require an evaluation firm to do this using already-collected baseline data. The types of things future evaluators should look for include experience with exporting. It is probably reasonable to assume, for instance, that firms with some experience exporting will likely see greater increases in their exports moving forward (even without the Trade Hub intervention) than firms with no experience exporting. If treatment and control firms are determined to have such differences at baseline, then, a difference-in-differences IE would not be possible because the parallel-trend assumption would not hold. As mentioned above, another way of selecting comparison firms in a QED would be to use secondary data of firms that did not apply to the Trade Hub. This might allow for the selection of comparison firms that are more similar to treatment firms—at least on observable criteria (such as the value of sales and exports, years in existence, sectoral expertise, etc.). However, if the Trade Hubs continue to select beneficiary firms through an application process, there would likely be fundamental differences between treatment and comparison firms. This is because firms that take the initiative to apply for Trade Hubs support are likely fundamentally different than those that don’t. They may be, for instance, more motivated and better connected (since they would have had to have heard about the Trade Hub from someone). As described above, while differences between firms at baseline do not necessarily matter in a difference-in-differences design, they can if they call into question the parallel-trend assumption. And, unfortunately, in this case, they probably do, as these two characteristics are also probably linked to the rate at which firms experience increases in their sales and exports. Other Challenges with Conducting an IE of the Trade Hubs. As described in the 2012/2013 IE of the Trade Hubs, another challenge the Trade Hubs might face with conducting an IE moving forward is that part of the Trade Hub development hypothesis and goal includes spillover effects. USAID and the Trade Hub implementing partners report that they hope that the Trade Hubs help to attract more Africa Trade Hubs Evaluation Report 151 buyers to Africa in general, not just to Trade Hub–supported firms. This makes it difficult to isolate the effects of the Trade Hubs to the treatment firms, which could mean that any IE conducted of the Trade Hubs might underestimate the impact of the Trade Hub intervention on firms. This does not mean that an IE should not be conducted or that it would not have meaning, but it should be understood that measureable results may be slightly skewed downwards. Another challenge with future IEs of the Trade Hubs Project is that, as mentioned in the 2012/2013 IE report, USAID plans to shift the focus of the Trade Hubs to assistance at the association level rather than the firm level. If this happens, an IE of firm level results would require designating treatment firms as those that work with associations supported by the Trade Hub and comparison firms as those that work with associations not supported by the Trade Hub. Also, it would be important that associations in each of the groups be similar with regards to the support they are providing firms at baseline (before the Trade Hub interventions). It might prove difficult to find associations that are similar enough to make this possible. Recommendations about the Future Use of IEs in Assessing Trade Hub Results. Based on the above information, USAID should only contract with an evaluation firm to conduct an IE of the Trade Hubs in the future if ONE of the following criteria is met: • Firms can be selected for treatment randomly either from all firms that apply or from all firms listed in some secondary source. This could be done without excluding firms from receiving Trade Hub assistance in order to form a comparison group if implementing partners stagger the intake of promising firms into their export promotion programs on a two-year lag. This methodology would require the use of longitudinal surveys to measure all treatment and control groups over time (and it would only allow evaluators to measure two years of Trade Hub intervention impact, rather than longer-term effects). However, it would eliminate any ethical reasons for not conducting an IE and would allow the Trade Hubs to continue to support the same number of firms while also allowing for a rigorous assessment of the impacts of such support. A variation of this would involve randomly assigning groups of firms to receive different types of trade assistance to test the relative effectiveness of each type of assistance. • Firms are selected for treatment using specific selection criteria but those not selected make a valid control group because they could reasonably be expected to experience similar changes in project outcomes without the project interventions. • Associations are selected to receive treatment randomly (assuming a statistically significant sample size) and firms can be compared between treatment and control associations. It should be noted that gathering a sufficient sample size will likely be much more difficult when assistance is clustered in associations. • Associations are selected for treatment using specific selection criteria but those not selected make a valid control group because the firms they support could reasonably be expected to experience similar changes in project outcomes without the project interventions (meaning treatment and control associations provide similar types of services to their member firms before the Trade Hub intervention begins, and these interventions are continued in both groups with the addition of Trade Hub interventions for treatment association firms). If none of the above criteria hold true, USAID will be unable to isolate the effects of the Trade Hub activities and, thus, attribute measured outcomes and impacts to the Trade Hubs Project. However, that does not mean that USAID can’t still identify project contributions to outcomes and impacts through a PE. Methods for doing so are described briefly in the PE section below. 152 Africa Trade Hubs Evaluation Report Performance Evaluation Should USAID, in consultation with the implementing partner and potentially an evaluation firm, determine that an IE of the Trade Hubs is not possible or is not desired for resource reasons or because the evaluation purpose and questions do not support the use of an IE, USAID can still conduct a PE. There are many PE designs that USAID or an evaluation firm might use to evaluate the effectiveness of the Trade Hubs Project. USAID should choose a design in consultation with other key stakeholders, and it should be based directly on the evaluation purpose and questions (as is also true with an IE). If, as was the case with the 2012/2013 Trade Hubs IE, USAID wishes to measure project contribution to outcomes such as sales, exports, employment and diversification, looking only at firm baseline numbers and endline numbers will not allow evaluators to assess contribution of the project to any changes in those numbers. However, evaluators can use the following methods to try to better understand project contributions: • They can compare trends in firm level results for a period of time before the Trade Hub intervention to a period of time after the Trade Hub intervention. If the percentage increase in outcomes and impacts increases between the two periods, then evaluators can assume that the Trade Hub is a likely contributor to this change. This is what the evaluation team did in analyzing the ECATH impact. The evaluation would have been stronger and the assumption of contribution even more likely if evaluators had been able to track outcome trends by year and had seen a jump in the year that a firm began working with the Trade Hub. This was not possible because firms could not remember subtle difference between years and had to instead rely on estimates related to specific events (a method for assisting with recall). But, if data had been collected throughout the course of the Trade Hubs Project, as described in the performance monitoring section above, a year-by-year comparison would have been possible. However, even with a year-by-year comparison, evaluators can only get at contribution of the Trade Hub to results and not attribution, because it is impossible to rule out all other potential causes of the changes in outcomes. The strongest PE designs will try to gather information on other causes and rule them out or attribute a portion of the effects to them—reducing the remaining effects to those that are most likely due to the Trade Hub. • They can compare trends in firm level results to trends in national and sectoral results. If the percentage increase in outcomes and impacts among beneficiary firms is higher than the general trend among national firms by sector, then evaluators can assume that the Trade Hub is a likely contributor to this change. The evaluation team was unable to conduct this analysis for the 2012/2013 ECATH PE because data was not available by year (as described in the Evaluation Report) and there was not sufficient national-level information for each of the value chains. In the future, if USAID wishes to pursue this option, it should consider aligning Trade Hub sectoral interventions with data tracked nationally at the sectoral level (the sectors must be directly comparable) or working with an organization to track national-level results (though if this is not already being done, it might be difficult to achieve due to the extensive resources that would be required). Also, even with a year-by-year comparison and national data by sectors, evaluators can only get at contribution of the Trade Hub to results and not attribution, because it is impossible to rule out all other possible causes of the changes in outcomes—especially, in this case, changes due to spillover effects of Trade Hub interventions or because the Trade Hubs firms themselves are so large (or there are so many of them) that they are driving the national￾level result changes. However, if through a comparison of beneficiary firms and national data on firms, evaluators find that beneficiary firms are experiencing greater increases in outcomes, then they can be certain (assuming beneficiary firms are not benefitting from some non–Trade Hub intervention that other firms in the country are not) that the Trade Hub has been a likely contributor to those improved results. Africa Trade Hubs Evaluation Report 153 • They can rely on beneficiary and other stakeholder perceptions about Trade Hub impacts at a snapshot in time, or they can look at perceptions of the state of exports in Africa before and after the Trade Hub intervention or over the course of several years of Trade Hub interventions. This data can be collected through surveys, key informant interviews, group interviews and focus group discussions. While they may sound less rigorous than some of the data described above, data on stakeholder perceptions can actually be very valuable and provide more nuanced findings than can some of the other data described above. In fact, stakeholder perceptions should always be used to triangulate and better understand the hows and whys behind findings coming from all of the above evaluation designs (both in IE and PE designs). Used as a design in and of itself, the collection of perception data in a snapshot, before and after, or time series design is strongest when multiple methods of data collection or sources of information are used to triangulate results. Data Analysis and Storage It is important that all monitoring data, evaluation reports and data and implementing partner report deliverables are maintained for the Trade Hubs over time. Part of the reason the evaluation team struggled with trying to evaluate the impacts of the Trade Hubs Project was because nobody—neither the implementing partners nor USAID—maintained all past project documents and data. Moving forward, this will be critical for evaluating project effectiveness. Evaluators will benefit from having access to all project documents as well as from knowing which firms were helped when and by what methods, information on how to contact those firms, information on how they were selected, monitoring data on all firms—as well as project- and national-level monitoring information (as described above). U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523