April 2010 This publication was produced for review by the United States Agency for International Development. It was prepared by Chemonics International Inc. PARAGUAY POVERTY REDUCTION PROGRAM FINAL EVALUATION REPORT FY 2010 JUNE 2007 – MARCH 2010 PARAGUAY POVERTY REDUCTION PROGRAM FINAL EVALUATION REPORT FY 2010 JUNE 2007 – MARCH 2010 Contract No. EEM-I-00-07-00008-00 Task Order No. 346 Chemonics International Inc. PARAGUAY VENDE II FINAL EVALUATION iii CONTENTS Executive Summary ............................................................................................................1 Section 1 – Background .......................................................................................................3 Section 2 – Overall Results ..................................................................................................5 Section 3 – Effectiveness of Project Components ...............................................................9 Section 4 – Interactions with Clients .................................................................................13 Section 5 – Sustainability of Results and Related Issues ...................................................16 Section 6 – Impact of Trade and Investment Capacity Building Efforts ...........................18 Section 7 – Principal Findings ...........................................................................................20 Section 8 – Recommendations for USAID and for Chemonics ........................................21 Annexes Annex A – Persons Interviewed ........................................................................................ 22 Annex B – Bibliography .................................................................................................... 23 PARAGUAY VENDE II FINAL EVALUATION 1 EXECUTIVE SUMMARY This is an evaluation of a USAID-funded business development project, Paraguay Vende II, which was implemented between June 2007 and June 2010. It was intimately connected to Paraguay Vende I, which ran from August 2003 to April 2007. Altogether the two projects received about $9 million in USAID funding. The project‟s approach was to use the “value chain” concept to link (largely) agricultural producers, the buyers of their produce, the processing firms that converted these products into finished, marketable items, and the ultimate consumers, whether domestic or foreign. The main intervention by the project was with the (generally relatively large) processing or manufacturing firms, though the methodology was used to identify other links in the value chain that should be targeted for assistance. Overall, the project was a spectacular success. With four months still to go before project closure, sales and exports connected to the project were more than twice the original target, and exceeded the revised targets of $94 million in sales and $64 million in exports. These are results that few USAID business-development projects can match. The major sectors assisted included sesame (37% of sales), processed meat (20%) and textiles (9%), but there were project activities in another ten areas. The project‟s work in sesame, which has rapidly become an important crop for small farmers, helped create a new major export sector, with exports rising from less than $5 million in 2003 to more than $100 million in 2008, when world prices were unusually high. An estimated 10,000 farmers were connected to project activities, mostly through assistance in using improved seeds and agricultural practices. The main thrust of the project was identifying promising enterprises in three geographic areas in the interior of the country, working with these enterprises to develop work plans to relieve key constraints to growth of the firm, and eventually ending the technical assistance once the key goals were achieved, or in some cases, when it became clear that the firm was unable or unwilling to undertake key actions. Three keys to success of the project might be identified. First, the continuation of assistance to about 20 firms begun under Vende I allowed ample time for relations of deep trust and confidence to be developed between Paraguay Vende staff and consultant and the assisted firms. As the period of relationship between the enterprise and the project frequently extended to four years or so, it seems clear that USAID projects extending beyond the typical three-year timetable are important to effectiveness. Second, the high quality of the human resources provided by the project, both in the Asunción headquarters and in the regional service centers was critical to project success. Again, the skills of the staff in identifying the right enterprises for assistance and then in building trust between the Vende advisors and the enterprise management, were critical. Finally, the project identified an important shortcoming in the Paraguayan business environment: the lack of a developed business services sector. Enterprises in the interior of the country lacked good information about how to acquire specialized assistance in building their 2 PARAGUAY VENDE II FINAL EVALUATION enterprises, and the relatively low level of trust within the business community was another contributing factor. The capacity of Paraguay Vende to be an „honest broker” in the provision of technical assistance to firms made participation in the project attractive to businesses. While assistance to firms was the central feature of the project, two other activities were also significant. First, the project staff engaged in public dialogue on economic policy issues, most notably producing a series of studies of high quality on specific sectors where Paraguay, with an appropriate policy environment, might be an effective player in world or regional markets. The studies of sesame, biofuels, and organic sugar are models for the effective presentation of both the basic facts about the technologies of production and the numerous issues related to the potential growth of the sector. The second program, “Tu Producto al Super,” or „your product to the supermarket,‟ begun in early 2009, sought to connect micro and small businesses to supermarket outlets for their products. The relatively short tenure of the project makes its effectiveness uncertain. The most important constraint is the difficulty for micro and small firms to obtain the necessary permits that would allow them legitimately to offer products for sale in the supermarket. While continuation of this activity under a successor project seems warranted, it is not clear that the results likely to be obtained are likely to repay the resources committed. The principal recommendation for USAID is to continue to support business development services using the value-chain model in future activity in Paraguay, as it appears to deliver substantial development impact. Manufacturing as well as agriculture offers potential for job creation, increased exports and other benefits. The report also recommends that the pilot program for linking micro and small enterprises be incorporated into the new Paraguay Productivo USAID project, as it may offer modest impact on the growth of such firms, and may also assist in improvements in government procedural requirements that adversely affect the smallest businesses – such as the requirement that registration of businesses take place only in the capital, Asunción. PARAGUAY VENDE II FINAL EVALUATION 3 SECTION 1: BACKGROUND Project Background The project is a continuation of an earlier project, Vende I, which was implemented between August 2003 and March 2007, at a cost of $4.8 million. Vende I was considered an impressive success, having been associated with more than $35 million in increased sales by participating companies, $22 million of them in exports, $2.8 million in new investments, and the equivalent of 17,000 additional jobs. All three figures were dramatically higher than the initial project goals established in 2003 (e.g., the initial goal for total sales was $9 million, of which $3 million was for exports). Project goals were raised twice during implementation, with the final outcome exceeding even the more ambitious targets in each area. After a two-month gap after the completion of Paraguay Vende I, a second phase, Paraguay Vende II, was begun in June 2007. This gap produced some discontinuities of staffing and programmatic activity, as financial commitments could not be made until funding for the new project was approved by USAID. Overall funding for Vende II was $4.2 million. Despite the discontinuity between the two projects, most firms assisted under Vende I continued to receive assistance under Vende II. Obviously, this makes evaluation of Vende II as a separate activity more questionable. Country Environment Background For the most part, the country environment became less favorable over the course of Paraguay Vende II. At the beginning of Paraguay Vende I, the country‟s exchange rate was extremely favorable to exports, reaching a level of 7,500 Guaranis/dollar in 2003. The exchange rate gradually deteriorated during both Vende projects, reaching a low of 4,000 Guaranis/dollar during 2008. It gradually recovered to about 4,700 Guaranís/dollar by late March, 2010. In its most recent Article IV report (October 2009), the IMF suggested that the Guaraní was 6-8% overvalued, in contrast to significant undervaluation during much of Vende I and the early stages of Vende II. The undervaluation of the exchange rate during the earlier stages of the project would have made both exporting and production for the domestic market easier for project clients, while the appreciation of the exchange rate during the latter stages would have made sales in both markets more difficult. Another environmental factor that affected project execution was the dramatic run-up in sesame prices in 2008, the result of adverse weather factors in the main sesame producers in Asia. This led to unprecedented world prices for sesame, which was a great opportunity for Paraguayan producers just as considerable additional production was arriving at the market. This gave a huge impetus to sales of assisted firms, counted as a project result. 4 PARAGUAY VENDE II FINAL EVALUATION A final environmental factor was the change in administration in 2008 as a result of presidential elections. It was an exercise in democracy, as a non-Colorado party president took office for the first time in 60 years. For the purposes of the project, however, the change was less favorable. The new government was less focused on economic reform and improvements in the business environment than its predecessor, and proved to adopt populist policies in some areas relevant to the project‟s success. The principal negative influence of the new government was in the area of sesame. After the unprecedented rise in prices in 2008, they fell to more normal levels in 2009. Some politicians pressed for subsidies to sesame growers to compensate for the price decline. This involved the government in the middle of relations between growers and processors, leading to substantial uncertainty in the sector and concerns by processors that government involvement would interfere with the sesame value chain in unproductive or unpredictable ways. As a result of these concerns, the project withdrew from further support for sesame production. All in all, then, the environment facing the project was one of gradual deterioration, with both the exchange rate and government policy moving in directions that would make success and sustainability of the project more difficult. Nevertheless, the extremely high prices for sesame in 2008 were a significant factor in the overall sales and export results reported by the project. PARAGUAY VENDE II FINAL EVALUATION 5 SECTION 2: OVERALL RESULTS General Despite the deteriorating environment mentioned above, the overall results of the project are remarkably outstanding. This project is a clear winner, big time. Sales attributed to the project through February, 2010 (four months short of the end of project) totaled $128 million, or 136% of the revised target, or considerably more than double the original target. As shown in Table I, all three other Vende II targets – for exports, investment, and job creation – were exceeded. In most cases, the revised targets were exceeded by comfortable margins. Despite the unfavorable movement of the exchange rate, exports were nearly triple the original target. These results merit further comment. The USAID investment of $4.2 million was associated with increased sales of $128 million, including $67 million in Paraguayan exports, and nearly 40,000 person-years of employment. Clearly, these are remarkable results. Subsequent sections of this evaluation will delve deeper into the question of the strength of the association between the Paraguay Vende assistance and the results reported, and ask about the sources of the effectiveness of the project in achieving them. At the same time, it should be recognized that 55% of all sales of firms reported under Paraguay Vende II came from firms that had been assisted under Paraguay Vende I. Clearly, the continuity between the two projects was important to the overall success of the endeavor. Table I Paraguay Vende Goals and Achievements Indicator Original Target Revised Target Achieved by 2/28/10 Revised Goal Achievement Increased Sales ($ Million) 57 94.5 128.6 136% Increased Exports ($ Million) 23 64 66.9 105% Increased Employment (Million person-days) 7.15 7.4 9.8 132% Increased Investment ($ Million) 4.65 5 6.1 122% The large impact numbers from the project were largely due to the increased sales and exports from a group of very successful companies. Table 2 lists the 19 firms that accounted for 92% of all the sales associated with the project. Another 36 firms were assisted, along with 26 supermarkets, and a number of micro and small businesses but those connections yielded substantially smaller results in increased sales, at least through February, 2010. It is possible that additional sales and exports will be generated in the future from these firms as a consequence of the assistance by the project. For the 19 firms where attributed results exceeded $1 million, sesame was the most important single product, accounting for 37% of all sales associated with the project. As noted above, it was followed by meat processing (chicken and pork) which accounted for 6 PARAGUAY VENDE II FINAL EVALUATION another 20%. Textiles came in third, with 9% of the total. Another ten sectors were represented among the firms that reached the $1 million level. Major Beneficiaries of Paraguay Vende II Support Client Product Additional Sales Domestic Sales Exports Added Employment Investment ($ Million) ($ Million) ($ Million) (million days) ($ Million) Hierbapar S.A. ARASY Sesame, organics 23.8 1.0 22.8 1.57 Shirosawa Company S.A.I.C. Sesame 21.7 0.0 21.7 0.90 Granja Avicola La Blanca Chicken 12.7 12.7 0.0 0.54 Qin Yi America S.A. Textiles 9.9 9.9 0.0 0.94 1.45 Cooperativa de Producción Agrícola Bergthal Ltda. Flour 9.5 9.5 0.0 0.93 0.73 INSAMA, Ingenio Santa Maria Organic sugar 8.7 0.8 7.9 1.36 0.03 UPISA Processed meat 7.8 3.9 3.9 0.37 POLLPAR Processed meat 4.5 4.5 0.0 0.15 REIMPEX S.R.L. Motorcycles 3.2 3.2 0.0 0.60 Compañía Paraguaya de Asfalto S.A. Recycling 2.5 2.5 0.0 0.03 0.01 Molinos El País S.A. Rice 2.2 0.0 2.2 0.21 Fibrac Fiberglass containers 1.8 1.8 0.0 0.24 0.40 Destileria del Sur S.A.C.I.A. Ethanol 1.8 1.8 0.0 0.17 Chung Bo S.A.I.C.I Textiles 1.6 0.0 1.6 0.23 Alimentos Ecologicos S.A. Sesame 1.5 0.0 1.5 0.12 Cooperativa Colonias Unidas Agropec. E Ind. Ltda. Milk 1.3 1.3 0.0 0.10 Forestal La Poderosa S.A. Charcoal 1.3 1.3 0.0 0.05 0.02 Dalazen S.A. Farm Machinery 1.2 1.2 0.0 0.12 AGRO ZAFRA Rice 1.1 0.0 1.1 0.13 Total 118 55 63 8.8 2.64 The increased sales and exports are far larger than those typically associated with donor projects. This result is examined in subsequent sections of this report, which analyze the correctness of the sales and export numbers used by the project, as well as various other issues relating to the project. But the employment and investment numbers also merit comment. For employment, the 8.8 million person-days translate into the equivalent of a year of employment for 35,000 workers. Many USAID projects have spent much more for a smaller impact on employment. The investment numbers may be more telling. Business firms often freely accept free technical assistance from donors, taking it or leaving it in terms of the value they give to it. Investment is another matter. The fact that assisted firms invested substantial amounts in coordination with the Vende assistance is clear evidence that the assistance was valued sufficiently to lead to additional capital investment. To appreciate the outcome of the project, and its impressive numerical results, it is useful to look at a few individual firms that were substantial beneficiaries of Vende II assistance. At the same time, it is clear that many of the heritage firms from Vende I produced the most successful results in sales or exports under Vende II. Some Specific Cases Examples of the interaction between Paraguay Vende and individual firms include some, particularly successful, cases: Hierbapar /ARASY. These are two firms with common ownership, one a producer of sesame and the other a producer of organic herbs and spices. These two enterprises PARAGUAY VENDE II FINAL EVALUATION 7 started up in 2000 by young high-school graduates. The association with Paraguay Vende began in September 2004, under Vende I. While the workplan negotiated between the project and the firm under Vende II dealt mostly with finding buyers for the firm‟s products, the firm‟s director stated that Paraguay Vende‟s earlier assistance was much broader. According to him, Paraguay Vende helped formalize the company, helped establish organizational systems for the various departments in the enterprise, and generally was a critical element of the firm‟s growth. This growth was spectacular, from less than $1 million in sales in 2003, to about $20 million in 2010, three-fourths in sesame, and one-forth in organic herbs. The firm is optimistic about the future, though unhappy with government policy to subsidize sesame. The firm has 15,000 producers of sesame in its database, with about 5,000 producers as steady sources of the product. COMPASA. This firm began working with Paraguay Vende I to recycle used oil from service stations in Paraguay. Often such used oil was simply poured into the ground, with undesirable environmental consequences. COMPASA started as an enterprise focused on capturing and recycling this used oil, under Vende I. Paraguay Vende provided assistance in strategic planning and in developing suppliers of used oil in the interior of the country. It also did training for suppliers and publicity about the need for more environmentally-friendly treatment of such wastes. COMPASA has since expanded into various other forms of recycling – oil filters, oil drums, and paper among others. The enterprise has proved to be both profitable and environmentally beneficial to Paraguay. Among other credits given to the project, the owner credits Paraguay Vende for the firm‟s ability to convince major firms in Paraguay, particularly multinationals, to allow COMPASA to recycle their waste material. MMKM. This is an apparel firm begun in 2001 by a Palestinian immigrant. The initial focus was on exports to Brazil. Although it is not listed among those companies with sales above $1 million in Table 2, it seems destined for rapid growth. After several years of seeking such contracts with the help of Paraguay Vende, it recently was able to obtain a contract to produce athletic shorts for Nike, and has begun their manufacture. The company‟s new facility, visited by the evaluator, was a model of for an apparel firm anywhere – well-lit, air conditioned, with work stations spaced well apart from each other. Qin-Yi. This is a producer of bedding and other textile products for export to Brazil and other neighboring countries. It was established under free-zone regulations supported by another USAID project. Besides providing technical support to the firm, the project assisted the firm in eventually gaining government approval for the issuance of visas to Chinese technicians working to install the needed machinery. (Because Paraguay recognizes the government in Taipei as the representative of China, this was a lengthy and difficult process.) During the evaluator‟s visit, the owner reported that his wife was currently in China attempting to interest other firms in locating in Paraguay. Fibrasa. This firm manufactures fiberglass tanks and other products. It received two rounds of assistance from the project, in internal organization and in development of new markets for its products. The firm complained that, while the first round of assistance 8 PARAGUAY VENDE II FINAL EVALUATION was delivered in a very timely manner, there was a six-month delay between the commitment to the second round and the actual delivery of assistance. Project staff reported that this was a consequence of funding constraints late in the project, when the appreciation of the Paraguayan Guaraní lowered the funding availability in local currency for project activities, requiring adjustments in activity levels. Escobas Santa Rosa. This firm is the poster child for microenterprises gaining access to supermarkets. The firm makes brooms of the traditional type, made of corn straw. Developed by María del Rosario Franco and her husband Rosalino, the company initially sold brooms to a few shops, with the brooms being delivered on the back of a motorcycle. Given access to some supermarkets via the TPS program, they were able to dramatically expand production to 180 brooms per day at their backyard enterprise. They now have fifteen farmers providing them with raw materials for brooms. Their success has allowed them to move from motorcycle transport to purchase of a pickup truck, allowing many more brooms to be sent at a time. The pickup has also allowed them to help other micro and small enterprises by carrying their products to supermarkets in more remote locations. Besides the successes, there were also a number of failures. A motorcycle assembler, Alex, was unable to gain expected access to the Brazilian market, and so was unable to increase its production. A call-center operation encountered operational and regulatory problems, and ceased operations. There were other cases where the assistance failed to achieve expected results, as is to be expected in any such project. PARAGUAY VENDE II FINAL EVALUATION 9 SECTION 3: EFFECTIVENESS OF PROJECT COMPONENTS The Asunción Head Office The Asunción head office managed strategic planning, information technology, accounting, monitoring and evaluation, and communication. This appeared to be a competently-staffed and relatively lean operation. There seems to have been little staff turnover of senior people at the head office. Indeed, the continuity of head office staff was impressive, with the chief of party continuing from the beginning of Paraguay Vende I, the M&E chief from late in Vende I, and the head of communications from early in Paraguay Vende II. The project director was extremely effective in providing a vision of what the project should do and in overseeing the staff and operations of the Economic Service Centers (ESCs). It is clear to this evaluator that the strong and effective leadership from the top of the organization was one of the key factors in the great success of the project. While the ESCs were the primary contact for client firms, it is clear that the project director and the head of M&E also had extensive contact with client firms. Thus, supervision from the head office was a major element for effective oversight of the ESCs. The Assistance provided by the Economic Service Centers (ESCs) Two of the three ESCs, Coronel Oviedo y Ciudad del Este, were visited by the evaluator. Although both were on the verge of closing permanently within a week of my visit, both centers were still very operational. Each center had highly-professional staff. Meetings with clients demonstrated that ESC staff had developed close working relations with the clients, and had established strong ties of trust with the clients. Most of the technical assistance activity of the centers is done by the long-term professionals in the centers, supplemented by short-term experts. Each center is operated by a sub-contractor to Paraguay Vende, rather than directly by the project. This arrangement seems to have worked well. Overall, the quality of the human resources in the service centers seems to have been a critical element of the success of the project. ESC staff members were energetic in seeking out firms that had serious potential for growth, and in rejecting those that offered less promise. This is a tribute both to their quality, and to the central management of Paraguay Vende in selecting appropriate professional staff for the centers. Quality of project monitoring and evaluation a. The Paraguay Vende Methodology for Measuring Results Vende II has a well-documented and solid methodology for monitoring project results. The written manual (Paraguay Vende 2007), describes the procedures in detail. Under 10 PARAGUAY VENDE II FINAL EVALUATION the methodology, only sales growth in excess of the expected growth in sales of firms in the sector is counted as additional. In addition, the increases in sales or exports were subjected to a test of relevance to the activities supported by the project. Where increased sales could not be linked to the project interventions, they were not to be counted as resulting from the project. Overall, the manual is a reasonable vehicle for attribution of impact. There are many ways in which the manual might have used other methodologies, and any economist might find grounds to quibble about any approach taken. This evaluator‟s biggest quibble was with respect to the sales growth that was assumed as a baseline rate. This baseline growth rate was used to separate the increased growth of sales or exports from that which would be expected in the absence of any assistance. Only sales in excess of this amount were to be included as benefits of the project. These baseline estimates, though calculated using a defensible methodology, appear relatively small. Nevertheless, adjustments in this baseline or natural rate of growth of sales of assisted firms would not have been large enough to affect the conclusion that the project was highly successful. Other changes in the methodology might have produced even more impressive results. Most notably, continuing to count sales increases by firms that had “graduated” (i.e., were no longer receiving assistance from Paraguay Vende) might have swelled the totals substantially. Such a methodological approach is easily defended. In sum, the results of the project might have been bigger using some methodologies, or smaller using others, but there is no doubt that the project was associated with positive results, big time. b. Implementation of the Monitoring and Evaluation System The M&E unit appears to have had autonomy in its application of this rigorous methodology to the results of the project. Both the current and the former head of this unit took their responsibilities seriously, seeking only to measure results that had a significant connection to the assistance provided by the project. Both recounted specific cases where they challenged optimistic estimates of project impact in specific cases. With their intimate knowledge of the scope of the assistance to each firm, and its connection to any increase in sales, they were able to sustain their view. Neither M&E manger reported any efforts by Paraguay Vende management to reduce the rigor of the attribution to increase the reported results. c. The Pedro Garay Study The most notable feature of evaluation of the project was a quasi-experimental study that sought to compare the economic consequences of technical assistance to a sample of small sesame seed producers with a similarly-situated set of producers who did not receive technical assistance under the project. The study (Garay, 2009), carried out by a researcher from Kansas State University, compared outcomes for a relatively small sample (40 assisted farmers and 40 unassisted farmers considered to be in similar conditions at the outset) to determine whether the Vende-provided technical assistance had been of value. Both groups produced sesame prior to the beginning of Paraguay PARAGUAY VENDE II FINAL EVALUATION 11 Vende, with an average of 3.5 hectares of production. Most farmers in both groups had shifted from cotton production to sesame, either because of low cotton prices or the expectation that sesame would be more profitable. In response to a question of whether they had received technical assistance for sesame production, 71% of the assisted farmers said that they had received it, compared with 27% of the farmers outside the project – who may have received support from other sources such as the Ministry of Agriculture. But, of those stating that they had received assistance, 41% of the Vende-assisted farmers asserted that it was “very good,” compared to only 9% from the control group. With respect to production, 43% of the assisted farmers reported increased production, compared to only 9% for the control group. In both cases, the increased production was attributed to higher yields. Most notably, 70% of the assisted farmers considered that they had improved their economic condition during the last five years, compared to only 25% of the control group. Such quasi-experimental comparisons of assisted and unassisted recipients are uncommon in donor programs, though there has been much interest in promoting such tests of the effectiveness of assistance by the academic community. In this evaluator‟s experience, this is close to being unique as a means for judging the impact of a USAID project. The project deserves recognition for its willingness to undertake such a survey, where the outcome simply cannot be known until the results are computed. d. Summary of M&E Findings In sum, the M&E for the project was highly professional in assuring that the results reported to USAID were the result of meaningful, project-related, actions by assisted firms. Moreover, the Garay study is a fine example of a project taking a serious risk of negative findings from a survey conducted by an outsider that could well have produced negative results for the project. Effectiveness of USAID Oversight USAID appears to have provided generally effective oversight of the project, with frequent contact between USAID staff and the project leadership. Unlike many USAID projects, there was substantial continuity of USAID staff involvement with the project, with consequent benefits of close understanding between the project and the USAID mission. USAID and the U.S. Embassy community generally have shown strong interest in the project. The Ambassador has been a frequent visitor to significant Vende events, On a visit to Paraguay, the U.S. Secretary of Commerce visited Vende projects. This has been a high-profile project for USAID, and USAID has provided continual oversight and advice on how to strengthen the project‟s impact. At the same time, the slowness in USAID at several points may have had negative consequences for the project‟s effectiveness. The first such problem was mentioned earlier, when a two-month gap separated the end of Paraguay Vende I from the beginning of Paraguay Vende II. Such gaps create uncertainties for project staff, and create 12 PARAGUAY VENDE II FINAL EVALUATION significant managerial obstacles to effectiveness. Related to this transition was the delay of three months (mentioned in the quarterly report for the fourth quarter of FY 2007), between the Vende II proposal for 26 firms from Vende I to be assisted under the new project and USAID approval of assistance to 22 of them. Such delays in approval are common in bureaucratic organizations, but they can be serious negative factors in the effectiveness of assistance to private-sector firms. PARAGUAY VENDE II FINAL EVALUATION 13 SECTION 4: INTERACTIONS WITH CLIENTS Specific support provided by VENDE a. Value Chain activities The methodology used to implement the project by the Paraguay Vende team was to focus on the value chain for individual products. About fifteen products were subjected to value chain analysis during the project. This approach looks at four elements: buyers of processed or manufactured products, the producer of these products, the suppliers of raw materials to the producer, and, in the case of agricultural products, the growers of the raw material, generally small farmers. In general, the main point of interaction along the value chain was the producer or processor, often a firm of substantial size. Paraguay Vende would hold conversations with potential clients – most apparently first contacted by Paraguay Vende, but some approached Paraguay Vende as a result of numerous newspaper reports on the project‟s activities. The evaluator carried out interviews with nine recipients of assistance, and files from the mid-term evaluation were reviewed for another seven firms. The interviews were conducted with all three types of enterprises assisted – large firms, supermarkets and micro and small enterprises. Nevertheless, an emphasis was given to the large firms, which are the source of the very large numbers in sales, exports and employment. Between the mid-term and final evaluation, firms accounting for 60% of attributed sales and 75% of attributed exports were interviewed. Although some technical issues regarding the estimates are discussed elsewhere in this report, no significant discrepancies were identified between reported numbers and actual firm performance. This by no means to say that the increased sales and exports from these firms were due entirely to, or caused by, Paraguay Vende. Any provider of assistance to a firm can play only a subsidiary role, offering ideas that the firm may accept or reject. In the jargon of the USAID economic growth strategy, firm leaders are the “drivers” of productivity growth. Technical advisors are “enablers,” whose advice may be helpful in providing useful ideas to the firm. But the firm, or the entrepreneur is the decision-maker. So, at best, Paraguay Vende contributed importantly to the capacity of firms to make good decisions. But the firms, not Paraguay Vende, are primarily responsible for success in exports or sales. b. Supermarkets The newest program developed by the project, “Tu Producto al Super,” or TPS, (Your Product to the Supermarket) was conceived at the end of 2008, and began operating in early 2009. Its intent has been to link micro and small businesses (MSEs) with supermarkets. The big challenge, beyond the products of the MSEs themselves, is the formalization required for providing goods to supermarkets. These include registration as 14 PARAGUAY VENDE II FINAL EVALUATION a business, tax registration, bar code registration, and other sanitary or environmental inspections, depending on the product. The Paraguayan government received a grant of around $1 million from FOCEM, a Mercosur development facility, to attempt to formalize five hundred of these enterprises. Discussions with VENDE project staff suggested that slowness by the Paraguayan government in implementation means that the effort is likely to fall substantially short of this goal, because of the difficulties involved. The supermarkets appear to have varied significantly in their enthusiasm for the program, some highly interested in stocking more Paraguayan products, while others saw the difficulties in dealing with a larger number of suppliers and the logistical problems involved. The project provided a junior person in each cooperating supermarket to maintain supplies of the products being sold under the program. Standards are important, or even critical, with respect to products that people consume or use in their homes. The recent scandals in the United States about food products containing contaminants, or construction drywall that could destroy plumbing pipes and electrical conduits make the potential problems clear, even in the highest-income countries. Consumers of supermarket products in Paraguay have a right to expect that the products they buy are safe. At the same time, standards for registration or product safety pose a barrier to micro and small firms wishing to gain access to the great opportunities that supermarkets offer. USAID has worked with the government of Paraguay to eliminate regulatory requirements that do not serve a public purpose, and some progress has been made. Nevertheless, there are still unnecessary obstacles. For example, the requirement that permits for firms be issued from Asunción is an important obstacle to firms of any size located in the interior of the country. c. MSE Activities The other end of the supermarket effort was the micro and small business sector. The project appears to have generated substantial enthusiasm on the part of many such firms through informational sessions. The evaluator visited only two of these firms. The “broom lady” mentioned earlier was the most notable success, but a cheese producer was also achieving substantial increases in sales. Quality of business plans developed for clients A substantial number of business plans for clients were reviewed as part of the evaluation. Most were too general to be of use in evaluating their success in changing the ability of assisted firms to meet the challenges of a changing marketplace. Instead, as with the mid-term evaluation, this evaluator relied on the testimony of the client enterprises for the primary judgment of their effectiveness. PARAGUAY VENDE II FINAL EVALUATION 15 Reliability of statistics generated from clients To assess the reliability of the statistics generated from clients, several activities were undertaken. First, the manual of procedures for measuring results was reviewed. Second, the current and previous heads of the Paraguay Vende M&E unit were interviewed. Third, interviews with client firms were used to validate the statistics. Fourth, the findings from a similar exercise from the mid-term evaluation of the project were reviewed. Three conclusions result from this review: 1. The Vende M&E unit was effective in assuring that the statistics produced for firms were reasonable in terms of the terms of reference for the assistance provided by Paraguay Vende. 2. Interviews with firms receiving assistance from the project, both in the final evaluation and in the mid-term evaluation, supported the general results reported in the project‟s reports. 3. The methodology for measuring results of the project was sound. In sum, the project made a serious effort to correctly measure the impact of its efforts, and the numbers produced by the project should be treated as reasonable measures of its success. 16 PARAGUAY VENDE II FINAL EVALUATION SECTION 5: SUSTAINABILITY OF RESULTS AND RELATED ISSUES Overall, the project appears to have produced significant results that are likely to be sustainable in the longer term. The more successful of the companies interviewed seemed poised for continued growth, and had internalized changes made as a result of the technical assistance. Some of the assisted companies had failed to meet expected opportunities, and might well disappear. For several, this related to the vagaries of barriers, some non-tariff obstacles, which arose with respect to exports to Brazil. Broadly speaking, the companies that were most successful in increasing sales were the most likely to continue to do so. Such firms had mastered the myriad challenges facing any business by having the flexibility and entrepreneurial capacity that allowed them to respond to problems as they arose. Beyond sustainability, two questions pose themselves about this project: Was the project “cherry-picking” with firms that were on their way to success in any event? What were the key ingredients of the project‟s success? a. Cherry-picking Given the typically-three year length of USAID contracts, together with a typical six￾month startup, any USAID project is subject to the accusation of “cherry-picking,” that is of choosing firms to assist that were already on the verge of substantial increases in sales or exports. The tendency for USAID projects to focus on near-term opportunities is obvious. That is what USAID is asking them to do – to produce tangible results during a relatively short time. Does this mean that the project did not add important value? This evaluator‟s view is to ask why Paraguayan firms prior to the project (or unassisted firms during the life of the project) did not exhibit the same dynamism as the firms assisted by the project. The typical economist, in the old joke, would refuse to pick up a $100 bill on the sidewalk, arguing that, if it were really $100, someone would already have picked it up. Some of the assisted firms in this project appear to have been $100 bills that no one had picked up. Their sales and exports grew much faster than those of the economy as a whole. That other Paraguayan firms did not show similar dynamism suggests that there was some missing ingredient in the Paraguayan environment that was somehow supplied by Paraguay Vende. Beyond the assistance to the successful firms, as discussed below, the project also undertook informational and public policy work on specific sectors where potential for PARAGUAY VENDE II FINAL EVALUATION 17 future growth may occur, based on learning by unassisted firms and improvements in public policy. b. Key Ingredients of Project Success There seem to have been three ingredients to the great success of this project. i. Competent, motivated Staff. As discussed above, the Paraguay Vende staff was both competent and highly motivated. This is certainly an important ingredient in the success of any enterprise. ii. The value-chain model. This model emphasizes the need to look at the entire process from initial producer (in most of these cases, a farmer) to the buyer of those products, to the processors or manufacturers who use those raw materials, to the buyer of the finished product. Under this methodology, Vende advisors worked with the firm to identify the key constraint to growth. In some cases, the key constraint was lack of buyers. In others, it was internal issues within the firm. In others, it was problems with processors of agricultural products grown by farmers. Finally, it might be the technology used by the farmers in growing the product, from either use of seeds that did not meet market demand, or cultivation processes, or other aspects of producing a product that met market standards. iii. Engendering Trust. In poor countries, trust between different members of a value chain or between buyer and seller are often weak, if not openly conflictive. The term coyote to identify the intermediary between the producer and the market is part of the common parlance in Latin America. This appears to be characteristic of Paraguay, where buyers are suspicious that sellers will not deliver the quantity and quality agreed upon, and sellers fear that buyers will not pay the agreed price for the product. In this environment of mutual suspicion, an “honest broker” can add substantial value. To this evaluator, the trust that the project was able to build between different parts of the value chain, and between project advisors and the firms they assisted was probably the most critical element of the success of the project. In a society where trust is in short supply, finding a source of help that you can trust is of great value. Gradually, the project – in combination with its predecessor, Vende I – seems to have acquired the stature in Paraguay that encouraged firms in the country to seek assistance from the project, where they would not have sought it elsewhere. 18 PARAGUAY VENDE II FINAL EVALUATION SECTION 6: IMPACT OF TRADE AND INVESTMENT CAPACITY BUILDING EFFORTS The bulk of the effort by Paraguay Vende was at the firm and product level, in accordance with the targets established by USAID for the project. Though no specific targets were established for outreach, the lack of specific commitments with regard to trade and investment capacity building, as is usually the case, meant that fewer resources were focused on these issues than on business development. Sector Reports The project contributed significantly to the knowledge of issues relating to specific products supported by the project. Six such studies were completed or in process by the time of the evaluation. Four were reviewed as part of this evaluation, of which one (organic sugar) was not yet printed. All four of the studies were of high quality, produced in a very professional manner. Though most of the studies tended to be long (60-90 pages), they were extremely readable, with many pictures and informative graphics. The content of most was mostly informational – details about the technologies of production, the organization of the market for these products, the prospects for the future of the sector, and the policy issues in Paraguay about their production and export. The four studies include: The Sesame Study. This 90-page study, completed in late 2009, is a very high-quality analysis of the technology, marketing, and other key issues relating to this product. Sesame was virtually unknown in Paraguay before 1990. As late as 2003 (the year Vende I began), exports of sesame were less than $5 million. With the rapid growth of production and the world price explosion noted earlier, exports in 2008 exceeded $100 million. The study provides a history of the production of the crop in the country, including information about the firms that pioneered its production, current issues and prospects for the future. (The leading pioneers of sesame production in Paraguay, Shirosawa, Kemagro, and Hiebapar, were all assisted by the project, mainly in helping the thousands of small sesame producers to meet the quality requirements of the market.) The study also addressed the current policy issues relating to sesame, notably the government‟s decision to subsidize prices. The report takes a strong, carefully-argued view that this is a mistake, and that the production and export of this product would be likely to be negatively affected if government became involved in setting prices or attempting to manage production in the sector. Through the release of the report at a public gathering, and through subsequent efforts to influence public opinion, Paraguay Vende sought to inform the public dialogue on this topic. PARAGUAY VENDE II FINAL EVALUATION 19 The biofuels study. This 80-page report, issued in August 2009, is a virtual guide for anyone interested in the production, processing or marketing of biofuels. The study makes clear that this is a promising area for Paraguayan comparative advantage. Small firms, great products. This report, an issue of the projects information bulletins rather than a stand-alone product, supported the project‟s efforts to promote sales by microenterprises to supermarkets. The cover featured the project‟s big success, the “broom lady” discussed earlier. Like the others, this is a slick, very readable product. Unlike the others, it lacks the technical detail that would help guide a potential producer or marketer into the business. But this is in keeping with the situation of the producers and the market, as discussed in the section on the supermarket project. The organic sugar study. This report, reviewed in draft, is a solid companion to the already-published reports – full of technical detail for the potential producer, international marketing issues for the potential exporter, and useful insights for the government policymaker. Altogether, these reports, along with various information bulletins and studies carried out under Paraguay Vende I, are serious and competent efforts to disseminate useful information regarding activities that have potential for increasing incomes and productivity in Paraguay. As such, they are useful contributions to Paraguayan development. At the same time, the fact that most of these reports were published in the last year before project completion may make them less useful than if they had been produced earlier. But this is a matter of speculation. There is no way to know if an entrepreneur interested in biofuels or organic sugar might use the study as a basis for entry into the field. Other Communications In other areas of communication, Paraguay Vende II was very effective in reaching the main newspapers of the country with hundreds of stories and other information about its activities and relevant issues. Though this evaluator has not counted them, there appear to be thousands of newspaper stories that feature Paraguay Vende II or activities that it promoted. This appears to have been one of the tools used by the project to encourage new firms to contact Paraguay Vende. Conclusions Paraguay Vende was able to use its frequent mentions in newspapers and other periodicals to create a reputation that denoted seriousness in promoting Paraguayan development. The long sectoral reports were of high quality and potentially of great use to entrepreneurs who might want to enter those sectors. The impact of such studies is not knowable, but could be significant. Only in a few years will the results of these reports become evident. But their high technical quality should be a useful guide for potential entrepreneurs in the sectors for which the project has conducted studies. In this respect, the project has produced what economists call a “public good,” something that may be of value to the economic growth of the country, but which is freely available to anyone. 20 PARAGUAY VENDE II FINAL EVALUATION SECTION 7: PRINCIPAL FINDINGS The principal findings from this evaluation are given below, with each finding referenced to an earlier section of this evaluation report. 1. The methodology used to monitor and measure results from the project is reasonable, and the results reported represent realistic estimates (Section III.c). 2. The Pedro Garay study of sesame producers – a quasi-experimental approach to measuring the impact of Paraguay Vende assistance – was a bold and commendable effort, and it clearly demonstrated that the project had significantly impacted the incomes of small producers (Section III.c) 3. The value-chain approach employed by Paraguay Vende is sound, and was effectively implemented (Section IV.a.i). 4. The quality of human resources employed by the project was very high, offering a principal explanation for the results obtained (Sections III and V.b). 5. Further to the previous point, interviews with clients demonstrated a strong link of trust and collaboration between Economic Service Center staff and assisted firms (Section III) 6. Further to point 3, most clients found Paraguay Vende technical assistance to be of high quality and delivered in a timely fashion (Section IV.a.i). In a few cases, financial issues (i.e., fluctuations in currency exchange rates) led to substantial delays in delivery of assistance (Section II.b; the Fibrasa case). 7. Project outreach to the government and to the public was substantial and effective. In addition to hundreds of newspaper articles and presentations to interested groups, the project produced a half-dozen studies of high quality and policy relevance. The report on sesame, which challenged government policy on subsidies, is perhaps the most important, but several other reports, most notably the one on bio-fuels, were also important (Section VI). 8. Clients tended to value the assistance they received from Paraguay Vende in one of three ways: that it introduced them to information or expertise that they would have had difficulty finding on their own; that it gave them a partner that could help overcome bureaucratic obstacles to progress by the firm; or that the association with Paraguay Vende gave them credibility in dealing with other firms (Section II.b). 9. The TPS program has shown some promise. It is too young to expect that it would continue on its own after the end of Paraguay Vende on June 30. The work on formalization through TPS is important, though difficult, and the effort needs to continue through other mechanisms (section IV.a.ii). PARAGUAY VENDE II FINAL EVALUATION 21 SECTION 8: RECOMMENDATIONS FOR USAID AND FOR CHEMONICS For USAID/Paraguay 1. Given the great success of this project, USAID/Paraguay should incorporate its main features in future projects. The general principle of “cut your losses, but let your winners roll” does not seem to have been applied by USAID in this case. As both manufacturing and agriculture are critical to Paraguayan economic growth and poverty reduction, both roads to poverty reduction need attention. And it is unfortunate that the Paraguay Vende trademark, well respected in the country, is to disappear. 2. Paraguay still needs improvements, big time, in its business environment. USAID is better at this than other donors, so work in this area should continue. There were some notable successes with the previous government under the MCC Threshold Program, but these have not yet been repeated with the current government. As this is important to long-run development in Paraguay, USAID should continue to look for opportunities in this area. 3. The TPS Program has some promise, but needs to continue, at least in modified form for a significantly longer period of time if the potential results are to be achieved. The Paraguay Productivo project should seek ways to integrate TPS into its portfolio of activities, and also support the efforts of the Paraguayan government to register informal businesses that was assisted by Paraguay Vende. 4. Three years is too little time for projects like Paraguay Vende and Paraguay Productivo. Most three-year projects involve a start-up period of six months or so, and so are under immediate pressure to produce short-term results. Four years is a much more reasonable implementation period for projects of this type that should involve significant institution￾building and development of better technologies, with the prospect of extending the project if it appears to be producing important results. For Chemonics 1. The Paraguay Vende value-chain approach is a good model, which should be replicated elsewhere. 2. The M&E system used in Paraguay Vende is a valuable tool for assuring that reported results conform to clearly-established criteria. It is also worth repeating. 22 PARAGUAY VENDE II FINAL EVALUATION ANNEX A: PERSONS INTERVIEWED Paraguay Vende Reinaldo Penner, Chief of Party Mercedes Juvinel, Monitoring and Evaluation Chief Ricardo Lesme Dalles, IT Specialist Melissa Martinez, Communications Coordinator Francisco Fretes, Director, Coronel Oviedo ESC Francisco Latourrette, Advisor Ciudad del Este ESC Clients Bernardo Barbosa, CEO, Fibrac Armando Duarte, Owner Armando Duarte Herbs Dominique Frossard, Owner Lacto Cremo María del Rosario Franco, Co-owner, Escobas Santa Rosa Mariano Herrero, CEO Supermercado Herrero Pablo Santacruz, Owner Agropoty Mustafa Shehade, CEO, MKMM Apparel Rubens Shih C.W., Qin Yi Mauricio Stadecker, Director of Arasy Organico and Hierbapar USAID Michael Kaiser, EG Office Fernando Balcevich Prieto Yergos, EG Office Other Organizations Nelson Amarilla Andino, Secretario de Industria y Comercio, Alto Paraná Santiago de Fillippis, Director, ParaguayPruductivo Henry Moriya, Paraguay Productivo Cristina Sanchez, Ministerio de Industria y Comercio PARAGUAY VENDE II FINAL EVALUATION 23 ANNEX B: BIBLIOGRAPHY Garay, Pedro, “Evaluación del Impacto en las Ventas y Empleos del Sector Sésamo a Partir de la Asistencia técnica brindada por el Programa Paraguay Vende II del Corredor Económico Centro” August, 2009. USAID, Paraguay Poverty Reduction Program: Paraguay Vende Project Final Report, June 2007. Paraguay Vende II, Quarterly Reports and Annual Work Plans Paraguay Vende, Biocombustibles: Alternativa de Negocios Verdes, 2009 Paraguay Vende, Sésamo: Innovación en Agronegocios, December 2009 Paraguay Vende, Welcome Yerba Mate: La Novedad en el Mundo de las Infusiones, May 2006 Paraguay Vende, Unidad de Monitoreo de Resultados, Manual de Medición de Resultados, November 2007. Riordan, James and Silvio Galeano, “Mid-Term Evaluation, Paraguay Vende, Chemonics International, February 19, 2009.