EVALUATION OF THE PARTICIPATORY AGRICULTURE PROJECT (PAPA) PROGRAM IN AZERBAIJAN Prepared for USAID/Azerbaijan Regional Mission for the Caucasus Prepared by David Neubert Checchi and Company Consulting,Inc./The Louis Berger Group, Inc. Joint Venture 1899 L Street, NW Suite 800 Washington, D.C. 20036 Contract No. AEP-I-822-00-00022-00 July, 2003 Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 3 Executive Summary The Participatory Agriculture Project in Azerbaijan (PAPA) operated from March 2000 to June 2003, under a cooperative grant agreement with USAID. Its total budget for the 39-month implementation period was 2.99 million USD, of which 80% was provided by USAID and 20% was provided by Land O’Lakes and/or other partners, including client beneficiaries. PAPA’s objective was to address USAID’s Strategic Objective SO 1.3. The PAPA activity also successfully addresses a number of USAID’s IR’s, which are focused on; increasing market responsiveness of client firms (IR 1.3.3) and developing private membership associations, which provide services and are financially self-supporting (IR1.3.4). The project worked in two sectors of the agribusiness economy: livestock and hazelnuts. Geographically, PAPA’s activities were concentrated in northern Azerbaijan, including the districts of Oguz, Gakh, Gabala, Balaken, Sheki, Zagatala, Guba, and in the Baku metropolitan area. Project activities and budget focused on three specific areas. About 40% of the project's total budget was expended on association development within the livestock, dairy and hazelnut sectors. Approximately 35% of the budget went towards technical workshop training and firm level TA while about 25% of the budget was expended on marketing TA and implementation of a market information system. The project’s approach to the livestock sector was to form dairy processor and livestock farmer associations and provide the organization with fairly intensive TA. The Azerbaijan Dairy Processors Association consists of 26 members. The members have an aggregate market-share of about 70% of all processed dairy products sold in the country. This organization is one of the strongest associations formed under the PAPA project and it’s likely to become an effective and sustainable body in the long-term. The project also formed three regional Livestock Farmers Associations. These organizations have a total membership of 169 individuals. The project provided the farmers with an intensive TA program, which included training in animal health, nutrition, farm record keeping, barn reconstruction, etc. Members of both the Dairy Processors Association and the Livestock Associations have reported to the evaluation team that they have been very satisfied with the types and timing of the TA provided. All associations reported that one of the key benefits resulting from PAPA was the creation of an industry “community” or network, in which common issues could be discussed; this is unprecedented in Azerbaijan’s agri￾sector. In the hazelnut sector, the project organized the Azerbaijan Hazelnut Processors Association and the Azerbaijan Hazelnut Farmers Association. The Hazelnut Farmers Association has 1,150 members in six districts of northern Azerbaijan. Association farmers produce about 10% of the total nut crop with a farmgate sale value of about 2.25 million USD annually. Given the large number of farmer/ members spread over a wide geographic area, the project took a more extensive approach to providing TA to farmers. This was done mainly through the training of trainers and village level workshops. The approach seems to have worked as farmers report a significant increase in yields from existing orchards (up over 30% based on optimistic anecdotal information). Additionally, farmers have planted about 1700 ha of new orchards, which is important in that this reverses the trend of declining orchard area that has occurred since independence. The Hazelnut Processors Association consists of about 14 members and handles about 50% of the country’s hazelnut exports, valued at about 45 million USD annually. The project’s TA approach to this organization was to provide a mixture of workshop and firm level assistance. Members indicated that they are happy with the type, timing and quality of PAPA’s TA. The result has been the development of new in-shell markets in China, and an improvement in farmgate nut quality that resulted in higher incomes to farmers and better quality nuts delivered to the processors. The Hazelnut Processors Association is most likely sustainable in the long run. Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 4 Methodology This report is part of a set of two evaluations, which investigates the performance of the Land O’Lakes PAPA Project and the IFDC AMDA Project in Azerbaijan. The evaluations were prepared over a 26-day period between 5 June and 5 July 2003. The following report is the final evaluation for the LOL PAPA project and examines the project’s accomplishments in terms of its contributions to achieving the Mission’s Strategic Objective 1.3. The evaluation processes included a full review of all relevant project documents, including but not limited to the original project grant agreements, project quarterly reports and training materials prepared by the project. Additionally, the evaluators reviewed the applicable USAID Strategic Objectives and Intermediate Results indicators and examined if and how the projects under review were contributing to these targets. The evaluation team conducted interviews with project managers and staff, as well as association members, farmers, SME￾food processors, exporters and marketers. The organization and questions addressed in the evaluation report directly follow the questions, order and organization found in the Scope of Work prepared by USAID Baku. In some cases, the wording of the questions has been modified for reasons of language efficiency. Additionally, several of the questions raised in the SOW have been collapsed into a single sector heading in an effort to minimize text redundancy. Acknowledgments The preparation of this report would not be possible without the generous assistance of the entire Participatory Agriculture Project in Azerbaijan (PAPA) staff. Specific thanks goes to Chief of Party David Blood and Deputy COP Jeton Starova for their assistance through the entire process of the evaluation. Also, special thanks goes to Peter Duffy and Jeff Lee of USAID-Baku for their work in organizing the evaluation team’s field work and providing insight into the unique nature of the Azerbaijan operating environment. Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 5 Map of Azerbaijan Acronyms AMDA Agro-Input Market Development in Azerbaijan AZ Azerbaijan CHF Cooperative House Foundation COP Chief of Party EOP End of Project GOA Government of Azerbaijan GMO Genetically Modified Organism HACCP Hazard Analysis and Critical Control Points IFDC International Fertilizer Development Center IR Intermediate Results – USAID IRC International Rescue Committee LOL Land O’Lakes PAPA Participatory Agriculture Project in Azerbaijan RECP Rural Enterprise Competitiveness Project SO Strategic Objectives – USAID SOW Scope of Work USD United States Dollar USAID United States Agency for International Development USDA United States Department of Agriculture YR Year Project Target Area Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 6 Table of Contents 1.0 General 1.1 Introduction 1.2 Interventions Most Effective in Achieving SO’s and IR’s 1.3 Recommended Changes In Implementation Resulting in Enhanced Results 1.4 Opportunities for Integrating Program Components Resulting in Greater Impact 1.5 Unexpected Benefits and Impacts As a result of Project Activities 1.6 Unexpected Negative Impacts The Program Needs to Addressed 1.7 Appropriateness of Activities for Agribusiness Development in Targeted Areas 1.8 Appropriateness of Assumptions and Activities Relative to Operating Environment 1.9 Intervention Cost-Effectiveness 1.10 Follow-up Activities that Should Be Considered to Maintain Momentum 2.0 Agriculture and Agribusiness 2.1 Opportunities for Improving the Agribusiness Sector’s Marketing, Volume, Product Quality and Business Skills 2.2 Project Effectiveness in Capturing Opportunities to Improve the Agribusiness Sector 2.3 Market Access for Products and Services Produced by Client Entrepreneurs 2.4 The Nature and Effectiveness of Assistance Provided 2.5 Constraints to Reaching Association Members 2.6 Overall Impact of Program-Assisted Associations/Enterprises on Agribusiness Development 2.7 Business Development Activities' Contribution to Improved Productivity 2.8 Enhancing Business Development and Training to Improve Productivity 2.9 Effectiveness in Facilitating Client Entrepreneurs' Access to Finance 3.0 Adoption of New Practices 3.1 Beneficiaries' Adoption of Recommended Practices 3.2 Types of Practices Adopted By Beneficiaries 3.3 Program Modification to Maximize New Practice Adoption Rates 3.4 Primary Source of Information Concerning Business Practices 3.5 Other Key Channels of Business and Technical Information 3.6 Client Satisfaction with the Quality of Technical Assistance Provided 3.7 Timeliness of Technical Assistance Delivery to Clients Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 7 4.0 Business Association Development and Capacity Strengthening 4.1 Overall Progress and Approach to Association Development 4.2 Effectiveness in Developing Association Capacity to Affect Change in Agribusiness-Policy and the Regulatory Environment 4.3 Strengthening Associations' Advocacy Capacity 4.4 Appropriateness of Approach in Organizing Associations to Serve as a Forum for Agri-Policy Dialogue With the Government 4.5 Impacts and Benefits to Association Members 4.6 Appropriateness of Training Materials and Marketing Information Supplied by the Implementer 4.7 Effectiveness of Associations in Delivering Improvements in Agri￾production Productivity, Product Quality, and Other Services 4.8 The Role Business Associations Play in Agribusiness Development 4.9 The Main Areas on Which Business Associations Need to Focus 4.10 Political and Economic Representation of Project Assisted Associations 5.0 Sustainability Issues Relating to Recommended Practices 5.1 Sustainability of Project Outcomes 5.2 Adopted Practices Most Likely to be Sustainable 5.3 Adopted Practices Least Likely to be Sustainable 5.4 Options to Increase Sustainability of Adopted Practices 5.5 Current Number of Operating Client Enterprises and Associations 5.6 Client Leveraging USAID Support Through Other Donors 5.7 Association Membership Dues and Fees 6.0 Monitoring and Evaluation 6.1 Achieving of Project Targets 6.2 The Quality and Reliability of Performance Indicators 6.3 The Use of M&E Data and Anecdotal Information by Project Management 6.4 Options to Better Utilize M&E Data and Anecdotal Information by Management 6.5 Effectiveness in Tracking The Progress of Client Entrepreneurs Annex Annex A Hazelnut Market Information Network Development Model Annex B The Use of Public Awareness Campaigns in Market Development Annex C Association Foundation Meeting Dates Annex D Azerbaijan Country Profile Annex E Evaluation SOW Annex F USAID Azerbaijan Strategic Objectives Annex G PAPA Activity Final Evaluation PowerPoint Presentation Master Slides Annex H Interviewee Contact List Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 8 Final Evaluation of the PAPA Program - Azerbaijan 1.0 General 1.1 Introduction The Participatory Agriculture Project in Azerbaijan (PAPA) began in YR 2000 and operated under a cooperative grant agreement with USAID through June 2003. Its total budget for the 3-year implementation period was 2.99 million USD, of which 80% was provided by USAID and 20% was provided by Land O’Lakes and/or other partners, including client beneficiaries. PAPA’s objective was to address USAID’s Strategic Objective SO 1.3. This SO aims to accelerate growth and development of private small and medium size enterprises in target areas. The indicators of success in achieving this SO include growth in number of assisted firms and increase in employment for both men and women in the target areas. The PAPA activity also addresses a number of USAID’s IR’s, which are focused on; increasing market responsiveness of client firms (IR 1.3.3) and developing private membership associations, which provide services and are financially self-supporting (IR1.3.4). To achieve these targets, the project is working in a number of areas as prescribed by USAID’s 2001-2003 Strategic Plan for Azerbaijan. These sub-level IR’s include; improving enterprise business, technical and marketing skills (IR 1.3.3.1), increasing adoption of grades and standards (IR 1.3.3.2) and improving access to market information (IR 1.3.3.3). As the PAPA activity is currently operating under a no-cost project extension and is scheduled to reach EOP by end June 2003, this report is a final evaluation. In the future, USAID may likely choose to build on the accomplishments of the PAPA activity by incorporating its intellectual and human capital into other USAID activities that are currently operating or are in the planning stages. 1.2 Interventions Most Effective in Achieving SO’s and IR’s The most effective interventions delivered by the PAPA project have been in the area of training. The project has operated a wide variety of technical and association development training activities targeting individual firms and farmers. As of the end of YR 2002, the project had provided training to 4009 persons employed by client beneficiary firms and/or farm owners-operators. PAPA’s project impact survey data shows that of the total number of persons trained (4009 people), 3478 individuals are applying the skills and knowledge acquired from the training at their present jobs. This equates to an 87% skills adoption rate. This high skill application rate by clients demonstrates the effectiveness that PAPA has had in selecting training subject areas that are relevant to client needs and delivered in a practical and comprehendible way. PAPA has contributed to USAID’s SO 1.3 and the targeted IR’s by helping client firms to become more competitive, increase gross revenues, increase capital spending, introduce new produces to the marketplace, make better business decisions based on market information and to improve product quality and standards. 1.3 Recommended Changes In Implementation Resulting in Enhanced Results In USAID’s future agri-sector activities, there are a number of specific steps that can be taken to help enhance project outcomes. To date, there has been a concentrated effort to develop officially registered agribusiness associations; this effort has been meet with considerable resistance within the GOA. The GOA has reacted to this effort by stalling the registration process through bureaucratic inaction. Given the reluctance of the GOA to move forward with a policy to register industry associations, USAID should consider other less formal ways Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 9 of organizing its beneficiaries. The association issue is not a hill to die on; there are other ways of achieving the desired outcome without confronting the GOA on this matter, at this time. For smaller regional organizations, such as a provincial livestock association, USAID implementers should consider forming other non-registered groups through which TA can be delivered (alliances, societies, clubs, etc.). These groups can be organized under many of the same principles and bylaws that govern a formally registered association without having to confront the GOA on the registration issue. USAID’s efforts to foster association development in Azerbaijan should not be completely abandoned. Implementers need to be more selective about which organizations should attempt official registration. Some of the deciding criteria for registration should include a group’s strong desire to become a registered association and the group should have an economy of scale, which would allow for its sustainability. That is, the group should be large enough in either number of individuals or financial terms to insure that the future association can be economically solvent and sustainable. Areas that future USAID projects should focus on to achieve enhanced SO and IR results are in the fields of improved business practices, food standards, sanitation, and market information. This TA should be delivered directly to food processing companies. By focusing TA at the firm level, implementers can also reach raw product producers (farmers) that supply such firms. 1.4 Opportunities for Integrating Program Components Resulting in Greater Impact It is recommended that in the future, when implementers consider resource allocation (for example: shop floor TA, workshop training, grants, etc.), priority should be to given to companies that are affiliated with donor efforts in industry organizations. This can be determined through their participation in project-supported alliances, societies, clubs, associations, etc. There are also future opportunities to create greater synergies between projects. To date, the PAPA project has worked with ACDI/VOCA’s Farmer-to-Farmer program, IFDC, CHF and IRC’s community/ economic development activities. These alliances should continue to be encouraged by USAID and implementers should understand that it is in their self-interest to cooperate and work with allied projects rather than assume a posture of turf protection. To date it appears that the relationships between USAID funded agri-sector projects are good. 1.5 Unexpected Benefits and Impacts As a Result of Project Activities One of the unexpected key benefits that the PAPA project’s association development work has had is creating a sense of community within the livestock, dairy processing and hazelnut sectors. In numerous interviews, clients explained how before the PAPA project started its association development work, individuals and firms had a “go it alone” outlook, and in most cases did not personally know the other firms/ individuals operating in the sector. The PAPA project was able to transcend many of the cultural barriers to building community within these sectors. It is difficult, if not impossible, to place an economic value on this benefit; however, creating a sense of community among the participants of an industrial sector is a key first step in creating an effective and sustainable industry association. 1.6 Unexpected Negative Impacts The Program Needs to Address There are two unexpected negative effects that have resulted from the PAPA project. The first is the friction between USAID and its implementing partners with the GOA over the issue of official registration of industry associations. To date, this has not become a major sticking point, but attention should be paid to this matter so it does not become a larger issue in the future. The second unexpected result of PAPA’s work and USAID’s activities in general has been a rising of expectations by client firms and individuals. This is not uncommon among donor￾funded projects in general, so one may say it is not totally unexpected. In the case of Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 10 Azerbaijan, several factors have coalesced to create a fairly high level of expectations as to what the project can do. In the former Soviet Union, all resources flowed from the state; therefore clients of projects such as PAPA view these activities as gateways to new resources (and a wealthy gateway, given the fact that it is a US government project). The second factor leading to unrealistic expectations is a lack of understanding of how USAID approaches economic development and the capacity to deliver benefits to clients. The senior PAPA staff understands this situation and has tried to establish realistic expectation levels among clients. Nevertheless, some clients still appeal for money to buy business related inputs, preferring this approach to one that emphasizes training and the development of intellectual capital within their organizations. 1.7 Appropriateness of Activities for Agribusiness Development in Targeted Areas The PAPA project is following a logical set of activities to achieve its SO’s and IR’s in target geographies and industries. As outlined in section 1.3 (Recommended Changes In Implementation Resulting in Enhance Results) of this report, there are improvements that USAID can make when implementing future activities, but in general, the strategy of providing TA at the firm level and using the client agribusinesses as conduits to reach the farmers that supply these businesses is sound. The appropriateness of focusing a larger amount of project resources into association development is less clear. Certainly some future work in association development should continue but implementers need to be selective when it comes to choosing which organizations they assist in attempting to attain official association status. 1.8 Appropriateness of Assumptions and Activities Relative to Operating Environment There has been a mismatch between the assumption that PAPA could rapidly organize and register associations and the official (or unofficial) policy environment that does not encourage the formation of industry association. Additionally, the assumption that small provincial-based associations will develop the managerial and financial critical mass necessary to achieve sustainability is yet to be proven. The endeavor to develop such associations is commendable and complements USAID strategy of helping Azerbaijan become a more democratic society; however, it is likely that this will be a long slow process, when it comes to organizing small rural associations made up of individuals with limited financial resources. 1.9 Intervention Cost-Effectiveness The monitoring and evaluation system developed for USAID in YR 2001 was not designed to capture data relative to the cost effectiveness of USAID’s projects operating in the agribusiness sector. Rather, this M&E system focuses on gross employment numbers, training, adoption of new skills and standards, the number of associations created, etc. As a result, there has been no systematic collection of cost per activity data on which to base a cost-efficiency calculation. PAPA’s accounting system is designed to address the information needs of USAID auditors. The type and organization of financial information that is required in an audit is significantly different from the type of information that is required to evaluate the cost effectiveness of a project. The evaluation Team recognizes that PAPA’s primary fiscal responsibility lies in meeting the needs of USAID and GAO reporting requirements. Nevertheless, the questions of cost effectiveness were raised in the Evaluation Team’s Scope of Work and the following is an attempt to address this question in an impartial way. The methodology used is not perfect, as the type of data needed to accurately undertake a cost effectiveness analysis is not available in the Project’s current accounting system. From a project management and evaluation standpoint, the team appreciates that there is a great need to establish a simple and effective job-costing system within PAPA and other USAID projects. At the same time, the team realizes that is was not a requirement in the Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 11 USAID-LOL cooperative agreement to do this. If the Project developed and implemented a job-costing system, there would be an impact on the Project’s administrative costs, even though these additional costs would be offset by efficiency gains and improved transparency. To calculate cost-effectiveness, the system would need to track costs per client, costs per activity, and impact per client measured in financial terms. Project management estimates that about 40% (1.2 million USD) of the total PAPA budget was expended on association development activities such as organization, registration, and association management TA, etc. This equates to about 883 USD per member, based on 1,359 members. The cost per member is significant but not outside of the unit cost range experienced by other USAID funded agribusiness association development activities1 . Estimated Costs per Project Activity Total Expenditures of 3 Million USD over 3 Years Market Information 25% Firm Level TA & Workshops 35% Association Development 40% Figure 1, Source: PAPA The “New Market Opportunities Realized” in the PAPA’s quarterly impact statistics measures the project’s market information development task. This indicator measures the various activities that PAPA is involved in which disseminate market information. For example, the dairy sector newsletter (with a circulation of 400 copies per quarter) counts as one “New Market Opportunities Realized” as does the development of a website for the Azerbaijan Hazelnut Processors Association. Based on this methodology and a total task budget of approximately 750,000 USD over 3 years (25% of total expenditures), the new market opportunity activities costs PAPA about 3,676 USD per “New Market Opportunity” or 552 USD per association member. It is estimated that about 35% (about 1,050,000 USD) of the PAPA project's total budget has been expended on firm level technical assistance and technical workshops. Based on this level of expenditure, PAPA’s cost per person per training session was 262 USD. If measured as the cost of adopting knowledge acquired in PAPA’s training activities (i.e. practical application on the shop floor of skills learned in PAPA workshops) the unit cost per adoption is 302 USD. The training provided by PAPA resulted in the development of new production, services and markets for client firms. The project cannot claim that it is solely responsible for all new products, services and markets that clients have developed over the past 3 years, but certainly it has made a very significant contribution to their development. In gross terms, the project 1 USAID Egypt, ALEB Project mid-term evaluation in YR 2001 showed that after 27 months of agribusiness association development work, the project had expended 1966 USD per association member. Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 12 spent 1,111 USD for each new product, service and market that was created by client firms between project startup to end YR 2002. 1.10 Follow-up Activities that Should Be Considered to Maintain Momentum The loss of momentum created by the closure of the LOL project is a major concern. It can be assumed that a significant amount of goodwill, organization and momentum will be lost between the closing of the PAPA project and the startup of new USAID agribusiness activities2 . This loss will be exacerbated if the new USAID agribusiness activity spends a significant amount of time studying the competitiveness situation in Azerbaijan and defers from actively working with associations and SME’s at the beginning of the new project. There are a number of future agri-sector activities that should be considered by USAID under SO1.3 these include: • Expanding firm level TA to agribusinesses, under the criteria that beneficiaries are members in an industry organization recognized by USAID (this may include but not be limited to a trade association, society, alliance, etc.). Specific areas of TA may include but not be limited to business and financial management, marketing, new product development, quality assurance/control and process operations. • Provide a small matching grant facility to encourage investment in improved technology, rural and urban SME’s, as well as cottage industries. Develop close working institutional linkage between the ACDI/VOCA Agro-Credit program and US Import/Export Credit programs to assist larger (more credit worthy) businesses with sourcing the credit necessary to purchase food-processing equipment from US suppliers. All of these activities should be done in concert with expanded firm level TA in an effort to strengthen business, marketing and technical skills. • Expand the market information system - This should be done in concert with a private company such as a newspaper, monthly agricultural magazine, cell-phone text messaging provider, radio or TV channel (or a combination of the forgoing). By partnering with an existing media/ communication outlet, the implementer will help the media company build capacity in agri-market information onto an existing distribution network. This will help insure sustainability of a functioning market information system over the long-term. • Expand international learning activities for client firms with education and market development trips to model Eastern European countries (such as Poland, Hungary, Bulgaria, Slovenia, etc.) to see how policy, agribusiness and markets operate in countries that have more effectively managed the transition from a centrally planned economy to a market economy. • Developing a nationwide Seal of Quality program for meat, milk and other processed food products. This activity should be supported with a significant media campaign that introduces the SOQ activity to the public. The SOQ activity will need to be structured under a national oversight body. Typically, this is done under an association of food processors; however, given the reluctance of the GOA to register new associations; it would be prudent to seek GOA approval on this concept before launching a push to develop the SOQ plan. This assumes that an association would be selected as the vehicle to move the SOQ program forward. 2.0 Agriculture and Agribusiness 2.1 Opportunities for Improving the Agribusiness Sector’s Marketing, Volume, Product Quality and Business Skills At the time that this Evaluation Report is submitted to USAID, the PAPA activity will already be closed and the opportunity to improve agribusiness under the project will have passed. 2 The new RECP agribusiness activity is scheduled to start in the third or fourth quarter of 2003 Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 13 There are, however, future agribusiness activities and currently operating projects (IFDC and ACDI/VOCA) that could incorporate lessons learned from the PAPA project and use this knowledge to expand opportunities to clients. Future USAID activities need to approach agribusiness development starting with the market. Farmers in Azerbaijan can produce food and fiber, but at this point, the fragmented market structure does not allow farmers clear channels to market their produce through. As a result, livestock farmers kill, clean and sell the animal products on the side of the road. Produce and forage crop farmers operate in much the same way. There are opportunities for future USAID activities to work with processors, wholesalers and retailers in the area of developing and implementing quality standards for goods purchased from local producers. Buyers need to structure a pricing mechanism that rewards quality. This approach has proven effective in the hazelnut industry, where the PAPA project worked with the Hazelnut Processors Association and the Hazelnut Farmers Association to establish an efficient grading and pricing system for in-shell nuts. Market price information has a short shelf life. To a trader, market information that is more that 48 hours old starts to lose value quickly. The Azercell SMS national network is a significant unexploited resource in distributing timely market information. Future USAID activities should look for ways to use the network to provide price, quality and volume information on agricultural products such as fresh fruit and vegetables, forage, nuts, etc. The cost of delivering this information via SMS is very cheap and implementers should be encouraged to explore and test the application of this technology. A functioning SMS market information is economically valuable and may become a future revenue source for associations or other agribusiness organizations. The print media can also be used to deliver market information, but with the exception of the newspaper, this information tends to be historical - showing trends (i.e. prices last week, last month and a year ago). Implementers should be encouraged to work in partnership with private media organizations to assist them in making market information part of their “regular content”. In this way, the delivery of market information can become sustainable. Other areas that pose opportunities include developing a "Seal of Quality" program in Azerbaijan, which includes a significant public awareness campaign. The public awareness campaign needs to impart to the consumer “ buy products with the Seal - it means you are buying quality, value and supporting the Azeri farmer”. Any future USAID supported agribusiness sector activity should focus resources at the firm level. These are the companies that buy raw product from farmers. The nature of the TA should include but not be limited to; improved business management in areas of cost analysis/ management, raw and finished quality specifications and quality based pricing systems for raw and finished goods, processing operations, packaging, storage, distribution, cash flow management. There should also be a significant amount of resources focused on farmers to assist them in improving efficiency, quality, and reducing cost. The effort to target the farmer should be channeled (delivered) through the processor(s) to whom the farmer is selling. In this way, capacity is build at the farm and firm levels. Additionally, as the firm is the buyer of the farmer’s products, the farmer will listen and abide by recommendations (or requirements) imposed by the buyers-processors. 2.2 Project Effectiveness in Capturing Opportunities to Improve the Agribusiness Sector The PAPA project has been effective in creating/ capturing market opportunities for its clients. Figure 2 shows the number of new products, services and markets developed over the 6 quarters period measured under the revised USAID M&E system. Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 14 It is not realistic to assume that Land O’Lakes project staff were responsible for all of the new products, services and markets, developed over the observation period; but after interviewing numerous project clients, it is reasonable to assume that the project had a very significant positive effect on maximizing the number of new products, services and markets developed. Client Firms New Products, Services and Markets 0 100 200 300 400 500 600 700 800 Baseline 7 8 9 10 11 Project Quarter No. of New Products, Services & Mkts New Products & Services New Markets Figure 2, Source: PAPA3 2.3 Market Access for Products and Services Produced by Client Entrepreneurs For the most part, Azeri markets are accessible to farmers, processors and traders. Official corruption is an issue, but it is part of the cost of doing business in the Azeri economy. Comparative anecdotal evidence suggests that Azeri hazelnut processors/ exporters official corruption costs are as much as 5 times higher that corruption costs in the Republic of Georgia, in the same industry. In both of these countries, the payoffs are not usually handled directly by the processors/ exporters, but are built into the cost structure of the freight forwarder (transport company) and included in the overall transport charges. In the dairy sector, the GOA recently tried to require ice-cream street vendors to be licensed by the government. The Association of Dairy Processors fought this regulation and won. At the current time, street ice-cream vendors do not need a license to operate, but they still must register with the government. This was a small step in developing open markets. 2.4 The Nature and Effectiveness of Assistance Provided The graph below, Figure 3, examines the number of new market opportunities that were delivered to project clients. In addition, the project provided training to groups and individual 3 Baseline information in Figures 2, 3 and 11 was taken from Project 4th Quarter data Web Based Market Information Market access for the hazelnut sector has been enhanced with the development of the www.azhazelnuts.com marketing website. This website, developed in concert with PAPA and the AZ Hazelnut Processors Association, offers basic information about the Azeri-hazelnut industry and provides contact information for exporters. The website is a good start but can be significant improved with a wide variety of industry information. In the future exporters would like to offer organic hazelnuts to the EU market and this product will require extensive certification/ documentation which can be added to the website Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 15 in the areas of; farm record keeping, animal health, barn reconstruction, artificial insemination, farm and processing plant sanitation, ISO, HACCP and new product development. New Market Information Opportunities Realized by PAPA Clients, Cumulative Data 3 75 100 126 170 204 0 50 100 150 200 250 Baseline 7 8 9 10 11 Project Quarter No. of New Mkt. Info Opps. New Market Information Opportunities Figure 3, Source: PAPA Client Firms Adopting and Complying With New Product Standards 28 26 24 21 7 18 0 5 10 15 20 25 30 Baseline 7 8 9 10 11 Project Quarter No. of Client Firms Firms Adopting to and Complying With New Product Standards Figure 4, Source: PAPA Over the course of the 3-year PAPA activity, sales of client milk processors have increased from 25,000 lt. per day to 96,000 lt. per day. The dairy sector clients have also increased raw milk quality with reductions in bacterial contamination, increases in milk fat levels, improved animal nutrition and health. Since the PAPA project began working with the hazelnut growers, processors and exporters, the sectors has expanded its orchard area, farm production, export volume, improved its quality, and entered new markets, including Russia and China. These new markets have great up-side potential. The project had success in assisting clients to develop and adopt new product standards as described in Figure 4. Much of the success in this area has been the result of firm level TA in Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 16 the hazelnut processing and dairy processing sectors. Some of the new standards that were adopted were a direct result of training provided to clients in quality control (for example, moisture and aflotoxin control in hazelnuts). In the dairy sector, several clients have been actively adopting ISO and HACCP protocols so that their products comply with international standards. 2.5 Constraints to Reaching Association Members There are several types of constraints that the Land O’Lakes project faced in the course of its implementation. One of the most important constraints faced by any business development activity in Azerbaijan is cultural; historically business linkages in the Azeri culture are based on family and clan relationships. There is no cultural or business precedent to the idea that a group of companies operating in a specific sector should sit down together and discuss matters as equals. However, this is exactly what is involved in association development activities and this is an important constraint. Many people (business owners and managers) in the Azeri culture are not comfortable with the idea of sitting around a table with their competitors talking about their problems. They are also uncomfortable exposing any part of their company to government and fear that by raising their profile they are inviting government to demand payoffs and/or inviting a visit from the tax authorities. This fact also has a direct impact on a firm’s interest and willingness to take part in lobbying activities. In addition to working with businesses, the project also worked with farms via their links with the PAPA assisted Hazelnut Growers Association. The large number of farms is itself a constraint and this is particularly true given the small total budget PAPA had to work with. To reach farmers, the project developed a conduit, and in PAPA’s case this was the association and the training of trainers. These trainers were employed to train hazelnut farmers on improved farming and post-harvest handling methods through village level workshops. Given the GOA apprehension to undertake the wholesale development of associations, the project spent a considerable amount of time and money working on this issue. Therefore, the GOA’s own policy on association development has become a constraint to realizing IR1.3.4 and LOL work toward this objective. 2.6 Overall Impact of Program-Assisted Associations/Enterprises on Agribusiness Development A key part of SO 1.3 is “Growth of assets of assisted enterprises by target area”. PAPA tracked the change in client assets on an annual basis (see Figure 5). The project defined assets as any animals, orchards, equipment, or infrastructure owned by the client. Under the PAPA definition, cash, receivables, and inventories were not included in the asset value calculation. The client aggregate asset value has increased over the life of the project as the client base has grown. The large increase between end 2001 and end 2002 was in part the result of establishing the Hazelnut Farmers Association and adding the value of these farmers’ Key Constraints to Association Development in Azerbaijan • No cultural precedents in joining with other people outside your family or clan to discuss and act on problems that affect your business • Difficulty in achieving an economy of scale, especially for low income rural groups • Board members of larger associations often perceive that they do not share a common interest • GOA policy of not registering new associations makes it difficult to collect dues, as the association is unable to open a bank account Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 17 orchards to the client cumulative asset calculation. The problem in measuring the project's overall impact on asset values is simply that PAPA kept growing its client base over time. Thus, they ended up measuring the growth in their client’s aggregate assets rather than determining if their clients were growing their assets. This was a fundamental flaw in the project’s M&E system design and it should be corrected in the futue. Client Firms Asset Value 0 20 40 60 80 End YR 2000 End YR 2001 End YR 2002 USD x Million Asset Value 5.28 14.2 59.76 End YR 2000 End YR 2001 End YR 2002 Figure 5, Source: PAPA At present, there are about 25 medium sized domestic processors and 6 exporters in the Azerbaijan Hazelnut Processors Association. The Hazelnut Processors Association has 13 members and accounts for over 50% of all hazelnut exported from Azerbaijan. Over the past several years, there has been an expansion of new private processing hazelnut￾processing facilities. The high level of capital investment into the sector has resulted in a processing over-capacity. The investment has been driven by the perception that it is a highly profitable sector. The effect on the industry has been that few (if any) firms operate at optimal levels of efficiency. This trend has been compounded by the Azeri tax environment, wherein farmers are exempted from most taxes through the year 2003, and exporters from VAT on export crops such as hazelnuts. The PAPA project was in no way responsible for the over-capacity condition, but it has helped processors by making available more nuts through its work with the Hazelnut Growers Association. This increase in capacity has benefited the farmer by encouraging price competition at the farmgate and encouraging farmers to expand their planted area. Project client farmers have reported that they added an additional 1700 ha of new orchards under the PAPA project. At maturity (in about 5 years after planting), these orchards will generate about 1.5 million USD of additional farmgate income. The export revenue generated from these new trees will be about 4.0 million USD. 2.7 Business Development Activities' Contribution to Improved Productivity One of the most common ways of measuring changes in productivity is to Addressing GMO Policy Azerbaijan has not yet developed a GMO policy. USAID’s overall development strategy focuses on the GMO issue by encourage governments to develop science-based policy. Given the Areri-competitive advantage in alfalfa and the fact that it’s an animal feed rather than human food, this maybe a good first crop to introduce GMO technology to Azerbaijan. In the next 2 to 3 years several US seed companies will have Roundup Ready Alfalfa commercially available. During this time USAID should work to create a regulatory environment, which will allow the introduction of RR-Alfalfa in Azerbaijan. Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 18 use Task Level Efficiency (TLE). Task Level Efficiency is used to compare productivity between companies and countries. The equation is usually designed to show a ratio, measuring USD gross revenue per worker and/or output per worker by weight, volume or manufacturing unit. The M&E system employed by USAID in Azerbaijan does not consider TLE and does not require projects (PAPA or others) to calculate these ratios on a client-by￾client basis. To help provide a more transparent picture of client productivity, it is recommended that USAID ask implementers to collect data on gross sales value, number of employees and units of production. In this way USAID can, in the future, construct a model and track changes in productivity between businesses, associations and implementers. Number of Persons Employed by Client Firms 0 2000 4000 6000 8000 End YR 2000 End YR 2002 Number of Employees Male Employees 430 3129 Female Employees 264 2967 End YR 2000 End YR 2002 Figure 6, Source: PAPA The annual employment data collected by the project (see Figure 6) considers the total number of workers employed by client firms. Most of the increase in total employment is a function of PAPA’s ever expanding client base, although some client firms did expand their work force. At EOP in YR 2002 (before the no-cost extension started), the client workforce totaled 6096 employees. Of this total, 51% were male employees and 49% we female employees. This was a significant improvement over end YR 2000 data when 62% of client employees were male and 38% were female. There is anecdotal evidence that client productivity has increased following PAPA TA activities. For example, the eight founding firms in the Azerbaijan Dairy Processors Association doubled their daily output between the 6th quarter and 11th quarter of the project. However, as can be noted in Table 1, the project has realized significant growth in terms of its client base and volume of milk processed, but this data is not disaggregated in a way that allows for the productivity of an individual processor to be scrutinized. 2.8 Enhancing Business Development and Training to Improved Productivity Training has been one of the LOL project’s strengths. Figure 7 investigates the cumulative number of people trained under the PAPA activity. By the end of 2002, the most recent data available, the project had trained over 4000 persons. In some cases, one person may have attended more than one training activity. Growth in Daily Output by the Azerbaijan Dairy Processors Association Table - 1 Quarter No. of Processors Milk, Lt. Processed Mean Lt. Per Firm 6 8 24,940 3,117 7 26 64,800 2,492 8 26 70,150 2,698 9 30 76,600 2,553 10 34 78,900 2,321 11 37 88,900 2,403 12 37 96,100 2,597 Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 19 Client Firm's Employees Training Opportunities, Cumulative 677 2274 2720 3624 4009 0 1000 2000 3000 4000 5000 7 8 9 10 11 Project Quarter Number of Employees Trained Employees Trained Linear (Employees Trained) Figure 7, Source: PAPA To enhance future training activities, USAID should stay focused on markets and at the firm level. Once a business sees a market opportunity its management is much more receptive to training which will aid in its effort to capture perceived opportunity. 2.9 Effectiveness in Facilitating Client Entrepreneurs Access to Finance In year one of the PAPA activity, the project employed one local staff person to assist clients to access credit. The project itself had no credit component, so the local staff person worked with clients to identify and obtain commercial credit from local banks or other institutions. After one year, the project showed little success in this area, and rather than keep continuing to spend the project's limited resources on this activity, the decision was made to cut losses and focus on association development, TA and market information. 3.0 Adoption of New Practices 3.1 Beneficiaries' Adoption of Recommended Practices4 When interviewing members of one Livestock Associations, the comment was made that the training was good but sometime too theoretical; the farmers wanted more practical training. In this case, “practical” meant they wanted LOL to provide de-worming drugs. They were less interested in learning how to set up an on farm screening system for parasites. This boils down to “is it better to give someone a fish or teach them to fish”? Of course, the farmers want the fish, or in this case de-worming drugs5 to apply to their herd, regardless if they have identified parasites in the herd or not. From a sustainability standpoint, the best path maybe incorporating some of the theory of how to set up parasite screening program along with linking clients with IFDC input dealers that would (sell) them quality de-worming drugs. 4 This sectors also addresses the question raised in the SOW’s section 2; how effective have associations been in supporting the members in the following areas: agri-production, improving productive, product quality, and providing other services? 5 Association member specifically asked for drugs manufactured in the West, due to their better quality. School - Milk Education Program One of the most popular promotional activities, that PAPA implemented, was the school milk promotion/ education program. The project provided 11 elementary schools in the Baku area with posters and printed information on the benefits of pasteurized milk in the diet and food safety. After the campaign finish one processor told the Evaluations Team that milk sales in the areas around the schools increased significantly. The Dairy Association plans to continue this activity. Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 20 Adoption rates of skills taught in PAPA workshop training and firm level TA activities are high. Figure 8 illustrates the adoption of skills by client employees and owners. When the data in Figure 8 are compared with Figure 7, the data shows an 87% adoption rate resulting from PAPA training activity. The message in these high adoption rates is that LOL is providing relevant training topics and delivering them effectively through a combination of workshops and follow up firm level TA. Client FirmsTrained Employees Using New Skills 482 2356 3353 3403 3478 0 500 1000 1500 2000 2500 3000 3500 4000 7 8 9 10 11 Project Quarter Client Firms Employees Trained Employees Using New Skills Figure 8, Source: PAPA 3.2 Types of Practices Adopted By Beneficiaries Training and the adoption of new skills and methods need to be viewed as a long-term activity in Azerbaijan. Technical skills are adopted faster than managerial skills, but it is important for the success of local agribusiness sector that training in both areas be provided and adopted. Clients adopt practices that provide a clear benefit to the user. In most cases, LOL has found that training in technical areas such as milk processing plant operations or practical on-farm training in tree crop or animal husbandry has higher adoption rates than does training in managerial areas such as personnel management. 3.3 Program Modification to Maximize New Practice Adoption Rates The training methodology employed by LOL was both straightforward and effective. The PAPA project worked with association members to identify training needs then organized workshops to train individuals and trainers. These activities were then followed up with in￾house firm level TA, which addressed and rectified specific problems experienced by the client. This is sound methodology, but improvements can be made to streamline the activity. In the future, the RECP and other agri-sector activities should commit more resources to firm level TA. Training should stress technical areas, but should also include training on managerial topics. Shop floor supervisors and cottage industry owners should have the opportunity to be trained in topics such as manufacturing cost analysis, market research and product development (to name a few). There should be an effort to move away from the current rigid vertical information structure to one that incorporates more horizontal information flow. Owners and employees should be provided with opportunities for training Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 21 in areas outside their narrow area of responsibility. This will results in a more efficient and flexible workforce. The PAPA project allowed its associations to select training topics that they were interested in. In a few cases, consultants in Azerbaijan already providing TA to one group would be taken to a second association even if it were not requested. In general, this is a good use of project resources and in some cases; it is simply a good idea not to rely only on the request of the group when selecting training topics. PAPA’s management has broad experience and this experience is beneficial in developing well-rounded training. From time to time, project managers should feel free to bring in consultants who had not been requested, assuming the topics they are covering are highly relevant to the clients. 3.4 Primary Source of Information Concerning Business Practices The primary source of information concerning business practices is word-of-mouth. Azeri￾company management structure tends to be dominated by strong vertical information flow. This is a result of the top-down nature of most companies' corporate culture. Managers are not accustomed to looking outside of the firm for guidance on how to operate or make their departments more efficient. Given this cultural precondition, it is very hard to overcome an upper or mid-level manager's tendency to simply take orders rather than look outside the firm for solutions and then implement change and take personal responsibility for a department's operations. 3.5 Other Key Channels of Business and Technical Information In the dairy processing sectors, equipment and other input suppliers tend to be an important source of technical information. In the hazelnut sector, buyers (importers) are the main conduits of technical and market information. For international exporters, the EU markets are a particularly important information channel when it comes to issues involving grades, quality, sanitation, equipment and packaging. As can be noted from Table 2, Azerbaijan is a competitive producer of alfalfa. However, to date, farmers, processors and traders have not capitalized on this price advantage to export alfalfa hay or finished dairy products. Information like this can be very useful in the development of new markets. Future USAID agribusiness activities can provide the technical knowledge to alert local businesspersons to these types of comparative advantages and help develop strategies to capitalize on the opportunities. 3.6 Client Satisfaction with the Quality of Technical Assistance Provided Interviews with association management and individual clients indicate that there is a high level of satisfaction with the quality of the project’s TA. However, clients have quickly learned how to play the “donor aid game” and they tend to request assistance in the form of fixed assets such as tractors, dairy processing equipment or production input items such as seeds and livestock drugs. Many client interviews began with a list of things (fixed assets and production inputs) they wanted but were not receiving from the project. After discussing the “want list” and the approach the project was taking – emphasizing training, clients always stated that they found the training useful and worthwhile. But in most cases, they would simply have preferred cash or assets. This attitude is not surprising when one considers the fact that in the centrally planned economy of the Soviet Union; all resources were handed out Alfalfa Prices in Selected Countries YR 2002 – Table 2 Country Summer USD/ MT Winter USD/ MT Mean Annual USD/MT Azerbaijan 41.00 80.00 60.50 Bulgaria 70.00 90.00 80.00 Romania 70.00 100.00 85.00 Macedonia 111.00 159.00 135.00 Kuwait 140.00(est.) Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 22 by the central government. Today, many clients have a difficult time differentiating between the hand of the central government and that of USAID. 3.7 Timeliness of Technical Assistance Delivery to Clients The PAPA project did not start organizing its client base and delivering TA until the second half of its first operating year. The primary reason for this seemingly slow start was the LOL grant porposal requirement that the project undertake a series of sectoral competitiveness studies and prepare other documents before start delivering TA (these reports included; a sectoral analysis report on dairy and hazelnuts, an international market position study, a farmer survey and a project work plan). Once this work was completed, the project began contacting potential association members, organizing them and implementing the TA program. The government’s reluctance to register associations also contributed (and continues to do so) to the speed at which the project was able to contribute towards SO 1.3. Also, in the first year of implementation, the project found itself behind in its projected spending and adjusted its burn-rate in years two and three to compensate for the low rate in year one. In hindsight, it would have been more efficient if grant proposal would have allowed the project to start organizing its associations and delivering TA sooner in year one. This could have been done simultaneously with developing the reports on competitiveness. Hopefully, the lesson learned regarding start-up timing and implementation on the PAPA project can be applied to RECP activity. 4.0 Business Association Development and Capacity Strengthening 4.1 Overall Progress and Approach to Association Development LOL’s approach in association development has focused on training in areas of: technical production, processing, management, association governance and lobbying. Association managers at first proposed that all TA be delivered through the associations (approved and coordinated by the association boards); however, this approach was dropped in favor of a system whereby approval and coordination of TA was shifted to project management. This system proved to be more efficient and less influenced by internal association politics than the former system. Overall, the PAPA project took a more intensive approach with the livestock and dairy processors association and a more extensive approach with the hazelnut sector. The intensive approach relied on heavy use of firm level TA, whereas the extensive approach focused on training trainers and village level workshops/demonstration. In the hazelnut-processing sector, TA was provided both through workshops and in-house (shop floor) training. Regardless of the sector, one of the most popular and sought-after TA activities was a trip abroad to meet other farmers, processors and explore markets. These trips, along with activities such as livestock parasite control clinics, have great appeal and are an effective tool to help bring people together to form an association. 4.2 Effectiveness in Developing Association Capacity to Affect Change in Agribusiness-Policy and the Regulatory Environment It is important to view association development in Azerbaijan in the context of its historical and political environment. First, it should be remembered that the association-developed activities by the LOL project are in their infancy; these associations between one to two years old at the time this report was prepared. Second, one needs to bear in mind the fact that Azerbaijan has no history of associations or democratic institutions and finally, consideration needs to be given to the fact that the GOA does not support the idea of association development and most likely only tolerates it because it is on the agenda of US and EU donor agencies. Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 23 Given this background, it can be said that the PAPA project has had a few small successes in the area of influencing government’s regulatory environment and resource allocation. This document has already discussed the efforts by the Azerbaijan Dairy Processors Association to lobby the GOA to drop the proposed licensing requirement for ice-cream street vendors. This was a small but significant victory for the association. Not only did it influence the regulatory environment, but it also created confidence within the association, that they can in fact influence government decisions. This in itself is a milestone. In Ismailly, the local livestock association lobbied the regional governor to supply them with office space and to write a letter to the Ministry of Justice supporting their application for official registration as an association. The governor acted positively on both issues. Again, the most important outcome of this effort may not be the free office space that the governor granted or the letter of support to the Ministry of Justice, but rather the confidence instilled in the association that it could go to government with a reasonable request and expect government to listen and act. These types of actions are the fundamental mechanisms of democracy. 4.3 Strengthening Associations' Advocacy Capacity The concept of strengthening the advocacy capacity of associations is appealing from the standpoint of building a health agribusiness sector and promoting democracy; however with the government's current position on associations, USAID should not expect this process to be fast or straightforward. The PAPA project provided training to associations in lobbying and government relations. This training was provided by a seasoned consultant (Elwin Guild) and targeted specifically at the associations' Executive Committees. The training was provided to all the associations the PAPA project worked with and was combined with training on association management. 4.4 Appropriateness of Approach in Organizing Associations to Serve as a Forum for Agri-Policy Dialogue With the Government USAID has made significant investments in association development in Azerbaijan and by its actions in this area, it has committed itself to working with groups of agribusinesses and producers. USAID should stand by the commitment it has made and continue actively working with the associations that it helped start. To fold up the tent and walk away from these associations at this time would send a strong signal to many rural communities that USAID is not a reliable partner. That said, USAID also needs to realize that the current government is at odds with them over the association issue and this has made moving forward difficult. Given this reality, it is recommended that future USAID activities such as RECP should not work to create new associations (until the government position on this matter changes), but rather should continue working only with existing associations. USAID and RECP can still strive to create organizations through which it can deliver technical and managerial TA and training in operations of democratic institutions, but these groups should not be created with the goal of applying for official association registration any time soon. USAID and its implementers may choose to call these groups something other than an association (i.e. alliances, societies, organizations, etc.) but in fact, these organizations can be the precursor to an association. When the government position on associations changes, these groups will be well positioned to apply for official recognition as associations. It’s a long-term approach but with out the long-term prospective USAID could end up expending much of its goodwill with the GOA and accomplish very little for its efforts. 4.5 Impacts and Benefits to Association Members When asked how association members benefited from their membership, one theme repeated in all interviews, was that before the associations were developed, individual firms (operating Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 24 in the same sector) would not speak with each other, and in most cases didn’t even know the other firms. After the organization of the associations, a business network was created, allowing for regular communication between members who felt this was very important, even though it is difficult to place a monetary value on it. In the area of market development, Hazelnut Association members realized the opening of the China market with sales valued at over 250,000 USD in YR 2002. Association members think that this market will most likely grow and become an important outlet for in-shell product. Dairy Processing Association members have benefited with numerous TA activities designed to improve product quality, food safety, sanitation, operating efficiencies and to develop new markets. LOL consultants have proven to be so practical, that in several cases (ISO certification and cheese making), individual processors have paid for these consultants to return to Azerbaijan to work with their company on a private basis. Producer groups have benefited from the project with practical TA training in the areas of livestock health, nutrition, farm record keeping, hazelnut production and post-harvest handling. The improved milk and nut quality that members realized resulted in an increase in sales revenues at the farmgate and allowed processors and exporters to operate more efficiently/ profitably. 4.6 Appropriateness of Training Materials and Marketing Information Supplied by the Implementer The type of data provided in PAPA’s market information activity is standard for USAID projects worldwide. The data provided is based on mean monthly wholesale and retail prices, geographically segregated and four-month price trends. Individual monthly market price data by itself has a fairly short shelf life (you can’t do statistics with a sample of one). To increase the usefulness of the data to the reader, additional time series data should be included so that longer trends can be determined. Often in monthly commodity news publications, time series data is given in the form of current month's prices, last month's prices and prices one year ago. One other option is to provide the reader with a 12 month high and low price. One complaint, which farmers mentioned about the market price data, was that it did not segregate price by product quality. This can be a difficult task in a country that has poorly defined and/or poorly enforced quality standards. If USAID chooses to continue supporting the publishing of market information, then it should try to upgrade the quality of the data with longer price time series and more volume and quality data where it is applicable. Overall, LOL has done a good job of selecting training materials relevant to its associations and firm level client needs. The Project has employed international staff members who are administratively and technically competent at selecting and developing appropriate training materials. Its has also selected and trained local staff capable of delivering training material in a clear and comprehensive manner. The type and quality of technical consultants brought to work on the project has also been good and has complemented the skills of the expatriate and local staff. 4.7 Effectiveness of Associations in Delivering Improvements in Agri-production6 Productivity, Product Quality, and Other Services Anecdotal evidence provided by PAPA’s dairy and hazelnut sector clients support the fact that product quality and incomes have increased as a direct result of project TA efforts. The specific areas on improvement include: 6 The following four sections: 4.7, 4.8, 4.9 and 4.10 address question that were included in sector 2.0 of the SOW. These questions have been moved to section 4.0 of the report to allow for a discussion of association issues under on major section heading. Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 25 • Hazelnut quality improvements through reduced moisture for product sold at the farmgate resulting in higher farmgate price, lower processing losses and better export quality. • Hazelnut production yields increased for client farmers by 625 kg / HA after TA was provided to farmers. The yield increase had an aggregate farmgate value of about 160,000 USD in YR 2002 • Farmers planted 1700 HA of new hazelnut orchards after PAPA started its TA activities in the sector. This reversed the trend of declining orchard area in Azerbaijan and will generate 1.57 million USD in marginal farmgate income annually and about 6 million USD in annual marginal export sales value. • Processed (pasteurized) milk sales doubled over the course of the LOL project • The Zagatela Livestock Association reported that death rate due to pathogens decreased by 10% as a result of project TA (a 15,500 USD savings in animal losses). 4.8 The Role Business Associations Play in Agribusiness Development One of most important functions that agribusiness associations can play in the Azerbaijan economy is to provide buyers and sellers a linking point at which to make contact. For example, a hazelnut processor may need 200 MT of nuts to fill an export order. Grower association members can be contacted as a group by using push-SMS technology. Farmers can then call the buyer to confirm the sale, which reduces the transaction cost dramatically and benefits both buyer and seller. Agribusiness associations in Azerbaijan are likely to provide an industry voice to government in the future. In the near term, this voice will be small but over time (Ministry of Justice policy allowing), the voice will grow louder and more articulate. Strong democratically managed agribusiness associations can become advocates for good government and help reduce corrupt government practices, such as preferential licensing deals. Agribusiness associations and other organized groups are also important conduits for training and TA delivered by donors, the GOA or by the association itself. Finally, agribusiness associations in Azerbaijan currently play an important role in developing a “community” within the sector, allowing firms and individuals to network. This facilitates business in both a commercial sense (i.e. deal making) and it facilitates trust and goodwill in the community, which improves the business climate in general. 4.9 The Main Areas on Which Business Associations Need to Focus The key future issues in USAID association development activities7 are association effectiveness and sustainability. Part of the sustainability issue relates to an association's economy of scale. Organization such as the three livestock associations developed under PAPA do not have a large enough member base individually to create an effective and sustainable association. Figure 9 examines the aggregate membership of the three livestock associations as of June 2003. The livestock associations average about 59 members each, with about 19 head of beef cattle and/or dairy cows per member. With a total combined membership of just 176 members and a total heard size of 3346 head, the organization cannot be described as robust in membership or economic terms. To overcome this constraint, USAID should consider folding these three organizations into one and have different regions (Isailly, Guba and Zagatala) operate as semi-autonomous chapters. This system would have several advantages: it would only require the legal registration of one association rather that three, and bylaws could be written in a way to allow for quick and easy addition of new chapters. This would help grow the organization geographically and in terms 7 Not including GOA policy toward association registration Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 26 of members. The potential size of a national livestock association would improve the probability of sustainability in economic terms and make it a more effective voice for livestock producers based on its membership numbers. Growth in Livestock Associations Membership & Herd Size 0 50 100 150 200 5 6 7 8 9 10 11 12 Project Quarter Association Members 0 500 1000 1500 2000 2500 3000 3500 4000 # of Head Members Head Figure 9, Source: PAPA Transparency is another area that needs to be considered. It is important that a more trusting relationship be developed between producers and processors. These groups are interdependent and the more efficiently they can work together, the more profitable they will become. To improve the working relationship between sellers (producers) and buyers (processors), their associations should have inter-locking boards of directors. This can done by having one seat on the producer’s board reserved for a processor and one seat on the processors board reserved for a producer. 4.10 Political and Economic Representation of Project Assisted Associations Different associations have different amounts of political and economic representation. The most economically represented groups are the processing associations. LOL estimates that 70% of all processed milk sold in Azerbaijan is sold by the association members. In hazelnuts, the market share numbers are also significant. It is estimated that the Hazelnut Processors Association members handle over 50% of all hazelnut export sales. Number of Communities in Which PAPA Worked, Cumulative 113 119 121 121 108 110 112 114 116 118 120 122 124 8 9 10 11 Project Quarter Communities Communities Assisted Linear (Communities Assisted) Figure 10, Source: PAPA Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 27 Figure 10 considers the geographic representation of PAPA activities. At the end of YR 2002, the project was working in 121 communities in the Guba, Zagatela, Oguz, Gakh, Gabala, Balaken, Sheki and Baku regions of Azerbaijan. On the producer side, the PAPA assisted livestock association only represents a fraction (about 1%) of the nation’s 350,000 cattle and dairy herd. With it 1,150 members, the Azerbaijan Hazelnut Farmers Association membership produces about 10% (4,800 MT) of the country’s 45,000 MT hazelnut crop with a farmgate value of over 2.25 million USD. The 1,150 members own 2,342 ha of orchards and lease an additional 4,122 ha, for a total farmed orchard area of 6,464 ha. 5.0 Sustainability Issues Relating to Recommended Practices 5.1 Sustainability of Project Outcomes8 It is highly likely that the Azerbaijan Hazelnut Association and the Azerbaijan Dairy Processors Association will continue to function as a cohesive group well into the future. These organizations are made up of firms that can afford to support the ongoing costs of an association and they feel that they receive an economic benefit by being a part of the organization. Since none of the associations has been formally registered with the government, their future structure could evolve into something more akin to a formalized business-networking club. No matter what its future form evolves into, the goals of the organizations will remain the same. These goals include; • Providing a form for sector participants to meet and discus common issues • As a group, trying to affect change in the governments regulatory environment • As a group, working together to improve the quality and availably of raw product • Networking within the group to develop synergistic business relationships • Attracting support to the association for training/TA via grants and other mechanisms The LOL client base in Azerbaijan’s dairy processing and hazelnut processing sectors include some of the country’s best-run agribusinesses. Agriculture is a risky business, due in part to the uncontrollable factors such as weather, and volatile markets, as well as the large amount of cash needed in some sub-sectors to purchase the seasonal crop. The PAPA project clients have proved themselves capable of managing these risks and this fact alone attests to the likelihood of the sustainability of these companies. USAID can make a positive contribution to the sector by providing TA, which helps private enterprises grow financially, geographically and demographically (in market terms), as well as providing a greater variety of safe produce to the market and does this with as little risk exposure as possible to the clients. 5.2 Adopted Practices Most Likely to be Sustainable The adopted practices, which are most likely to be sustainable, are the practices that are most likely to contribute to a client’s bottom line. For example, as a result of PAPA TA, hazelnut farmers have modified their orchard cultural practices (pruning, weed control, cultivation, etc.) to maximize yields and income. They have also changed their post-harvest handling of the nuts to minimize moisture content and reduce losses, again in an effort to maximize income. Dairy processors have adopted new methods of manufacturing, developed new products and packaging, as well as entering new markets in an attempt to increase profits. The Ismailli Livestock Association has begun a commercial business of selling cotton seed cake as a supplemental protein source for milk producers. This has resulted in increased milk production and (correspondingly) increased sales revenues for farmers. 8 This section also addresses the question posed in section 5 of the SOW “what is the long-term likelihood of client enterprises and associations to be sustainable. The two sections were combined due to the similar nature of the questions. Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 28 5.3 Adopted Practices Least Likely to be Sustainable There are several activities the PAPA project undertook that appear not to be sustainable without ongoing donor support. For example, working with the livestock associations, PAPA implemented an animal parasite control activity. This activity was very popular with association members and helped to bring considerable community awareness to the project and associations (good PR). Given the high cost of the drugs used in the activity, farmers could not afford this input on the own. Considerable debate took place within PAPA regarding the wisdom of moving forward with this activity, but in the end, it was given a green light with the condition that farmers contribute a token amount to the cost of the activity. Project management knew the risks involved with raised expectations, but chose to implement the activity largely for reason of public relations. The market price information service is another activity that will be difficult, but not impossible, to make sustainable. Price information services are a regular activity in USAID funded agribusiness projects. They usually result in improved market transparency and train staff in market research (data collection) and analysis. The trick to making this activity sustainable is to link it to a private sector or government organization that is willing to pay for the information. In the US, the USDA Market News Service funds, collects and organizes the data. This data is then provided to print, radio and TV news services that use it as “content”. In Azerbaijan, future USAID activities should try to link the agri-market price reporting service with an existing private sector or startup news service. There is media market value in price, quality and volume data, but the question that remains to be answered is if this data can be collected, organized and distributed at a profitably. In a survey of PAPA’s market information publication readers, 72% of the respondents said they would be willing to pay for the market information contain in the newsletter. 5.4 Options to Increase Sustainability of Adopted Practices To maximize the adoption of sustainable practices, USAID should focus its future TA on firm level assistance, as well as existing associations and other groups. In all cases, TA needs to be practical and driven by its contribution to the firm’s or individuals bottom-line. Markets drive production, and therefore markets should be the key part of any new activity aimed at increasing sector growth, profits, sustainability and TA adoption rates. Efforts should be put into building consumer awareness of quality agricultural products from Azerbaijan. At the same time, food processors and sellers need to improve their product quality and safety (through a Seal of Quality or similar program). Farmers are an important part of the agribusiness’s profit equation; however, given their small size and their broad geographic distribution, farmers are often difficult to reach. To help strengthen the supplier-buyer linkage, USAID should funnel its TA efforts to improve farm productivity through buyers (processors, wholesalers, and exporters). The buyer knows where Open vs. Secret Ballot Elections When LOL started its association development activities in Azerbaijan it let members choose between a system of open or closed (secret) ballot elections in selecting their executive officers. What LOL found was that powerful individuals usually ran for the association’s executive committee seats and they wanted open (raise your hand) elections. People were sometimes intimidated into voting for these powerful individuals for fear of reprisals at a future date. In cases where open elections were held, LOL then asked for a revote, this time under a secret ballet. The result was that the same people stayed in office but the voting margins changed by about 20%. The lesson learned by LOL was that it’s better to require a secret ballet from the start. Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 29 the farmers are located, who the good producers are, etc. The farmers will listen to the buyer, as they are “the market” and if a processor requests a specific type of handling after harvest, the farmer is likely to comply. 5.5 Current Number of Operating Client Enterprises and Associations Table 3 apprises the number of members and persons employed in each of PAPA association and the percentage each individual association represents relative to PAPA total association client base. As can be noted, the Hazelnut Growers Association makes up the largest group with 1,150 members and 84%, whereas the Hazelnut Processors Association is the smallest group with just 13 members and 1% of total client members. Table 3 also splits the association’s membership in terms of sectors; 15% of total members are in the livestock and dairy sector and 85% of members are in the hazelnut sector. Not all clients are association members and PAPA provided TA to four dairy processing companies that were not in the Dairy Processors Association, and 2 hazelnut processors that were not in the Hazelnut Processors Association. At the farm level, PAPA provided TA to 40 farmers in south and central Azerbaijan who were not association members, in cooperation with the ACDI/VOCA and IRC dairy projects operating in those regions. 5.6 Client Leveraging USAID Support Through Other Donors The PAPA project has provided its associations with training in grant writing and this has resulted in several of the groups obtaining grants from IREX and the US Embassy economic development fund. The amount of the grant was small (<10,000 USD each) but it was significant by the fact that the associations conceptualized and wrote the grant requests by themselves. The PAPA activity has also worked closely with British Petroleum (BP). As part of their community development activities, BP funded 50% of the cost to develop four rural cheese-manufacturing facilities in northern Azerbaijan. The LOL project supplied 25% of the facilities cost and 25% was supplied by the local enterprise’s owner/operator. These were Client Association Membership By Agribusiness Sector, Table-3 Association No. of Members Members Percent No. of Employees Employees Percent Zagatala Livestock Association 65 5% 240 4% Guba Livestock Association 52 4% 297 6% Ismailly Livestock Association 52 4% 216 4% Azerbaijan Dairy Processors Assoc. 26 2% 513 9% Subtotal Livestock Products Sector 195 15% 1,266 23% AZ Hazelnut Growers Association 1,150 84% 466 8% AZ Hazelnut Processors Association 14 1% 3,828 69% Subtotal Hazelnut Sector 1,163 85% 4,294 77% Grand Total, Livestock & Nut Sectors 1,359 100% 5,560 100% Linking with Other USAID Project As part of its market development and education activity the PAPA project took clients to several international trade shows, visited to food processing firms and equipment suppliers. PAPA worked with the World Learning Project and Land O’Lakes agribusiness projects in Macedonia and Bulgaria to organize these trips. Clients found the trips to be educational in terms of what the international market demands. Additional, some dairy processors purchased milk￾processing equipment while on a PAPA trip to Turkey, saving 15,000 USD over locally supplied hardware. Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 30 important projects in that the businesses that were created provide a cash market to village milk producers that heretofore had a very difficult and limited market access. Each of these new businesses can process up to 300 kg of milk per shift, resulting in potential sales of over 30,000 USD annually. 5.7 Association Membership Dues and Fees Figure 11 examines how many associations have been created under the PAPA activity and how many of the associations are paying dues. Associations Created and Association Paying Dues Over Time 0 1 2 3 4 5 6 7 Baseline 7 8 9 10 11 12 Project Quarter Number of Associations Associations Created Dues Collected Figure 11, Source: PAPA Dues Paid by Associations 0 500 1,000 1,500 2,000 2,500 3,000 3,500 4,000 Dairy Processors Assoc. Guba Livestock Assoc. Hazelnut Processors Assoc. Hazelnut Growers Assoc. Ismailly Livestock Assoc. Zagatala Livestock Assoc. USD/YR June 2003 YR 2002 Figure 12, Source: PAPA Figure 12 illustrates the amount of dues collected from each association in YR 2002 and January through June 2003. As can be noted, the Azerbajian Dairy Processors Association has collected the greatest amount of dues, whereas the Zagatala Livestock Association has failed to collect dues in YR 2003. 6.0 Monitoring and Evaluation 6.1 Achieving Project Targets PAPA project management has a solid understanding of the activity's goals and objectives relative to USAID SO’s and IR’s. They fully understand the objectives of creating functional Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 31 and sustainable agribusiness associations, as well as the need to develop agribusiness SME’s which are responsive to the marketplace and are financially viable. Some local project staff members appear to have a slightly different view of the project deliverables and tend to focus on process indicators, such as the number of persons trained by the project or the number of grants received by clients as a result of PAPA training in association development. To more effectively achieve project targets (SO’s and IR’s), the local staff, the senior staff and client association members should have shared vision of the project objectives. All staff and association members need to have a common vision of the project’s target outcomes rather than simply measuring performance by process indicators. 6.2 The Quality and Reliability of Performance Indicators When the PAPA project started in YR 2000, USAID and Land O’Lakes agreed to a set of project specific performance indicators. These performance indicators included a number of firm level financial performance measurables, such as gross sales revenue. A year later, USAID redesigned the M&E system, opting for indicators that emphasized employment and training. This shift allowed USAID to measure data that is relevant to SO 1.3 but does not allow for an analysis of the fundamental financial health of client firms. In the future, as USAID’s agribusiness portfolio moves toward emphasizing competitiveness, it will need to start measuring changes in gross revenue, revenue per employee and other objectively variable financial and productivity indicators. Measuring process indicators (such as number of people trained or associations developed) still need to be a part of the future M&E system, but they are most useful when they relate to actual cost and sales data for a fixed set of firms. Measuring growth in asset value is a key part in determining the project's contribution to SO 1.3. Unfortunately, the methodology developed by the USAID M&E design team did not effectively capture the change in assets of the individual businesses or groups of businesses, but rather, it measured the aggregate value of project client assets. Thus, as the number of clients increased, so did the reported aggregate asset value. This method falls short of supplying USAID with the type of information it needs to determine if its projects are effectively addressing SO 1.3. 6.3 The Use of M&E Data and Anecdotal Information by Project Management The PAPA project management used M&E, as well as anecdotal information, in their daily decision￾making processes. Much of this information was collected “first-hand” through fieldwork, while other data was collected via quarterly surveys of clients and association members. Additionally, the project held Monday morning management meetings where staff members updated each other on individual project-department achievements and constraints as well as discussed crosscutting issues that may affect several departments within PAPA. Typically, if a potential client showed an interest in working with the project, this information usually came out in Tracking Changes in Marginal Gross Income at the Farm and Plant Gates Land O’Lakes calculates that PAPA’s TA to its three livestock associations resulted in a 20% increase in per cow milk yield and significantly contributed to the increase in the association’s milk cow population. When PAPA first started working with these groups their total milking herd size was 916 head, by EOP this grow to 1673 head. This increase in milking head plus the increase in milk mean yield per cow from about 4.3 lt./day to 5.18 lt./day resulted in an increase in aggregate marginal farmgate income of 274,000 USD annually. This is before calculating change marginal income from meat sales (so it’s a conservative estimate). LOL estimates it expended 525,000 USD over three years in providing TA to the 3 livestock associations involved. Bases on this level of investment and the increase in marginal income, USAID and LOL realized a 52% Return On Investment (ROI) from this activity. Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 32 the weekly management meetings via shared anecdotal information. When this occurer, the project typically focused resources on the potential client. 6.4 Options to Better Utilize M&E Data and Anecdotal Information by Management USAID does not require its contractors or allow implementers to account for spending by project activity (job costing), such as funds expended on firm level TA or workshop training. Rather projects are required to aggregate their accounting data into line items such as salaries, travel, office/overhead, etc. Since the implementers are not asked to segregate accounting data by project activity, it is impossible to develop reliable unit-cost calculations or comparable cost effectiveness calculations within or across projects. In the future, it would be very useful from an evaluation standpoint if USAID would ask its implementers to collect accounting data per activity (i.e. total project funds expended on workshop training or client shop floor TA). This would allow USAID to develop the statistical base from which it could truly evaluate the cost-effectiveness of its implementers. 6.5 Effectiveness in Tracking The Progress of Client Entrepreneurs9 USAID and LOL had agreed to a set of project target deliverables under the original PAPA project workplan in mid-YR 2000. About a year later, USAID replaced the project targets with a list of new indicators, which were to be tracked. When the former measurables and targets were discarded and no new targets were created, USAID only asked that the new measurables be tracked. LOL has done a good job in tracking the USAID measures. Starting from project quarter seven, LOL regularly reported on indicators, including but not limited to: • Assets of assisted enterprises, reported annually • Total number of employees at client firms, quarterly • Number of new products and services, quarterly • Number of new markets entered by client firms, quarterly • Number of people trained and the adoption rate of new skills learned, quarterly • Number of firms complying with new product standards, quarterly • USD sales value of new products which comply to new standards, quarterly • Number of opportunities for market information, quarterly • Number of new associations created, quarterly • The amount of dues collected by associations, annually • Number of communities assisted by the project, quarterly In addition to tracking the above, LOL prepared a detailed process report each quarter. These reports were well written and included narratives on work, which took place within each association and many highlighted the progress of client firms as result of project TA. The only recommendation in this area is that USAID may want future implementers to reduce the detail in the quarterly reports, limiting them to three to five pages so more project resources can be focused on clients and less on managing the bureaucracy. If detailed information is required to document project activities, this may be better done in a single “Project Annual Report”. 9 Section 6.4 Effectiveness in Tracking The Progress of Client Entrepreneurs, was addresses in section 2.0 of the SOW but has been move to Section 6.0 of this report in an effort to group questions regarding tracking as well as M&E under one major section title. Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 33 Annex Annex A: Hazelnut Market Information Network Development Model Annex B: The Use of Public Awareness Campaigns in Market Development Annex C: Association Foundation Meeting Dates Annex D: Azerbaijan Country Profile Annex E Evaluation SOW Annex F USAID Azerbaijan Strategic Objectives Annex G PAPA Activity Final Evaluation PowerPoint Presentation Master Slides Annex H Interviewee Contact List Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 34 Annex A Hazelnut Market Information Network Development Model Using Cell Phone Push – Pull Technology (+/-) 1,150 Hazelnut Growers Association Members (Sellers) Azerbaijan Hazelnut Association (AHA) (+/-) 14 Commercial Hazelnut Processors and Exporters (Buyers) #4 Sellers (nut farmers) phones in sells commitment to buyers and arranges for delivery Transport Company: #3 Individual firm price, volume & quality needs pushed to buyers/subscribers via SMS from Azercell computer #1 Nut buyers phone in offer prices, volumes and quality to Assoc. # 5 Buyer books transport order and confirms deliver time to processing plant Azercell Push/ Pull Technology #2 AHA organizes data and downloads price, volume and quality info to Azercell computer #7 Seller ships nuts to buyer #6 Transport Co. confirms pickup time and place Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 35 Annex B The Use of Public Awareness Campaigns in Market Development In most developing countries there is general agreement between the business, government and donors that the most effective strategy to increase economic activity within the agricultural sector is to stimulate the demand for agricultural goods and services. This strategy is usually referred to as “market driven agri-sector development”. USAID has a number of tools that it employs to help its clients (farmers, food processors and traders) increase market demand for their products and thereby grow the sector through investments in people, equipment and production inputs. Some of the more common tools that USAID employs include: • Development of intellectual capital through the training of agribusiness client owner-operators, technical staff and management resulting in efficiency gains, market expansion and increased production • The development of safer and higher quality foods through specific training and firm level TA in HACCP, ISO, and food Good Manufacturing Practices (GMP) • The development of more transparent and efficient markets through the creation of a market information system • The infusion of capital into the sector through short and medium term credit facilities and/or credit guarantee programs • Grant programs design to encourage capital investment of partner client firms • Improved policy environment through agribusiness advocacy groups (associations) and in direct consultations and education activities targeting ministry staff and elected officials • Increased understanding of international market opportunities and export requirements through international learning (international travel to food shows, trade fairs, etc) Market promotion campaigns are also an important tool available from USAID. To date, few projects have employed this tool to help drive market demand. Experience has shown that it can be an extremely effective instrument in helping to drive markets, encourage industry cooperation (association development), increase the demand for training and improve the adoption rate of skills delivered through workshops and other firm level TA activities. Many USAID association development projects find it difficult to provide members with tangible benefits, particularly in the first three years of an association’s development. Often projects provide benefits to member such as grants for new equipment, market development grants, trip to international trade shows, which may be interesting, but seldom result in a significant amount of new sales across the sector. Clients usually appreciate TA that improves their product quality but businesses often find that consumer buying habits are focused on price and not quality; and this sometimes results in a slow market response to new/ improved products. In Macedonia, USAID funded a consumer awareness/ marketing campaign which brought together the elements of workshops and firm level TA to improve meat and dairy products quality, a mass media campaign to educate consumers and influence buying habits and a development activity that organized a self-governing industry association that set food quality standards, tests product quality and promotes Macedonian food products under a common identity (seal of quality). This strategy requires significant financial resources, primarily for print and TV ads, but it is a highly efficient way of creating market demand, improving food quality and safety while building a sustainable self-governing industry organization. Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 36 Annex C Association Foundation Meeting Dates Dates of Foundation Meetings Zagatala Association of Livestock Producers and Processors - July 6, 2001 Guba Association of Livestock Producers and Processors- September 21, 2001 Ismailly Livestock Association- February 16, 2002 Azerbaijan Association of Hazelnut Growers- October 11, 2001 Azerbaijan Association of Dairy Processors -November 16, 2001 Azerbaijan Association of Hazelnut Processors - July 20, 2001 Annex D Azerbaijan Country Profile Government • Capital: Baku (Baki) • President: Heydar Aliyev • Prime Minister: Artur Rasizade Geography/Population/Economy • Area: 86,600 sq km, slightly smaller than Maine • Language: Azerbaijani (Azeri) 89%, Russian 3%, Armenian 2%, other 6% (1995 est.) • Currency: Azerbaijani manat (AZM) • Population: 7,771,092 (July 2001 est.) • Population Growth Rate: 0.32% (2001 est.) • Life Expectancy: male - 58.65 years, female - 67.49 years (2001 est.) • Infant Mortality: 83.08 deaths/1,000 live births (2001 est.) • GDP: purchasing power parity - $23.5 billion (2000 est.) • Real Annual GDP Growth: 11.4% (2000 est.) • Annual Inflation: 1.8% (2000 est.) Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 37 Annex E Evaluation SOW STATEMENT OF WORK Evaluation of PAPA and AMDA Programs in USAID/Caucasus-Azerbaijan This evaluation will examine two activities being implemented under USAID/Caucasus￾Azerbaijan’s Strategic Objective (SO) 1.3, Accelerated Development of Small and Medium Enterprise. These activities are: Participatory Agriculture Project in Azerbaijan (PAPA) and the Agro-Input Market Development in Azerbaijan (AMDA). I. PURPOSE OF THE EVALUATION: The principal focus of the evaluation shall be the performance of PAPA and AMDA activities in accomplishing the terms and objectives of their respective Cooperative Agreements and their contributions in achieving the Mission’s Strategic Objective 1.3. Evaluation of the AMDA program will help assess how the program is being implemented in order to identify areas of improvements and possible mid-course corrections, whereas evaluation of the PAPA program will help assess program impact on the well being of the targeted population. The evaluators shall determine the status of the current activities, their successes and weaknesses, and provide recommendations for USAID regarding possible improvements and adjustments that might enhance the future performance of AMDA under the current Cooperative Agreement. The evaluators should also identify any unforeseen constraints and obstacles that may have affected PAPA and AMDA performances. The evaluation should assist USAID/Caucasus-Azerbaijan to formulate ideas regarding future support to agriculture and agribusiness development in Azerbaijan, including, but not limited to Rural Enterprise Competitiveness, Agriculture Marketing, Rural Community Business Development programs. In this regard, the evaluators should provide the Mission with a better understanding of the current overall environment for the development of agriculture sector in Azerbaijan, the sustainability of current efforts, and, may provide recommendations for future interventions in the agricultural sector. Brief description of AMDA and PAPA programs is attached in Annex 1. II. QUESTIONS AND ISSUES TO BE ADDRESSED: The evaluation team should address the following questions and issues: 1. General: a) What interventions are most critical and/or have been effective in achieving program objectives and intermediate results? What improvements can be made in program implementation in order to enhance the results? Are there opportunities for integrating program components that could result in greater program impact? b) Are there any unexpected but important benefits or impacts that should be documented? Are there any negative impacts or unintended consequences of the program that need to be addressed, and how? c) Are planned activities appropriate for improving agribusiness development in the targeted areas? Do the assumptions and program design (activities) match the sector conditions and policy environment? d) How cost-effective have the interventions in the two activities PAPA and AMDA been? Are the targets/results/impacts accomplished to date commensurate with resources invested? Are there follow-on activities that USAID should consider to maintain the momentum? 2. Agriculture and Agribusiness: Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 38 a) What are the major opportunities for improving the sector? Improving marketing, increasing sales volume, improving the quality of produce, building business skills etc.? Does the program take adequate advantage of these opportunities? b) Is there a reasonable market access for products and/or services produced by the program-assisted rural entrepreneurs? c) What was the nature of assistance that was provided to rural entrepreneurs and trade/business associations supported by the two activities and how effective such assistance has been? Are there specific constraints to reaching some business association members? How could the activity (especially AMDA) be broadened to more fully incorporate these business association members? d) How effective the business associations have been in supporting their members in the following areas: improving agricultural production, improving productivity and product quality, and providing other services (e.g. access to finance etc.)? What role the business associations played in agribusiness development? What are the main areas on which business associations need to focus? How representative are these business associations, both politically and economically? e) Assess the overall impact of these program-assisted business associations/enterprises on agribusiness development in Azerbaijan? f) Have business development activities and training tailored to individual rural entrepreneurs needs contributed directly to improved productivity and business skills, and/or improved access to finance. Are there ways to enhance the impact of business development and training activities in improving productivity and access to finance? g) How effective have AMDA and PAPA been in tracking the progress of rural entrepreneurs, in facilitating their access to finance from various sources, and assisting the entrepreneurs in applying newly acquired enterprise management, marketing and business skills? h) Are the field demonstration projects under the AMDA program based on the conditions and problems facing farmers? 3. Adoption of New Practices: a) Are beneficiaries adopting recommended practices (technical, enterprise management and business practices)? Which practices have beneficiaries been more inclined to adopt, and why? How can the program be modified to address these constraints to adoption? b) What is their primary source of information concerning business practices? What are other key channels of information? c) Are the beneficiaries satisfied with the quality of technical assistance is provided? Is technical assistance to beneficiaries provided in a timely manner? 4. Business Association Development and Capacity Strengthening: a) Assess the overall progress and approach to business association development followed by PAPA and AMDA. b) Is the program effectively developing the capacity of business associations to support their member needs and advocate for a more agribusiness-friendly policy and regulatory environment on a regional and/or national basis? If not, how the business associations advocacy capacity can be strengthened? Is the current approach to organizing business associations satisfactory to serve as a forum for policy dialogue on agribusiness issues with the government? c) Assess the impacts/benefits accrued to associate members. d) Are the training materials and marketing information supplied by the implementer appropriate for the beneficiaries? Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 39 5. Sustainability issues relating to recommended practices and business associations: a) Are the outcomes related to adoption of better practices sustainable, i.e. are the participants likely to continue after program ends? Which outcomes are likely/unlikely to be sustainable, and why? What can be done to increase sustainability? b) Are these newly created and/or improved enterprises and business associations likely to remain in operation after the programs are terminated? If not, why and what can be done to enhance their sustainability? c) How many program-assisted enterprises and associations are still operational? How could assistance be adjusted to ensure sustainability of the program-assisted entities? d) Are the business associations/entrepreneurs seeking grants/credits and/or technical assistance from other donor programs to leverage USAID-provided support? Have they mobilized any resources internally (i.e., membership fees)? If yes, provide anecdotal evidence. 6. Monitoring and Evaluation (M&E): a) Has the program achieved its targets to date? If not, assess reasons for shortfalls. Do the performance indicators provide useful and reliable data on program progress and impacts? b) Are M&E data and anecdotal information used for management purposes? Does the technical staff use M&E data and anecdotal information to conduct their work and assess progress? Can M&E data and anecdotal information be better used for program management? III. METHODOLOGY: Rapid Appraisal techniques (e.g. key informant interviews, site observations, mini-surveys) are recommended for conducting this evaluation. However, the team should develop an appropriate methodology to address the evaluation statement of work. IV. DELIVERABLES: The team will submit a detailed work plan along with the schedule of field work specifying how the information will be collected, organized, and analyzed to meet the information needs specified in the scope of work not later than three days after the team arrives in the country. Upon the completion of the evaluation, the team will brief the Mission staff, and will submit a draft evaluation report two working days before the team departs from Baku. The Mission will provide comments and suggestions within one week after receiving the draft. The Final Report (5 bound copies and an electronic version in PDF format) will be provided to the Mission within two weeks after completion of evaluation in Azerbaijan. In addition, a copy of the final report should be submitted to: United States Agency for International Development PPC/CDIE/DI Ronald Reagan Building Washington, DC 20523 To ensure that the evaluation findings and recommendations are presented in a way that is useful for the Mission personnel and program implementers, the following outline is recommended: - Executive summary not to exceed two pages in length composed of findings, a brief methodology statement, conclusions and key recommendations for each program evaluated; - Introduction and background section for the overall evaluation. - For each program: Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 40 - Brief description of interventions; - Implementation progress, achievement of results and program impacts; - Discussion of SOW questions by applying the following format (findings, conclusions and recommendations). Each section should not exceed 20 pages. The report should also include a response regarding the cause(s), if any, of deficiencies or weaknesses. It will also include findings and recommendations on priorities for further assistance/activities in the area(s), improvements and possible synergies that can be achieved in USAID’s programs supporting agriculture and agribusiness. A discussion of lessons learned and best practices that should be captured for consideration in the implementation of future activities. V. TIME FRAME: The evaluation is scheduled to commence in early June 2003. Duration of the evaluation is estimated to be approximately four weeks – 24 working days (2 days in Washington D.C. to interview program managers and document review, 1 day in￾country for in-briefing SO team on work plan, methodology, and clarification of SOW questions; 1 day in-country for out-briefing SO team; 15 days in country for collecting and analyzing data; 3 days for drafting report, and 2 day for revising the final report). A six-day work week is authorized for the team. VI. TEAM MEMBER QUALIFICATIONS: Team Leader 1. At least an M.S. degree in agriculture or agricultural economics. 2. Extensive experience in analyzing agricultural development activities, pertaining to dairy, livestock and inputs. Be familiar with recent agricultural development in the Caucasus and/or NIS regions. 3. Prior experience in leading evaluation and/or design teams. 4. Exceptional interpersonal, leadership and management skills. 5. Excellent writing skills. Agribusiness Specialist 1. Advanced degree in agribusiness and/or agricultural marketing. 2. Experience in implementing, assessing and evaluating agri-business programs preferably in NIS regions. 3. Experience with a variety of data collection techniques. 4. Organizational, listening and analytical skills. 5. Excellent writing skills. The Team Leader’s principal responsibilities: 1. Maintain contact with the technical office at USAID/Caucasus-Azerbaijan. 2. Brief the Mission on findings and recommendations. 3. Prepare and submit a final report to USAID/Caucasus-Azerbaijan within 10 working days following the receipt of comments from the Mission on the draft report. VII. LOGISTICAL ASSISTANCE: The Mission will make available reports and other background materials appropriate to the evaluation. The team will be required to provide all other logistical arrangements such as international travel, accommodations in Baku, interpreting, secretarial and other services. The team will be responsible for arranging local transportation (hiring vehicle and driver). The team will travel within Baku and to selected regions in Azerbaijan to review Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 41 programs, activities and interview beneficiaries. Detailed schedules for site visits and interviews should be developed by team members in consultation with USAID and it’s implementing partners. Logistical issues to be resolved in advance including the number of sites to be visited, host partner institutions to be interviewed, timing of visits to regional offices, etc. The USAID Mission and the implementing partners will assist the Evaluation Team in scheduling meetings and site visits, including the names and contact information for key individuals to be interviewed in Baku, other cities and regions of Azerbaijan. The team should conduct interviews at least with the following people: - USAID/Caucasus-Azerbaijan senior staff members - LOL and IFDC staff members in Baku: LOL-David Blood, Jeton Starova, Rauf Akhundoc; IFDC-Manfred Smotzok, Ylli Bicoku, Farid Firidunov) - Staff members of other USAID-assisted programs, such as: ACDI/VOCA-David Sulaberidze, Ismail Rafi, Matthew Weber; IRC-Pamela Husein, Jeanne Izard; CHF-Beverly Hoover. - Representatives of major donor organizations: World Bank, EBRD, TACIS and GTZ; - Beneficiaries and partners of AMDA and PAPA programs associations and enterprises established and/or supported by these programs. The name will be provided by implementing partners. State-side contacts: Prior to departure from the U.S. the team leader should schedule interview with Mr. Jim Herne, Central Asia and Caucasus Coordinator of Land O’Lakes (telephone: 763- 785-0282, ext.: 4298, e-mail: jhern@landolakes.com ) and Mr. Scott Wallace, IFDC Agro-Business Specialist in Market Development Division (telephone: 256- 381-6600, e-mail: swallac@ifdc.org) to learn more about AMDA and PAPA activities in Azerbaijan. USAID participation in field work: USAID/Caucasus-Azerbaijan may choose to participate in the field work, site visits as an observer. VIII. REVIEW OF DOCUMENTS: Prior to arrival the assessment team should review the background materials listed below and brief the Mission in conclusions drawn from these materials: - Cooperative Agreements between USAID and Land O’Lakes, and USAID and IFDC - AMDA work plan (Year 2002) - PAPA work plan (Year 2000, 2001 and 2002) - AMDA quarterly reports - PAPA quarterly reports IX. DISSEMINATION CONSIDERATIONS: All documents prepared by the evaluation team and delivered to USAID/Caucasus-Azerbaijan, including the final report, are viewed as the sole property of the U.S. Government, and only with the concurrence of USAID/Caucasus-Azerbaijan they should be disseminated to third parties. Annex 1: Participatory Agricultural Project in Azerbaijan (PAPA) The Participatory Agricultural Project in Azerbaijan (PAPA) is a three-year project funded by a $2.4 million cooperative agreement with USAID in March 2000 and implemented by Land O’Lakes. USAID has recently awarded PAPA with a no-cost extension to run through June 2003. The objective of the project is to develop Azerbaijan’s dairy, livestock and hazelnut industries, with an emphasis on the target regions of Baku, Guba, Ismaily and Zagatala. Strategic technical assistance and training is helping the beneficiaries of the program to increase profitability through the development of agri-business associations, the expansion and improvement of production and value-added processing, the forging of marketing links, and increased availability and access to market information. These goals directly contribute Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 42 to USAID Strategic Objective 1.3 (Accelerated Growth of Private Small and Medium Enterprises in Targeted Areas) and it’s accompanying Intermediate Results. A fundamental aspect of PAPA is its emphasis on agribusiness associations as engines for industry development. Land O’Lakes is utilizing associations as both a conduit for its assistance activities and as a means for supporting local capacity to support the agricultural sector. Therefore, PAPA’s assistance is generally provided only to association members in order to encourage active participation. In addition, association members are required to contribute a share of overhead and variable costs for their activities, as well as to assist in their implementation. The overhead cost required of each association is expected to increase from 50% to 75% by the end of project. Since PAPA has helped to establish independent, democratic organizations, it does not dictate the activities and services that they should pursue, but rather, the project responds to their requests for assistance. PAPA is currently in the third year of its activity, where association development is playing an important role. Assistance to the dairy industry is focused on strengthening the Azerbaijan Association of Dairy Processors and regional livestock farmers’ association in Guba, Ismailly and Zagatala as well as on creating a new livestock farmers’ association in Masalli or Lenkoran. Hazelnut industry assistance will be targeted toward strengthening the National Association of Hazelnut Growers and National Association of Hazelnut Processors. Agro-Input Market Development in Azerbaijan (AMDA) USAID awarded IFDC a Cooperative Agreement in January 2002, to implement the Agro￾Input Market Development in Azerbaijan (AMDA) project. The three-year $3.1 million project is designed to advance USAID Strategic Objective 1.3-Accelerated Growth and Development of Private Small and Medium Enterprises in the targeted areas. AMDA’s geographical coverage includes the following four regions of Azerbaijan: Masalli, Guba, Ganja and Sheki. The AMDA directly contributes to the result indicators of SO 1.3 by increasing trade volumes of fertilizer, seed, CPC and animal feed; supporting agro-input dealer enterprises and assisting them to establish an effective and sustainable trade association, introducing new products, and facilitating increased access to credit and investments for clients. AMDA project concentrates on the following four major components: business development and training; access to credit and finance, association building and development; technical training, transfer and extension services. Business development and training for targeted dealer clients in both, group and individual settings will be critical to facilitation of increased access to credit, forming effective trade associations and providing extension via agro-input enterprises. Emphasis is placed on practical business planning, market analysis as well as training in such areas as accounting, marketing, demand-oriented customer service and international business standards. One of AMDA’s main objectives is to establish a trade association that grows in size and influence and serves as a vector for technology transfer and policy change. AMDA is planning to form four regional sub-groups and work through them to demonstrate the value of a trade association through holding regular meetings and networking, business training, study missions and technical assistance in international procurement, group purchases of inputs, distribution of marketing information and collective advocacy efforts for policy reform. AMDA’s transfer of technical information and skills adds value to business of distribution and marketing of the agro-input products. AMDA conducts training of client dealers in technology and information transfer methodologies aimed at improving of services and increase in usage of economic and environmentally safe agricultural inputs. The technical training component includes field demonstrations under project supervision and is supported by classroom/workshop activities in private sector extension as a means of adding value to marketing inputs. The in-country training is supplemented by study missions to observe private sector extension programs and agro-input marketing in other countries. Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 43 ANNEX: F USAID Azerbaijan Strategic Objectives SO 1.3 Accelerated Growth and Development of Private Small and Medium Enterprises in Targeted Areas Timeframe: 1 - 3 Indicators: Growth of assets of assisted enterprises by target area; Increase in employment of assisted enterprises by target area desegregated by gender. IR 1.3.1 Increased Access to Production Inputs Timeframe: 1 - 3 Partners: ACDI/VOCA Indicator: Sales volume of (a) fertilizers (b) improved seeds; Number of enterprises purchasing inputs IR 1.3.2 Increased Access to Credit Timeframe: 1 - 3 Partners: Shorebank, ACDI/VOCA IFC, AED, EU/TACIS, SCF, FINCA Indicators: Total value of loans by target area and gender; Number of borrowers by target area and gender IR 1.3.5 Improved Policy/Legal/ Regulatory Environment and Physical Infrastructure Supporting Small and Medium Enterprises World Bank, EU/TACIS IR 1.3.4 Private Membership Associations Organized Timeframe: 1 - 3 Partners: ACDI/VOCA CDC, Shorebank, Eurasia, AED, LOL, FINCA Indicators: Number of associations providing services whose revenues cover operating expenses; IR 1.3.3 Increased Market Responsiveness of Enterprises Timeframe: 1 - 3 Partners: ACDI/VOCA, CDC, Shorebank, AED, LOL Indicators: Product/ service differentiation; Number of markets; Number of nontraditional IR 1.3.3.1 Improved Enterprise Business, Technical, and Marketing Skills Timeframe: 1 - 3 Partners: ACDI/VOCA, CDC, Eurasia, AED, LOL, CHF Indicator: Number of enterprises that apply training skills IR 1.3.3.2 Increased Adoption of Grades and Standards Timeframe: 1 - 3 Partners: ACDI/VOCA, CDC, LOL Indicators: Number of products/services for which grades and standards are established; Sales of products/services meeting grades and standards IR 1.3.3.3 Improved Access to Market Information Timeframe: 1 - 3 Partners: Shorebank, ACDI/VOCA, AED, LOL Indicator: Share of market price received by farmer PAPA Final Evaluation Prepared For USAID / Azerbaijan June 2003 Prepared By David Neubert / Checchi & Co. Inc. Project Target Area SOW Objectives „ “The principal focus of the evaluation shall be the performance of PAPA & AMAD activities in accomplishing the terms & objectives of their respective Cooperative Agreements and their contribution to SO 1.3” USAID, Baku PAPA: Sources & Uses of Funds „ Total Budget 2.99 Million Over 3 Years „ 80% USAID Funded 20% Match Contribution „ Uses of Funds - Estimates „ Association development: 40% of budget „ Technical workshops & firm level TA: 35% „ Marketing/market information system: 25% Contribution to SO 1.3 – Asset Value Client Firms Asset Value 0 20 40 60 80 End YR 2000 End YR 2001 End YR 2002 USD x Million A sset V alue 5.28 14.2 59.76 End Y R 2000 End Y R 2001 End Y R 2002 Contribution to SO 1.3 - Employment Numbe r of Pe rsons Employe d by Clie nt Firms 0 2000 4000 6000 8000 End YR 2000 End YR 2002 Num be r of Em ploye e s Male Employees 430 3129 Female Employees 264 2967 End Y R 2000 End YR 2002 Annex G PAPA Activity Final Evaluation PowerPoint Presentation Master Slides Prepared For USAID, Baku, Azerbaijan 30 June 2003 Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 45 Contribution IR’s 9 New Products, Services & Mkt’s 757 9 New Enterprises Entering Mkt 188 9 Association Created 6 o Association Collecting Dues 5 9 New Product Standard Used 28 9 Sales of Products w/ New Standards 5.0 million USD Contribution to IR’s „ New Opportunity to Receive Market Information: 204 over 11 quarters „ Share of Market Price Received - Milk „ Farmgate 850 AZM (100%) „ Informal Market 1000 AZM (85%) „ Pasteurized 2200 AZM (39%) „ UHT 3200 AZM (27%) „ Share of Market Price Received - Hazelnuts „ Farmgate price .46 to .58 USD/kg (100%) „ CIF Germany 1.65 to 2.00 USD/kg (28 to 29%) Number of Association Members „ Zagatala Livestock Association: 65 „ Guba Livestock Association: 52 „ Ismailly Livestock Association: 52 „ Azerbaijan Dairy Processors Assoc. 26 „ Subtotal Livestock Products Sector 195 „ AZ Hazelnut Growers Association 1,150 „ AZ Hazelnut Processors Association 13 „ Subtotal Hazelnut Sector 1,163 „ Grand Total, both Sectors 1,358 Training & Results „ Training opportunities provided: 4009 „ Employees using skills from training: 3478 (87% adoption rate – very good) „ Clients adopt skills and methods that have a positive impact on their bottom line „ Technical skills have proven easier to transfer than managerial skills Association Lobbying: Small but Important Steps „ Dairy Processors Association effectively lobbied the GOA to drop plan to require licenses for ice-cream street venders „ Ismailly Livestock Association successfully lobbied regional government to write letter of support of their Association registration to Min. of Justice and provide them with office space Association Development Key Benefits “ Before the project helped us set up the association, I never talked with other people in my industry, if fact I didn’t even know who they were… The most important benefit from the project has been the creation of a network for people in my industry….” paraphrased from interviews Hazelnut TA Successes „ Improve hazelnut farmgate quality „ Played key role in developing “quality based pricing system” for hazelnuts at farmgate „ Assisted in increasing on farm yield & income „ Helped reverse trend of declining hazelnut orchard area „ Played key role in developing China market for in-shell nuts (new product and geography) Dairy Sector - Impacts „ 14 new milk plants, processing 30 mt/ day and employing 111 people started with Land O’Lakes TA. „ 4 old milk plants processing 8 mt/day and employing 102 persons restarted with Land O’Lakes TA. „ Average milk production in comparable season per milking cow increased by 19% by Guba, Ismailly and Zagatala livestock association members. „ Marginal increase in farm gate milk sales of $274,000 by Guba, Ismailly and Zagatala livestock association members, resulting in 52% ROI for this activity. Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 46 M&E: Targets „ Under the revised M&E system (9/2000) no targets were provided for PAPA measurables „ Develop target for each project in USAID AZ portfolio „ Use Task Level Efficiency to measure productivity (competitiveness) „ Use Job-Costing to determine project cost effectiveness M&E: Client Asset Values „ PAPA kept growing its client base over time, so they ended up measuring the growth in their client’s aggregate assets rather than determining if their clients were growing their assets „ Need to design asset value measurable that examines growth per firm Future Approach to Food & Ag „ Focus TA at the firm level (including farmer/ supplier TA) „ Keep market focused; provide training & support for increased consumer product quality and consumer education „ Encourage horizontal information & training within client firms „ Use international learning as tool of train and raise interest in formalized groups „ Use “Push” for market info distribution GOA Association Development Policy „ The Ministry of Justice has not processed any of the association applications it was sent by PAPA „ Is this a hill to die on?…..probably not „ USAID should look for alternative organizational forms Loss of Momentum „ Do not abandon associations which USAID has already started but look for alternative org. forms for new groups „ At RECP startup, work simultaneously on group organization, technical TA and competitiveness research, (this is a lesson learned from PAPA) Thank You Questions and Answers Land O’Lakes Azerbaijan PAPA Project, Final Evaluation 47 Annex H Interviewee Contact List Name Title and Organization Contact Info. Rob Nooter Director of Operations Land O’Lakes – IDI, Arlington, VA. rinooter@landolakes.com David Blood COP, Land O’Lakes, Baku, AZ Jeton Starova Deputy COP, Land O’Lakes, Baku, AZ + 994 50 221 5464 Kenul Ismailova Market Information Specialist Land O’Lakes, Baku, AZ + 994 93 6662 Dr. Hassan Musayev Technical Assistance Coordinator Land O’Lakes, Baku, AZ + 994 93 6662 Vugar Akhmadov Livestock Specialist Land O’Lakes, Baku, AZ + 994 93 6662 Gurbanali Alekperov Association Development Specialist Land O’Lakes, Baku, AZ + 994 93 6662 Nargiz Abdullayeva Field Representative Land O’Lakes, Zagalal, AZ + 352-1816 Nizami Bagirov Nizami Bagirov Cheese Company Ismailly, AZ Zabit Bagirov President, Ismailly Livestock Association Ismailly, AZ + 994 178 5 38 57 Dashamir Safaraliyev V.P., Ismailly Livestock Association Ismailly, AZ + 994 178 5 38 57 Islam Huseynov President, AZ Dairy Processors Association Baku, AZ ase2001@hotmail.com Malik Aliyer Member, AZ Dairy Processors Association Buka, AZ Ismail Huseyrov Member, AZ Dairy Processors Association Baku,AZ Dr. Movsum Shikhiyev Owner & G.M. of Milk-Pro Ltd. Buka, AZ + 994 12 904554 Murtuzali Shabanov V.P AZ Hazelnut Growers Association Zagatala, AZ Meheti Omarov President Zagatala Livestock Association Zagatala, AZ Rasul Razzagov V.P. Zagatala Livestock Association Zagatala, AZ Fuad Mammadov V.P AZ Hazelnut Processors Association Zagatala, AZ. +994 17 454606 David Sulaberidze. COP, ACDI/VOCA Buka, AZ +994 12 97 6268 Mirajeddin Amirov Cottage Industry Cheese Marker Bargov Village, Guba, AZ Shakir Huseynov Cottage Industry Cheese Marker Amsar Village, Guba, AZ D. Ian Gregory Director Market Development, IFDC, Muscle Shoals, AL igregory@ifdc.org David J,. Garms Development Officer, IFDC, Washington DC dgarms@ifdc.org Manfred Smotzok COP, AMDA, Baku, AZ smotzok@ifdc-az.org Ylli Bicoku Assoc. & Business Advisor IFDC, Baku, AZ. Jeffery Lee Deputy Country Coordinator USAID, Baku, AZ lee@usaid.gov Peter Duffy Economic Development Advisor USAID, Baku, AZ 980-335 pduffy@usaid.gov Leyla Najafzade Translator Baku, AZ + 994 50 352 1816