MID-TERM EVALUATION OF THE AGRIBUSINESS COMPETITIVENESS ACTIVITY IN TAJIKISTAN April 2022 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by ME&A, Inc. MID-TERM EVALUATION OF THE AGRIBUSINESS COMPETITIVENESS ACTIVITY IN TAJIKISTAN (ACAT) April 2022 Contract No./Order No.: IDIQ 72011518D00003/TO 72011521F00005 USAID/Tajikistan Contractor: ME&A, Inc. 4350 East-West Highway, Suite 210 Bethesda, MD 20814 Tel: (301) 652-4334 www.meandahq.com Evaluation Team: Katerina Stolyarenko, Team Leader Firuz Ordinaev, Local Economic Technical Specialist Mukhammadjon Eshbadalov, Agribusiness Specialist / Evaluator Mirela McDonald, Project Director/Quality Control Alex Dow, Program Manager/Data Analyst Wondimu Wolde, Senior M&E Specialist DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTENTS EXECUTIVE SUMMARY................................................................................................................................................... i 1.0 EVALUATION BACKGROUND, PURPOSE, AND QUESTIONS......................................................... 1 1.1 Evaluation Background and Purpose............................................................................................... 1 1.2 Evaluation Questions.......................................................................................................................... 1 2.0 PROJECT CONTEXT AND BACKGROUND............................................................................................ 2 2.1 Project Context................................................................................................................................... 2 2.2 Project Background............................................................................................................................. 2 3.0 EVALUATION METHODS AND LIMITATIONS ....................................................................................... 4 3.1 Evaluation Methodology..................................................................................................................... 4 3.2 Evaluation Limitations......................................................................................................................... 7 4.0 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS ................................................................ 8 4.1 EQ 1: Given the performance of ACAT’s grants component to date, what should be changed to ensure that the PIF achieves key program objectives, e.g., increasing production, productivity, and competitiveness of agribusinesses? .......................................... 8 4.2 EQ 2: What improvements to the management structure might be needed to ensure that ACAT achieves the activity goal and objectives?....................................................................... 16 4.3 EQ 3: Midway, what about ACAT’s market system approach to improving agribusiness competitiveness in Tajikistan is working well and what is not?.............................................. 21 4.4 EQ 4: What should be improved to ensure scaling up and sustainability of the market system approach?............................................................................................................................... 32 ANNEXES ......................................................................................................................................................................... 38 Annex 1: Statement of Work.......................................................................................................................... 39 Annex 2: List of Meetings and Interviews..................................................................................................... 44 Annex 3: List of Documents Consulted........................................................................................................ 49 Annex 4: Evaluation Matrix.............................................................................................................................. 51 Annex 5: Level of Achievement of Set Targets by ACAT as of the End of FY 3 ................................ 54 LIST OF TABLES Table 1: Interview Status by Group ............................................................................................................................. 5 Table 2: FGDs by Category and Disaggregated Number of Participants ........................................................... 6 Table 3: Total Number of Interviews by Group ....................................................................................................... 6 Table 4: Improvements in Procurement Processes by ACAT in 2020-2022 (# of days) .............................. 13 Table 5: Summary of KII Responses – EQ 2 ............................................................................................................ 19 Table 6: Summary of Findings from Direct KIIs (Stakeholders) – EQ 3 ........................................................... 27 Table 7: Grant Status by FGD by Value Chain – EQ 3 ......................................................................................... 27 Table 8: Summary of Findings from Direct KIIs (Stakeholders) – EQ 4 ........................................................... 33 LIST OF FIGURES Figure 1: Average Duration of Revision of Grant Applications ............................................................................. 9 Figure 2: Number of In-Kind Grant Agreements Signed by ACAT with Grantees Disaggregated by Year and Value Chain (N=162) ................................................................................................................................ 9 Figure 3: Type of Grants Disbursed by ACAT Through PIF as of December 31, 2021 ............................... 10 Figure 4: Country of Origin of Equipment Purchased for Grantees by ACAT ............................................... 10 Figure 5: Actual PIF Fund Budget Utilization Rate Disaggregated by Year ....................................................... 12 Figure 6: PIF Funds Distribution .................................................................................................................................. 13 Figure 7: Average Duration of ACAT’s Grant Processes Disaggregated by Year .......................................... 14 Figure 8: Grantees’ Assessment of the Level of ACAT’s Team Communication and Responsiveness ..... 18 Figure 9: ACAT Grantees by Value Chain (N=163) .............................................................................................. 22 Figure 10: Share of Grantees Per Value Chain by Budget Category (N=71) .................................................. 23 Figure 11: Assessment of the Impact of the Loans for Business (N=77) .......................................................... 25 Figure 12: Perception of Grantees on the Main Outcomes Achieved Thus Far as a Result of Participation in ACAT (N=92) ............................................................................................................................................. 30 ACRONYMS Acronym Description ACAT Agribusiness Competitiveness Activity in Tajikistan ADB Asian Development Bank ALG Agriculture, Land, and Governance Activity BDSP Business Development Service Provider CATI Computer Assisted Telephone Interviews CLA Collaborating, Learning, and Adapting COP Chief of Party COVID-19 Coronavirus Disease 2019 CPARS Contractor Performance Assessment Report System CTJ Competitiveness, Trade, and Jobs Activity DCOP Deputy Chief of Party DO Development Objective EQ Evaluation Question ET Evaluation Team EU European Union F&V Fruit and Vegetable FAO Food and Agriculture Organization FGD Focus Group Discussion FI Financial Institution FY Fiscal Year GD Group Discussion GDP Gross Domestic Product GoT Government of Tajikistan HQ Headquarters HR Human Resources IFAD International Fund for Agricultural Development IPM Integrated Pest Management IR Intermediate Result KBA Khatlon Business Association KII Key Informant Interview M&E Monitoring and Evaluation ME&A ME&A, Inc. MEL Monitoring, Evaluation, and Learning MCC Milk Collection Center MCCP Milk Collection Cooling Point MCF Microcredit Fund MCI Making Cents International MCO Microcredit Organization MEL Monitoring, Evaluation, and Learning MFI Microfinance Institution Acronym Description MSME Micro, Small, and Medium Enterprise NGO Non-Governmental Organization PHH Post-Harvest Handling PIF Partnership and Investment Fund PIFAC Partnership and Investment Fund Advisory Committee PO Purchase Order PR Purchase Request RFQ Request for Quote SME Small and Medium Enterprise SOW Statement of Work STTA Short-Term Technical Assistance TA Technical Assistance TAWA USAID Tajikistan Agriculture and Water Activity UNDP United Nations Development Program USAID United States Agency for International Development USG United States Government Winrock Winrock International WFP World Food Program ZOI Zone of Influence i EXECUTIVE SUMMARY EVALUATION PURPOSE The purpose of this evaluation is to assess the progress of the Agribusiness Competitiveness Activity in Tajikistan (ACAT) funded by the United States Agency for International Development in Tajikistan (USAID/Tajikistan) toward projected outcomes and objectives to enhance learning and provide accountability. The evaluation also considered which of the Activity’s interventions are working well, which are not, and why, together with an understanding of what has been accomplished against the intended results of ACAT’s implementation mandate. PROJECT BACKGROUND The agriculture sector in Tajikistan, and the Khatlon Region in particular, regarding the horticulture and dairy sub-sectors largely remains underdeveloped; it is characterized by poor crop yields, inefficient production and marketing, high levels of post-harvest loss, and the inability of value chain actors to meet export market quality standards. A solution to addressing these constraints and fostering growth and development is diversification into higher value-added products through the adoption of new technologies and innovations, for example in food processing, which could significantly enhance domestic and international trade opportunities within Tajikistan’s agriculture and agribusiness market ecosystem. ACAT’s role is to act as a catalyst in driving initiatives forward and toward sustainability in the longer term for project beneficiaries in Khatlon. ACAT is a five-year program implemented by Winrock International (Winrock) and subcontractor Making Cents International (MCI). The activity runs from July 2018 through June 2023, with a total estimated cost of $18.6 million, including a $3.7-million grant component. The activity is implemented in 13 districts in the southwestern Khatlon Region of Tajikistan. EVALUATION DESIGN, METHODS, AND LIMITATIONS To conduct the evaluation of ACAT, the evaluation team (ET) employed a mixed-methods approach which allowed for triangulation of findings. The team gathered the data and information needed to respond meaningfully to USAID/Tajikistan’s four evaluation questions (EQs) posed in the statement of work (SOW) from several qualitative and quantitative sources. Qualitative sources included review of project documents, fieldwork consisting of 66 key informant interviews (KIIs) and seven focus group discussions (FGDs) with 57 project beneficiaries and stakeholders, and three group discussions (GDs) with seven young women entrepreneurs. In addition, a local research company, M-Vector, conducted a telephone mini-survey with 182 participants to obtain feedback from project partners and key stakeholders, ACAT grantees, and the project’s small and medium enterprise (SME) loan recipients. The ET encountered three notable challenges to its research activities: 1) a presidential visit to Khatlon during the field trip that required a rescheduling of interviews; 2) a delay in meeting with government officials caused by the need for DipNotes1 to be approved at the highest levels before interviews could be conducted; and 3) the team leader’s inability to travel to Tajikistan for in-person interviews due to Coronavirus Disease 2019 (COVID-19) travel restrictions. However, these challenges did not affect the ET’s findings in any significant manner. Its two experienced local experts carried out face-to-face interviews, while the team leader monitored progress and checked quality of data on a daily basis, building on his experience conducting evaluation remotely. 1 Diplomatic Notes from USAID to government ministry counterparts confirming the purpose of the evaluation and requesting permission for the ministry to participate. ii FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS EQ 1: Given the performance of ACAT’s grants component to date, what should be changed to ensure that the Partnership and Investment Fund (PIF) achieves key program objectives, e.g., increasing production, productivity, and competitiveness of agribusinesses? Findings and Conclusions ACAT grant component experienced a number of bottlenecks in the first 2.5 years of implementation which caused delays in concept notes and grant applications screening, procurement and delivery of equipment and slow PIF budget utilization rate. The main reasons for that were a lack of technical staff and qualified procurement staff, underestimation on the complexity of procurement procedures for in￾kind grants, absence of specific targets on the number of grantees to be selected per value chain, a lack of collaboration among key project departments involved in the ACAT grant mechanism (PIF, Market Systems, Procurement, and Finance Departments), improper budgeting of PIF budget spending, and COVID-19. ACAT’s PIF component, after a slow start, has picked up speed in Year 3 and is making progress toward reaching out to and addressing the needs and expectations of its intended beneficiaries (partners). However, the significant delays with disbursement of grants by ACAT had a direct negative impact on the grantees. Delays with delivery and installation of in-kind machinery and equipment influenced timely installation and use of equipment for improving production, productivity, and competitiveness of agribusinesses. These delays have also significantly reduced the time available for ACAT to provide technical assistance (TA) and build the capacity of grantees to stimulate incorporation of professional and competitive approaches into their day-to-day operations. Recommendations • To speed up the process of purchase order (PO) approval and signature of grant agreements, increase the level of authority for signature of procurement documents by the Chief of Party (COP) from $5,000 to at least $25,000. • To ensure selection of proper local vendors, include the following criteria in the selection process: 1) prior vendor experience in delivery of the similar type of equipment as per the Request for Quote (RFQ), 2) provision of maintenance services for a year and availability of experienced maintenance engineers and technicians, and 3) existence of own warehouses for equipment storage. • Budget sufficient human resources for procurement in case the Activity plans to disburse a large number of in-kind grants. • To streamline the grant application process, from project start, set concrete targets and milestones on the number and type of enterprises to be supported through grants per value chain. EQ 2: What improvements to the management structure might be needed to ensure that ACAT achieves the activity goal and objectives? Findings and Conclusions The market system approach is complex and is highly demanding on project management, staff, and partners; the complexity of ACAT was initially underestimated. From a slow start at the beginning to Year 3, the ACAT team structure has solidified, with positions strengthened where required, additional appointments made, and roles and responsibilities clarified in line with the tasks to be carried out. This was realized due to the change in the Activity’s leadership. The management structure, as it is now, appears adequate to carry out ACAT’s remaining tasks. While respondents interviewed would have little knowledge or understanding of these internal staffing arrangements, from the comments, observations, and perceptions offered, an acceptable level of goodwill exists between those who have come in contact with ACAT staff, directly or indirectly, and their respective organizations. iii The expectation, of course, is that this goodwill continues and is further enhanced with the planned activities and interventions in ACAT’s Year 4 Work Plan. Based on comments from the Activity’s COP, significant progress will be made as well regarding the provision and level of technical support demanded by beneficiaries to improve their performance and profitability. Recommendations • ACAT should continue revising an organizational chart every year and reshuffle staff as needed. Consider further strengthening of the Bokhtar office through relocation of the M&E Director and Social Inclusion and Communication Specialist from Dushnabe to Bokhtar to better monitor the performance of grantees as well as to collect and document the successes and lessons learned of the model piloted by ACAT. • Consider decentralizing the human resources (HR) function to the Winrock Country Office to save time on recruitment within ACAT. • Strengthen the Activity’s monitoring and evaluation (M&E) system by setting up the database of all ACAT-supported grantees which would allow the Activity to track the performance of each grantee against the grant agreement obligations and roadmap targets. EQ 3: Midway, what about ACAT’s market system approach to improving agribusiness in Tajikistan is working well and what is not? Findings and Conclusions ACAT should be seen as a complex intervention with ambitious targets. Apart from strengthening agribusiness competitiveness using a market-driven approach it is also expected to contribute to jobs creation. This is difficult to achieve when the Activity is focusing only on one region, which is a relatively poor when compared to the other regions of the country where businesses are less developed to provide job opportunities. The other limiting factors are the limited project timeframe and delayed grant implementation. The value chains selected were appropriate, but the Activity implement strategically for the first two years and it engaged with hundreds of enterprises without looking at their potential to contribute to strengthening the value chain. As a result, midway in its implementation, ACAT has to adapt its implementation strategy to empower lead firms as drivers of value chain innovation. Recommendations • ACAT needs to focus on operationalizing the lead firms model, working with the suppliers, and building grantees’ capacity, particularly on developing their soft skills. The linkages within the value chains should also be explored and facilitated along with further work with microfinance institutions (MFIs), business development service providers (BDSPs), and agro-logistic centers on coaching the borrowers on how to use and manage the credit for their business. • Focus on strengthening the competitiveness of the private sector and creating employment using the market system approach by expanding the geographic coverage and supporting national-level projects in the future. • To ensure better sustainability of access to finance work and facilitate system-level changes, consider and explore the possibility of working directly with banks using an impact investment funding approach. EQ 4: What should be improved to ensure scaling up and sustainability of the market system approach? Findings and Conclusions The sustainability and scale-up of SMEs depends on activities that include employing highly qualified processing technologists, improving packaging, upgrading equipment, and updating the knowledge and skills of employees. Considering the significant delays in selecting grantees, forming the pool of dynamic lead firms (which occurred in early 2022), and providing equipment to the most grantees (which did not begin iv until the second half of 2021 and with many firms still waiting on receipt), ACAT is running out of time to ensure the scaling up and sustainability of the market system approach. At least one to two full harvest seasons after the initial investment are required to start to observe the results of ACAT’s assistance among the supported value chain actors. Thus far, ACAT has only been able to establish a fragile model of a market system in agribusiness and much work remains to be done with enterprises to help them to become more sustainable. Recommendations • Consider the extension of ACAT for at least two more harvest seasons to allow full rollout of the lead firm model and promote better sustainability of BDSPs. • Concentrate on building the capacity and sustainability of local BDSPs and shifting the focus from non￾governmental organizations (NGOs) as BDSPs to private sector providers which are more sustainable. Support further strengthening of industry associations organizational and functional management as well as the design and promotion of a range of fee-based services for the horticultural sector. • Work further on strengthening, operationalization and marketing of the agro-logistic centers for expanding access to professional and efficient logistic solutions. • Support further agri-inputs network to ensure its transition into farmer service centers (FSCs) capable of offering high-quality agri-inputs and professional and efficient related services at the district level in Khatlon. 1 1.0 EVALUATION BACKGROUND, PURPOSE, AND QUESTIONS 1.1 EVALUATION BACKGROUND AND PURPOSE United States Agency for International Development Central Asia Mission (USAID/Central Asia) contracted ME&A, Inc. (ME&A) to conduct the mid-term performance evaluation of the Agribusiness Competitiveness Activity in Tajikistan (ACAT), a five-year program implemented by Winrock International (Winrock) and subcontractor Making Cents International (MCI). The activity runs from July 2018 through June 2023, with a total estimated cost of $18.6 million, including a $3.7-million grant component. ACAT’s goal is to strengthen agribusinesses and agricultural production in the horticulture and dairy value chains by using market-driven, private-sector-led approaches to provide assistance to diversify product lines and leverage promising domestic and international market opportunities for agricultural associations, cooperatives, market integrators, and micro, small, and medium enterprises (MSMEs). The Activity is implemented in 13 districts in the southwestern Khatlon Region of Tajikistan. ACAT looks to promote innovations and enhance adoption of new and existing technologies in the market system. The purpose of the mid-term performance evaluation of ACAT was to: “assess the progress of ACAT toward projected outcomes and objectives to enhance learning and provide accountability.” The evaluation also considered which of the Activity’s interventions are working well, which are not, and why, together with an understanding of what has been accomplished against the intended results of ACAT’s implementation mandate. The findings of the evaluation will enable USAID, Winrock, and related stakeholders to have a clearer understanding of the Activity’s results and contributions to date, with suggested modifications or mid-course corrections needed to help guide or refocus ACAT’s second-half implementation activities. This evaluation was conducted from September to November 2021, by an evaluation team (ET) that included a team leader, an agribusiness specialist/evaluator, and a local economic technical specialist. The team was assisted by a local firm, M-Vector, which conducted in-field data collection. 1.2 EVALUATION QUESTIONS The ACAT evaluation’s statement of work (SOW) provided a set of questions to be answered by the ET, which were subsequently elaborated upon in the ET’s work plan and evaluation design. They are as follows: 1. Given the performance of ACAT’s grants component to date, what should be changed to ensure that the Partnership and Investment Fund (PIF) achieves key program objectives, e.g., increasing production, productivity, and competitiveness of agribusinesses? 2. What improvements to the management structure might be needed to ensure that ACAT achieves the activity goal and objectives? 3. Midway, what about ACAT’s market system approach to improving agribusiness in Tajikistan is working well and what is not? 4. What should be improved to ensure scaling up and sustainability of the market system approach? 2 2.0 PROJECT CONTEXT AND BACKGROUND 2.1 PROJECT CONTEXT While agriculture is the largest sector in Tajikistan’s economy, contributing 21 percent of the country’s gross domestic product (GDP) and employing 70 percent of the labor force,2 annual growth, particularly in the horticulture and dairy sub-sectors, remains restricted due to several important factors: poor crop yields, inefficient production and marketing, high levels of post-harvest wastage, and the inability of value chain actors to meet export market quality standards. The situation is more acute in Khatlon than in other regions due to Khatlon’s dependency on agriculture, where low wages have driven male labor migration to urban areas and Russia, resulting in higher poverty levels for women and the establishment of numerous women-owned farms and agribusinesses. The Khatlon Region accounts for 53 percent of Tajikistan’s arable land, including 54 percent of grain fields and 63 percent of cotton fields. It produces 55 percent of the country’s vegetables, 78 percent of melons, 40 percent of fruits, 52 percent of meat, and 48 percent of milk.3 As such, the potential for the region to develop and prosper is significant. However, the region needs a more robust competitive agribusiness sector, especially in the horticulture and dairy value chains, to encourage and stimulate agricultural commercialization, export-led growth, food import substitution, enhanced food security, employment creation, and poverty reduction. Diversification into higher-value-added products through the adoption of new technologies and innovations, e.g., in food processing, would significantly enhance domestic and international trade opportunities within Tajikistan’s agribusiness market ecosystem. 2.2 PROJECT BACKGROUND USAID/Tajikistan’s decision to support and fund ACAT was based on several important factors, including building on previous USAID projects, in particular, to support the potential of the agriculture sector in the horticulture and dairy value chains in 13 districts in the southwestern Khatlon Region.4 The primary goal of ACAT is “Sustained economic growth through interventions aimed at improving competitiveness of Tajikistan’s agribusiness enterprises leading to increased economic development, employment, and livelihoods.” In turn, this will contribute to the USAID/Central Asia Regional Development Cooperation Strategy Development Objective (DO) 1: Expanded diverse and competitive trade and markets. The intermediate results (IRs) applicable to ACAT include IR 1.1: More diverse and competitive private sector, and IR 2: Enhanced agricultural competitiveness and food security. ACAT was effectively designed to contribute toward the creation of a market-driven, private-sector-led, sustainable economy focusing on agribusinesses in Khatlon and, more specifically, to transform agricultural production in the horticulture and dairy value chains from subsistence farming to commercial orientations. According to ACAT’s Monitoring, Evaluation, and Learning (MEL) Plan, originally prepared in September 2018 and subsequently modified in October 2019 and February 2021, ACAT pursues its 2 Data from United Nations Development Program (UNDP), December 2019, project document “Strengthening Communities in Khatlon Region and Rasht Valley in Tajikistan.” 3 Fez “Kulob” – State Institution of Administration of Free Economic Zone “Kulob” was established on the basis of Decree No. 97 of the Government of Republic of Tajikistan from March 1, 2019, on a total area of 309.32 hectares of land of industrial-innovative type, for a period of 25 years. 4 Khuroson, Abdurahmoni Jomi, Yovon, Kushoniyon, Vakhsh, Levakand, Jaloliddini Balkhi, Jaihun, Dusti, Kubodiyon, Shahritus, Nosiri Khisrav, and Panj. 3 primary goal through three objectives: • Objective 1: Diversify the agricultural sector to include higher-value commodities and value-added products. • Objective 2: Strengthen linkages between aggregators, wholesalers, exporters, and international and domestic end-markets. • Objective 3: Partner and leverage resources to catalyze systemic change in the agribusiness sector. ACAT aims to prioritize private-sector-led service delivery and strengthen linkages among buyers, producers, and service providers to maximize results and transform and diversify the market system for horticulture and dairy products. Program activities include supporting lead MSME firms, including entrepreneurs, business development service providers (BDSPs), and financial institutions (FIs), to seize on scalable and replicable opportunities, improve operational performance, enhance adoption of new and existing technologies, enable opportunities for women and youth, and build stronger value chain relationships to catalyze systemic change.5 In addition, ACAT works to enhance the adoption of new and existing technologies and overcome the main barriers of technology adoption faced by small enterprises with low margins by utilizing a PIF. The PIF takes multiple forms, including providing cost-sharing grants to buy down risk of new business models, a challenge fund to spur technology innovation, and subcontracts to deliver services. Winrock works in close collaboration with subcontractor MCI, which focuses primarily on challenges in capacity-building and access to finance for previously marginalized groups, women, and youth in agriculture. Enterprises that would qualify for support through ACAT were categorized and grouped by cluster value chains in two sectors—dairy and horticulture—and six value chains: fruit and vegetable (F&V) canning, F&V drying, services for horticulture value chains, nut processing, dairy processing, and dairy aggregation. ACAT classifies potential beneficiaries (partners) as follows: 1. Lead firms have a minimum of five full-time staff, have the potential to attract a number of value chain actors, understand business strategies, and can act as models for other actors. 2. Small enterprises employ at least three full-time staff and have an organizational set-up but no clear business strategy or capacity to lead. 3. Micro enterprises are small family or individual enterprises, usually without a formal employment system and without the capacity to operate an agribusiness. 4. Service providers, including financial services institutions that have an agreement with ACAT (and a grant for micro-loans) and BDSPs and other stakeholders with an agreement with ACAT for the implementation of project activities. To monitor progress on the achievement of predetermined targets against indicators, ACAT compiles data and information in its quarterly reports highlighting actual versus plan relative to each of the three objectives above. The performance indicators6 ACAT needs to achieve are aimed to align with the IRs to be met under each objective. 5 ACAT Year 1 Annual Work Plan. 6 These are reflected in ACAT’s Logical Framework Matrix in its MEL Plan. 4 3.0 EVALUATION METHODS AND LIMITATIONS 3.1 EVALUATION METHODOLOGY To conduct the ACAT evaluation, the ET employed a mixed-methods approach which allowed for triangulation of findings. Data and information needed to respond meaningfully to USAID’s four evaluation questions (EQs) posed in the SOW were gathered from several qualitative and quantitative sources. Qualitative sources included a review of project documents and fieldwork consisting of key informant interviews (KIIs) and focus group discussions (FGDs) with project beneficiaries and stakeholders. In addition, the ET commissioned a telephone mini-survey, engaging a local research company, M-Vector, to obtain feedback from project partners, key stakeholders, ACAT grantees, and the project’s small and medium enterprise (SME) loan recipients.7 Quantitative sources included a review of ACAT’s monitoring and evaluation (M&E) reports and other relevant documents and reports prepared by the Activity, including work plans, surveys, assessments, and financial reports. The ET also reviewed official publications, including data and statistics relevant to the team’s investigations from government ministries and departments, The World Bank, European Union (EU), International Fund for Agricultural Development (IFAD), Food and Agriculture Organization of the United Nations (FAO), and United Nations Development Program (UNDP), among others. Field work was conducted from October 19 to November 5, 2021 and the follow-up remote meetings took place between February 7 and March 2, 2022. The evaluation was implemented using a hybrid approach, with the team leader operating remotely and the agribusiness expert, local technical specialist, and the local survey firm (M-Vector) operating in the field. The team leader conducted virtual interviews with stakeholders, including USAID staff, ACAT personnel, donor community stakeholders, business associations, national government, private sector (industry associations), and others. The field team conducted in-person KIIs, surveys, and FGDs of several groups of stakeholders and beneficiaries, including industry associations, microfinance institutions (MFIs), community-based organizations, and district government officials. 3.1.1 Qualitative Data Collection and Analysis The qualitative component of the performance evaluation drew on four data-collection methods. Structured Desk Review The ET was provided with project documents by USAID/Tajikistan prior to mobilization.8 The local team members and home-based team leader reviewed and retained these project documents for reference throughout the evaluation, including during the in-country field activities. The ET reviewed ACAT’s available work plans, annual and quarterly reports, annual implementation plans, annual surveys of target enterprises, Coronavirus Disease 2019 (COVID-19) impact survey, a baseline enterprise assessment survey, official government publications and regional development strategies, as well as other relevant secondary information, including publications from ACAT’s partners and associations. Published reports by organizations such as IFAD, FAO, the World Bank, UNDP, Asian Development Bank (ADB), World Food Program (WFP), Aga Khan Foundation, and International Monetary Fund (IMF) were also consulted for facts and data pertinent to Tajikistan’s economy and the Khatlon Region, in particular. 7 Loans disbursed via ACAT’s four micro-finance banks: MFI Sarvati Vakhsh, MFI Chilu chor Chashma, MFI Mehnatobod, and Microcredit Fund (MCF) Imkoniyat. 8 See Annex 3 for a complete list of all documents reviewed by the ET. 5 Team members accessed additional available published material and documents, including Government of Tajikistan (GoT) economic reports and development strategies, and documentation on agriculture and agribusiness via online searches on project-related issues. These documents served to provide the context for ACAT’s project interventions and inform the ET on past, present, and planned activities by interested parties—local and international—on rural agricultural development in Tajikistan and Khatlon in particular, focusing on the potential for improved agribusiness opportunities both domestically and for export markets. Key Informant Interviews The ET conducted KIIs with ACAT staff in Dushanbe and Bokhtar, direct project beneficiaries, project partners, and other project stakeholders, including Khatlon government and district officials, business development associations, MFIs, the international donor community, and women’s organizations. The team conducted 66 KIIs in Dushanbe, in the 13 districts in Khatlon (ACAT’s area of activity) (40 initial KIIs and 26 additional KIIs), and remotely between the team leader and interviewees based in Tajikistan. In total, 83 stakeholders who had a direct or indirect relationship with or knowledge of the project participated in meetings. All KIIs used semi-structured, open-ended questionnaires tailored to the category of project stakeholder interviewed while keeping the four EQs in mind. Table 1 indicates the number of interviewees by group. Table 1: Interview Status by Group Focus Group Discussions The ET also conducted seven FGDs in the Khatlon Region. FGDs used semi-structured, open-ended questionnaires to gather data and information from participants to inform the ET’s understanding of their respective roles and/or connection with ACAT and its various activities. FGDs were designed to obtain feedback from direct beneficiaries in terms of their individual and/or collective experiences with ACAT, resulting in meaningful and valid comments, observations, perceptions, and suggestions related to project interventions and initiatives. FGDs included participants gathered from all seven of ACAT’s value chains in various locations in Khatlon districts. Fifty-seven (57) individuals (38 male and 19 female) attended FGDs. Table 2 provides the categories and number of disaggregated participants. 9 Includes four USAID projects: USAID’s Future Growth Initiative (FGI) Project, USAID/DAI’s Agriculture Land Governance Project, USAID/Aga Khan Foundation THRIVE Project and USAID’s Trade Central Asia (TCA) Activity Interview Status by Group No. of Interviews No. of Participants Interviewed 1 USAID personnel 6 8 2 ACAT personnel 17 18 3 MCI 1 1 4 International donor community 99 13 5 MFI project partners 6 7 6 Government departments (Dushanbe & Khatlon districts) 8 16 7 Business associations 6 7 8 BDSPs 3 3 9 Non-governmental organizations (NGOs) 3 3 10 Lead firms (2 on dairy and 2 on horticulture) 4 4 11 Micro-enterprises (2 on dairy and 1 on horticulture) 3 3 Total 66 83 6 Table 2: FGDs by Category and Disaggregated Number of Participants Category M F T Position (s) Location Date 1 Milk suppliers 5 6 11 Owners Yovon District October 2021 2 Loan recipients 4 4 8 Individual Entrepreneurs Khatlon Districts (Various) October 2021 3 Horticulture 6 2 8 Head of Enterprise Khatlon Districts (Various) October 2021 4 Dairy value chain 7 1 8 Head of Enterprise Khatlon Districts (Various) October 2021 5 Post-harvest handling (PHH) – in-kind grants 6 2 8 Head of Enterprise Khatlon Districts (Various) November 2021 6 BDSPs 4 2 6 Directors Khatlon Districts (Various) November 2021 7 Enterprises selected – no grants yet 6 2 8 Owners Khatlon Districts (Various) November 2021 Total 38 19 57 Group Discussions The ET also conducted three group discussions (GDs) with young women to acquire their views and perceptions in the absence of older participants. The purpose was to ensure they could talk freely, which may not have been the case had their seniors been present. A total of seven women participated and offered their thoughts and comments in a calm and confidential atmosphere. 3.1.2 Quantitative Data Collection and Analysis In addition to the above methods, the ET conducted a telephone mini-survey, implemented by M-Vector, which reached out to the following three categories of respondents: 1) key stakeholders and project partners; 2) project grantees; and 3) loan recipient project beneficiaries. The information gathered from this mini-survey contributed to the ET’s overall understanding of ACAT’s activities to date, and gathered important feedback on issues of concern raised by respondents, which were cross-referenced with data and information gathered from KIIs and FGDs. Table 3 provides a brief summary of the completed mini￾surveys. Table 3: Total Number of Interviews by Group Group Description Number of Partners in the List Number of Successful Interviews Total Males Females Total Males Females 1. ACAT participants/grantees – Total, including: 155 95 60 94 63 31 Agri-logistics 3 3 - 3 3 - F&V processing, canning 3 2 1 2 1 1 Carton boxes for F&V 3 3 - 1 1 - Dairy products 7 6 1 5 4 1 F&V drying 4 3 1 2 1 1 Ice cream 1 1 - 1 1 - Milk collection, cooling, storage 4 4 - 1 1 - Nut processing 1 1 - 1 1 - Nut shelling 21 14 7 13 9 4 Peanut processing 1 1 - 1 1 - Persimmon drying 4 2 2 3 2 1 7 Group Description Number of Partners in the List Number of Successful Interviews Total Males Females Total Males Females Pistachio processing 6 3 3 6 3 3 Pruning, spraying, grafting 88 50 38 49 34 15 Qurut production10 9 2 7 6 1 5 2. Recipients of loans supported by ACAT – Total, including: 340 79 261 77 20 57 Greenhouse 83 36 47 25 14 11 F&V processing 26 8 18 5 - 5 Dairy processing 53 32 21 8 6 2 Cow/calf 174 - 174 39 - 39 Cold storage 1 1 - 0 - - Processing 3 2 1 0 - - 3. ACAT partners and other stakeholders familiar with ACAT 15 10 5 11 6 5 Consulting and training services 10 7 3 6 3 3 Financial services 5 3 2 5 3 2 The mini-surveys were conducted from M-Vector’s call center in Dushanbe, Tajikistan using Computer Assisted Telephone Interviews (CATI). The professional interviewers/operators received training prior to starting the mini-survey, between October 22–29, 2021. The ET decided to use this survey mode for the following reasons: • Timing: The ET only had one week to complete the mini-survey. • Survey duration: The mini-surveys did not exceed 15 minutes, so it was appropriate to use CATI in this case. • Value: Using CATI allowed the ET to conduct three customized mini-surveys by beneficiary group instead of the one planned originally. • Quality: CATI allowed for very thorough quality control of each enumerator and interview in an office setting. • COVID-19: In the attempt to reduce COVID-19 risks for both respondents and enumerators, this mode was preferred over other options. Instruments/Questionnaires Separate questionnaires were developed for each mini-survey. The questionnaires were translated into local languages, tested/piloted in the field, and revised and finalized based on the conducted pilot. Final questionnaires were programmed for CATI. Sampling ACAT provided a list of 155 grantees, 340 loan recipients, and 15 partners. The survey reached out to all participants from the provided list and successfully interviewed 37.7 percent of them based on their availability and willingness to participate in the survey. Overall, 94 grantees (61 percent), 77 loan recipients (23 percent), and 11 partners (73 percent) were available and willing to take part in the survey. 3.2 EVALUATION LIMITATIONS The ET encountered the following issues that limited its research and investigations: 10 Qurut or Kashk is a range of dairy products produced from strained yogurt. 8 1. A presidential visit to the Khatlon Region (October 26–29, 2021) during the ET’s field trip there caused significant delays in organizing meetings, as few people were willing or able to attend. However, the team quickly rescheduled interviews and conducted the delayed KIIs and FGDs within the timeframe allocated. 2. The ET faced an issue reaching government officials in connection with DipNotes,11 which resulted in delays organizing meetings with ministerial representatives in particular. However, the ET eventually received permission to conduct interviews with no significant detrimental effect on its investigations. 3. Due to COVID-19, the team leader did not travel to the field but conducted interviews using Zoom, WhatsApp, and Microsoft Teams. The ET identified and secured interviews with all available stakeholders working closely with USAID and Winrock to ensure the most relevant stakeholder meetings with the team leader. To mitigate the team leader’s absence from the field, the ET utilized local firm M-Vector to conduct mini-surveys, both in-person and by phone, to ensure the highest number of respondents were reached. 4. The ET could not reach as many young women as it planned because most of the young women in rural areas were not allowed to leave their houses to participate in interviews. 4.0 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 4.1 EQ 1: GIVEN THE PERFORMANCE OF ACAT’S GRANTS COMPONENT TO DATE, WHAT SHOULD BE CHANGED TO ENSURE THAT THE PIF ACHIEVES KEY PROGRAM OBJECTIVES, E.G., INCREASING PRODUCTION, PRODUCTIVITY, AND COMPETITIVENESS OF AGRIBUSINESSES? 4.1.1 Findings As is evident from the document review, ACAT grant mechanism is composed of five stages: Stage 1 – Concept Note, Stage II – Grant Application, Stage III – Procurement, Stage IV – USAID Grant Package Approval, and Stage V – Grant Agreements. The evaluation interviews (KII=21) revealed that ACAT experienced challenges in four out of five stages (i.e., from Stage I to Stage IV) during 2018-2020. As per ACAT’s annual progress reports, the Activity has received at total of 813 concept notes; however, 52 percent of them (420) were not eligible. From the eligible concept notes, the Activity approved only 37 percent (147). Interviews with ACAT staff (KII=6) revealed two primary causes for the high rate of ineligible applications: a) the initially high criteria for the type of businesses to be supported, and b) enterprises’ lack of understanding of ACAT’s objectives, resulting from many years of donors’ interventions in Khatlon region that focused on the production rather than processing level. In total, ACAT has received 252 grant applications either through Annual Program Statements (APS) or Requests for Applications (RFAs), from which 95 applications were approved. From the document review, the Activity experienced significant delays in revision of both concept notes and grant applications. Twenty (20) percent of applications for in-kind grants (or 32 applications) stalled at pending revision for periods of time between nine months to 2.5 years. Moreover, Figure 1 shows that average duration of grant application review by the PIF ranged from 374-531 days in the first half of the project implementation. 11 Diplomatic Notes from USAID to government ministry counterparts confirming the purpose of the evaluation and requesting permission for the ministry to participate. 9 Figure 1: Average Duration of Revision of Grant Applications 465 531 374 194 0 200 400 600 2018 2019 2020 2021 days Year Source: PIF Dataset This led, in turn, to the small number of grant agreements signed by ACAT in the first two years of project implementation. In total, ACAT disbursed 162 in-kind grants in horticulture and dairy from September 2019 to February 2022. As shown in Figure 2, ACAT signed 78 percent (127) of the in-kind grants’ agreements in Fiscal Year (FY) 3, 20 percent (32) in FY 2, and the rest, 2 percent (3), in FY 1 and FY 4 Figure 2: Number of In-Kind Grant Agreements Signed by ACAT with Grantees Disaggregated by Year and Value Chain (N=162) 2 32 127 1 0 50 100 150 2019 2020 2021 2022 Source: PIF Dataset Based on feedback from the evaluation interviews (KII=5), this happened primarily because ACAT did not set targets from the start of the Activity in terms of the number and type of enterprises which should be supported in the targeted value chains. As a result, the ACAT team (the PIF Department and Market Systems Department) lacked the necessary resources to quickly review the huge number of applications (1,065 concept notes and grant applications, in total); the ACAT team spent time determining the criteria for funding businesses, including discussion of applications in the Partnership and Investment Fund Advisory Committee (PIFAC) and working with the grantees to determine their equipment needs. The start of the COVID-19 pandemic in early 2020 exacerbated the situation restricting technical staff from visiting potential grantees as part of the application screening process due to health concerns. Within this challenging context, there were additional difficulties which caused further delays. The Dushanbe and Bokhtar offices (PIF Department and Market Systems Department) failed to communicate well, ACAT lacked an internal time management structure for the grant application review process, and the process itself was poorly supervised by ACAT management based in Dushanbe. The high workload resulted in high staff turnover in the PIF Department. For example, the PIF Director position, one of project’s key personnel, has changed three times in 16 months (between August 2018-December 2019). In addition, ACAT initially budgeted only 30-40 percent of the PIF budget for in-kind grants with the remaining 60-70 percent budgeted for cash grants and sub-contracts. However, ACAT’s financial records indicate that, in fact, in-kind grants constituted 77 percent of PIFs committed, with cash grants and subcontracts constituting 17 percent and 6 percent, respectively. ACAT did not communicate the implication of this to the Mission in a timely manner, resulting in low ratings on schedule and cost control 10 in the first three years of implementation in the Contractor Performance Assessment Report System (CPARS). Figure 3: Type of Grants Disbursed by ACAT Through PIF as of December 31, 2021 Source: ACAT Financial Team In the course of implementation, the ACAT team realized that these grants involve complicated procurement issues since much of equipment has to be procured outside of Tajikistan, even in case of local vendors. The PIF data indicate that 57 percent of the equipment was purchased locally and the rest (43 percent) was procured internationally. ACAT purchased the small-scale equipment locally, mainly for nut processing enterprises, while the larger-scale equipment, for dairy and F&V processing enterprises, had to be sourced outside Tajikistan, mainly from Russia, Turkey, Central Asia (Kyrgyzstan, Kazakhstan, and Uzbekistan), China, and Ukraine. Figure 4: Country of Origin of Equipment Purchased for Grantees by ACAT 94 29 28 8 4 1 1 1 0 20 40 60 80 100 Tajikistan Russia Turkey Kyrgyzstan China Kazakhstan Ukraine Uzbekistan Number of grantees Source: PIF Dataset When equipment is procured internationally, this process requires the development of detailed technical specifications, knowledge of the exact quantities and quality of the required equipment, and alignment of selected equipment with existing grantee systems (e.g., electrical outlets, infrastructure, etc.). This necessitates a specific set of skills; and ACAT needed to add an engineer to the technical team, a position not initially in the budget. As a result, the team experienced significant challenges in preparing technical specifications, i.e., 43 Purchase Requests (PRs) for necessary equipment for grantees. Further, ACAT also lacked the staff and resources to implement proper procurement. The five-person PIF Department included just one local Procurement Specialist, who lacked the skills for such complex procurement processes. The project experienced challenges with the local vendors as well. Ensuring adequate pricing for insurance and transportation by local vendors required significant expertise and local knowledge to adequately assess competitiveness. In some instances, the team selected local vendors with insufficient experience who were unable to deliver the equipment in a timely manner. In one example, procurement of 20 nut cracking machines, delivery of the equipment was delayed for two years. 11 Activity-specific management and coordination bottlenecks added substantial delays to the procurement process as well. As previously stated, the coordination and exchange of the information among different departments—particularly the PIF, Procurement, and Financial Departments—complicated the procurement process, adding to its length. Each department maintains its own database of grantees and has separate grant mechanism procedures, which negatively impacts procurement planning, particularly establishing clear timelines for the procurement process (from Request for Quote [RFQ] to equipment on the ground). Preparation of the grant packages which requires a number of documents (e.g., negotiation memo, selection memo, scoring sheets from PIFAC, pre-waiver assessment, check in Lexis-Nexis system) was also time consuming. Other delays to the procurement process included delayed starts (the Procurement Specialist would wait to start the procurement process until the grant was complete) and inefficient approvals (purchase orders $5,000 have to be approved by Winrock Headquarters [HQ] rather than the ACAT Chief of Party [COP]). Both added extra time for the approval processes for the procurement of equipment for in-kind grants. Also, COVID-19 negatively impacted procurement due to border closures and challenges in shipping and importing equipment. These challenges led to the substantial delays with delivery of the equipment. As per PIF records, delivery of equipment to the grantees by ACAT started only in FY 2. Six percent of grantees (10 enterprises) received their equipment in FY 2, 62 percent (101 enterprises) in FY 3, and another 31 percent (51 enterprises) are currently waiting for equipment. The pending equipment is scheduled to be delivered in FY 4 (April-May 2022). However, due to the start of the Russian-Ukrainian conflict on February 24, 2022 and subsequent disconnection of Russian banks from the Society for Worldwide Interbank Financial Telecommunications (SWIFT) international payment messaging system on February 26, 2022,12 there may be delays with equipment delivery for an undefined period of time. KIIs and FGDs with ACAT grantees further validated the delays in delivery of equipment. The FGD with horticulture enterprises (six men and two women) revealed that none had received their equipment yet, although they signed the grant agreement almost a year ago. While the delivery of equipment is in process, most of them do not believe they will get the equipment on time because of delivery delays caused by COVID-19 and the border blockade with Kyrgyzstan, both of which have affected equipment imports. The FGD with dairy enterprises indicated that three out of eight participants had received some equipment; others have signed equipment contracts, and delivery is in process but delayed. The impact of COVID-19 on this process is significant. Because of COVID-19, vendors have not delivered equipment and five suppliers have dropped out of the process completely, with still others uninterested in continuing. In FGDs with ACAT BDSPs, mainly non-governmental organizations (NGOs), except for the Khatlon Business Association (KBA), two out of six participants indicated they had received in-kind grants. However, as with the horticulture enterprises, delivery of these grants was also delayed. This is partly due to COVID-19 and partly to ACAT’s grant procedures and long decision-making process, as BDSP KII participants noted. Because of these delays, the partners were told to change the work plans ACAT initially approved. The partners stated this will affect their ACAT-related work and activities as well as their other projects noting that—within the signed grant agreement—ACAT requires partners to do more than just build capacity. The FGDs with ACAT grantees (N=34 participants) showed that the levels of satisfaction were relatively low across all the cluster value chains as far as delivery of in-kind grants was concerned. Given the seasonal nature of agribusiness, this raised concerns for the respondents in terms of having what they need for their respective ventures on time. The FGDs also indicated the need for specific technical assistance (TA) 12 https://foreignpolicy.com/2022/03/08/swift-sanctions-ukraine-russia-nato-putin-war-global-finance/ 12 to support the grants to ensure that the enterprises are best equipped to produce to meet market demand at the right time. The delays with grants application selection and procurement resulted in low spending of PIF funds. As of December 31, 2021, the actual PIF budget utilization rate constituted just 27 percent. The evaluation interviews (KII=5) showed that ACAT had a high PIF pipeline especially in the first two years of implementation but was not able to spend $455,000 per month as planned. The amount of actual monthly expenditures ranged from $240,000-$280,000. Figure 5: Actual PIF Fund Budget Utilization Rate Disaggregated by Year 0% 0% 19% 27% 0% 5% 10% 15% 20% 25% 30% 2018 2019 2020 2021 Source: ACAT Financial Reports Under the Activity design, ACAT planned to start spending PIFs beginning in FY 1. This timeline was unrealistic given the amount of preparation required before launching the grant mechanism. To help address and alleviate this, in June 2021, ACAT decided to review and reassess its implementation strategies with a view to maximizing results for Year 4 and beyond. Following its deliberations, ACAT prepared a pause and reflect report as part of its Collaborating, Learning, and Adapting (CLA) approach and, accordingly, adjusted the PIF’s procedures to reflect the realigned project strategies. Partner selection criteria now focus on identifying entrepreneurial partners that can serve as models for replication. Once found, those partners would benefit from capacity building on important business skills such as marketing, product development, and improvement of productivity and quality. In addition to the pause and reflect exercise, the Activity decided to prepare a conceptual framework for the PIF to plan and project how its remaining funds should be allocated, guided by ACAT’s revised strategy and the PIF’s database of concept notes and grant applications. These actions led to an increase in PIF commitments. As per recent data shared by ACAT, as of December 31, 2021, committed PIFs total $3.38 million (91 percent of total funds), which includes $1.7 million already disbursed (i.e., 50 percent of the committed funds). ACAT plans to use the remaining balance of $277,868 (8 percent of total funds), to further support its partner enterprises with capacity-building measures and other related activities to enhance partners’ prospect of long-term stability. Figure 6 depicts the PIF’s current fund-distribution status. 13 Figure 6: PIF Funds Distribution Source: PIF Dataset The ET found ACAT has attempted to address these issues and improve the procurement process. Adding resources (e.g., attracting procurement consultants with specific expertise around international sourcing of equipment) and replacing the poorly performing Procurement Specialist with the international Procurement Manager has improved the procurement process, for example. Moreover, project records (Table 4) indicate ACAT has also improved its in-kind grants procurement when comparing FY 3 and FY 4 implementation. This was also acknowledged by USAID in the CPARS. There were decreases in the number of days required for the preparation of PRs (on ≈4-7 days), preparation of RFQs, and publishing (on ≈14-22 days) as well as in the selection of vendors (on ≈9-16 days). The overall average decrease in the whole procurement process constituted 34 days, i.e., from ≈178 days to ≈144 days. Table 4: Improvements in Procurement Processes by ACAT in 2020-2022 (# of days) FY PR RFQ publishing Selection Purchase Order (PO) Approval Delivery Payment Total Winrock HQ USAID Min Max Min Max FY3 (2020- 2021) 14 30 21 7 14 60 120 3 147 209 FY4 (2021- 2022) 7-10 RFQ preparation: 1-2 RFQ publication: 7-14 5-12 7 14 60 120 2-5 103 184 Source: ACAT Financial Team Yet, despite the improvement in the procurement process, the whole ACAT grant process—from submission of the concept note to the receipt of equipment by grantees—is still quite lengthy and varied from 941 days in 2018 to 422 days in 2021 (see Figure 7). 14 Figure 7: Average Duration of ACAT’s Grant Processes Disaggregated by Year 941 789 639 422 0 200 400 600 800 1000 2018 2019 2020 2021 days year Source: PIF Dataset The significant delays with disbursement of grants by ACAT had a direct negative impact on the ACAT grantees. Grants for enterprises were in-kind grants for the purchase of equipment for processing, packaging, shelling, cold storage units, carton box production, tools for pruning, and modular and milk analyzers for milk collection cooling points (MCCPs). The grant agreements were signed for a definite period of time (either by the end of December 2022 or March 2023) with a set of anticipated results related to the improvement in production, sales, and employment creation of supported enterprises. Therefore, delays with delivery and installation of in-kind machinery and equipment influenced on time available for usage of the installed equipment for improvement of production, productivity, and competitiveness of agribusinesses. Also, it has significantly reduced ACAT’s available time for provision of TA and capacity building to the grantees to stimulate incorporation of professional and competitive approaches into their day-to-day operations. Given that PIF Funds are mainly fully committed and the grant-making procedures have been already significantly improved and streamlined by ACAT, the ET prepared a set of recommendations on grants mechanism for future programming based on ACAT experience. 4.1.2 Conclusions # Conclusion C1.1 The ACAT grant component experienced a number of bottlenecks in the first 2.5 years of implementation which caused delays in review of concept notes and grant applications, procurement, and delivery of equipment, and a slow PIF budget utilization rate. The main reasons included: a lack of technical staff and qualified procurement staff, underestimation of the complexity of procurement procedures for in-kind grants; absence of specific targets on the number of grantees to be selected per value chain; a lack of collaboration among key departments involved in the ACAT grant mechanism (PIF, Market Systems, Procurement, and Finance); improper budgeting of PIF spending; and COVID-19. C1.2 The evidence presented to the ET indicates that ACAT’s PIF component, after a slow start, has picked up speed and is making progress toward reaching out to and addressing the needs and expectations of its intended beneficiaries (partners). All indications show that in the time remaining, the current COP has fully grasped what needs to be done regarding the grants and other connected activities, to ensure the PIF achieves key program objectives. 15 4.1.3 Recommendations # Recommendation Addressed to Timeframe Priority S/T L/T L M H Recommendations for current program R1.1 Speed up the process of PO approval and signature of grant agreements, increase the level of authority for signature of procurement documents by the COP from $5,000 to at least $25,000. Winrock HQ ▲ ▲ Recommendations for future programming R1.2 Budget sufficient human resources (HR) for procurement in case the Activity plans to disburse many in-kind grants. ACAT’s experience suggests that, due to the complicated process of procurement and the need to source almost all the equipment outside Tajikistan, it is important to budget at least two highly qualified Procurement Specialists with prior experience in setting up the operational procurement mechanism in line with the USAID standards, preparing technical specifications for the equipment to be purchased (PRs), and performing online searches for equipment in line with required specifications and geocode. To ensure any potential conflict of interest, hire an International Procurement Specialist as Procurement Team Lead. Winrock HQ ▲ ▲ R1.3 At the start of the Activity implementation, discuss with the USAID the type of grants which should be disbursed by the Activity (to avoid reliance on one type of grant) as well as the timeline implications for each type of grant. Winrock HQ and USAID ▲ ▲ R1.4 To ensure selection of proper local vendors, include the following requirements in the selection criteria: 1) prior vendor experience in delivery of the similar type of equipment as per the RFQ; 2) provision of maintenance services for a year and availability of experienced maintenance engineers and technicians; and 3) existence of vendors’ own warehouses to store equipment. Moreover, to safeguard the receipt of expressions of interest from qualified local vendors, hold information sessions with a pool of potential vendors to explain the Activity’s equipment requirements and needs. Winrock CO ▲ ▲ R1.5 To streamline the grant application process, from Activity start, set concrete targets and milestones on the number and type of enterprises to be supported through grants per value chain. Winrock HQ and CO ▲ ▲ R1.6 Budget the PIF expenditures (i.e., grants to enterprises) starting from mid-Year 2 to Year 3 of the Activity implementation as adequate time (1-1.5 years) is needed to set up the Activity and develop and operationalize the grant mechanism processes. Winrock HQ ▲ ▲ R1.7 To avoid delays with procurement, ensure that grant applicants’ selection and procurement happen in parallel tracks, i.e., the procurement process starts at the concept note stage rather than after the grant package is finalized which will save 3-6 months on the procurement process timeline. Winrock HQ and CO ▲ ▲ 16 # Recommendation Addressed to Timeframe Priority S/T L/T L M H R1.8 To ensure the selection of the right grantees (i.e., committed enterprises), at the grant application review stage, conduct interviews with short-listed grant applicants. This will enable the Activity to better determine the potential of the enterprise to implement the proposed activities and its motivation and level of interest in strengthening its businesses. Winrock CO ▲ ▲ R1.9 From the start of the Activity, develop the unified PIF database which would combine the information on grant application processes, procurement, and grant agreements, including all planned targets disaggregated by grantee. This will ensure a proper tracking system is in place and enable the Activity to obtain accurate information promptly from all relevant departments (PIF, Procurement, Finance, Operations and Admin, Technical, and M&E). Winrock CO ▲ ▲ 4.2 EQ 2: WHAT IMPROVEMENTS TO THE MANAGEMENT STRUCTURE MIGHT BE NEEDED TO ENSURE THAT ACAT ACHIEVES THE ACTIVITY GOAL AND OBJECTIVES? 4.2.1 Findings As outlined in the ACAT performance work statement, the Activity’s organizational structure consisted of a team composed of 23 staff members, including three key personnel. The team was intended to be located in two offices—one in Dushanbe and the other in Bokhtar—with 60 percent of staff located in the capital, including all leadership and management staff, and the other 40 percent, the technical team and some support staff, in the field. The evaluation interviews (KII=17) revealed that the initial organizational structure of the project was not fully efficient. The ET identified several gaps in the original ACAT management structure: a. Wrong organization of the management in Winrock Country Office: The original COP, based in Dushanbe, focused more on overall and financial management and was not involved in technical issues. However, the Deputy Chief of Party (DCOP), designated to manage technical issues, was also based in Dushanbe rather than Bokhtar, the location of the Activity’s target areas. As a result, the Bokhtar team did not have a manager and the leadership and management did not provide them with clear guidelines and framework for the implementation of activities. This created a chaotic and uncomfortable atmosphere and resulted in high staff turnover. In total, 35 percent of staff (nine out of 26 staff members) left ACAT between August 2018-January 2022. b. Absence of understanding of the market system approach by ACAT staff: The market system approach is a very new concept for Tajikistan. In KIIs with ACAT staff and stakeholders (N=18), participants repeatedly mentioned that at the start of implementation local staff knew little about the market system approach and the previous leadership did not provide sufficient orientation and guidance on the approach and how it should be applied in the context of agribusiness in Tajikistan. As a result, as implementation started, ACAT did not follow the market system approach. c. Selection of staff based on the previous experience with the USAID rather than on technical skills: Winrock HQ emphasized hiring staff for ACAT based on prior experience with the USAID projects and knowledge of USAID procedures. As a result, Winrock hired 50 percent of ACAT staff from the previous USAID Tajikistan Agriculture and Water Activity (TAWA), which had more of a production focus. Early on, this created an implementation challenge. Staff knew the 17 players in Khatlon and how to work in production; and they tended to propose and plan activities that focused on production (i.e., supporting farmers to produce more through purchase of agricultural equipment) rather than competitiveness (i.e., supporting processing, sales, and export of products produced by farmers to the new markets). d. Lack of attention on the importance of leadership skills: Innovative projects like ACAT require a strong, vocal leader on the ground who is able to set and clarify a vision; engage, orient, and capacitate the staff on competitiveness and market systems; and shift the focus away from production towards competitiveness. The initial ACAT COP had strong technical expertise but lacked the necessary leadership skills to manage this particular project. He did not follow up on tasks in a timely manner and did not provide detailed guidance to the team. e. Insufficient budgeting of staff for the volume of work to be done: ACAT is a very technical project with a large grant component ($3.7 million) and in the view of interviewed stakeholders (N=15) required more staff in Procurement and Market Systems to ensure smooth implementation. f. Lack of communication and coordination between different ACAT’s departments: ACAT’s MEL Department worked without technical guidance from the COP, DCOP, and technical experts and with little to no communication between them, as the evaluation interviews reveal (N=10). MEL staff counted the indicators without consulting the technical team. The MEL, Market Systems, and Communications teams also failed to coordinate site visits to grantees. Review of ACAT’s M&E records showed that the M&E Department did not develop a consolidated dataset to track the performance of each grantee against the roadmap targets set in grant agreements. The MEL team also had challenges tracking and reporting on the “Value of new USG commitments and private sector investment leveraged by the USG to support food security and nutrition” (EG.3.1-14). Some grantees found it difficult to provide documentation (e.g., checks, invoices) to verify their cost-share contribution (e.g., repair or construction of the building). Moreover, due to the insufficient coordination between MEL and PIF teams, post-implementation monitoring of ACAT grantees is not feasible as this provision was not included in the grant agreements. g. Lack of detailed supervision by Winrock HQ of country office: Although ACAT faced a number of challenges and had substantial delays in issuing grants and procuring equipment, important changes in ACAT’s management structure only occurred at the start of the third year of implementation with arrival of the new COP and functional analysis of ACAT’s staff roles. h. Insufficient attention to the country context and workplace organization: In several KIIs with ACAT staff (N=4), participants mentioned that several staff members left the project citing lack of comfort with the open space work environment (uncommon in Tajik culture), which did not allow them to concentrate and work efficiently. Following the resignation of the original ACAT COP in January 2021 and subsequent appointment of his replacement in February 2021, ACAT reviewed each staff member’s roles and responsibilities. As a result, the Activity made a number of organizational changes and recruited an international procurement expert, with the goal of strengthening the capacity of ACAT technical team members and BDSPs. Management recognized the need for more of a presence in the field and the DCOP moved to Bokhtar. As of the end of 2021, 46 percent of ACAT staff are based in Bokhtar and the rest (54 percent) in Dushanbe. ACAT also hired a new procurement specialist, who serves as a bridge between the Technical and Grant and Procurement teams and supports the translation of documents, research (verification of quotes), and other related tasks. ACAT’s experience indicates that the organizational structure needs to be very dynamic, enabling reshuffling of the staff as needed, because the focus of the Activity changes from year to year. Moreover, after the arrival of the new COP, Winrock HQ became more involved and began weekly calls with the ACAT COP and provide the support as needed. The Winrock country team also mobilized a 18 home office senior advisor to develop a formal capacity development program for ACAT staff. Coordination among ACAT departments has also improved through regular weekly staff meetings and senior staff meetings. In October 2021, ACAT also launched Quarterly Partner Enterprise Meetings (QPMs) to promote professional networking and peer learning with the aim of strengthening support for product promotion, aggregation, market access, and business networking within value chains. In addition, it is important to highlight that ACAT was able to establish close and effective collaboration with other USAID projects as is evident from interviews with ACAT partners and stakeholders (N=10). For example, ACAT has collaborated with 1) the USAID Competitiveness, Trade, and Jobs (CTJ) Activity on export of Tajik watermelons and strawberries to Kazakhstan; 2) the USAID Agriculture, Land, and Governance (ALG) Activity on women and youth inclusion and provision of support to partners; and 3) the USAID THRIVE Project on provision of small loans ($10,000-$15,000) to some ACAT grantees for further expansion of their businesses. Also, USAID CTJ and ALG provided co-funding for the business fora organized by ACAT. During interviews, KII participants (N=8) mentioned other challenges including the internal procedures, particularly Winrock’s highly centralized system for recruitment and procurement. ACAT experienced problems with recruitment of personnel under the previous COP. For example, the recruitment process for some staff positions lasted for a year as a result of HR being managed from HQ without a solid understanding of the local context. The mini-survey results among ACAT grantees and beneficiaries (Figure 8) reveal respondents’ general satisfaction of the communication between ACAT’s direct grant beneficiaries and the staff with whom they dealt. Maintaining this mutual collaboration as the grant component comes to a close is key in terms of the procurement process and delivery and installation of machinery and equipment, as agreed to under individual and collective contracts. Figure 8: Grantees’ Assessment of the Level of ACAT’s Team Communication and Responsiveness 11% 10% 61% 59% 28% 31% 0% 20% 40% 60% 80% 100% Communications/coordination between ACAT and grantees (N=90) Flexibility of ACAT staff in responding to emerging needs throughout the implementation (N=87) Poor Average Good Excellent Source: Mini-surveys among ACAT grantees and beneficiaries, November 2021 Table 5 summarizes KII responses regarding respondents’ attitudes toward ACAT personnel with whom they had direct or indirect communication or collaboration, although these do not identify individual ACAT positions. Participants’ views on ACAT’s interventions are also included as, by extension, they imply the effectiveness of key program staff in delivering results. 19 Table 5: Summary of KII Responses – EQ 2 KIIs EQ 2: ACAT’s management structure to achieve the activity goal and objectives Responsive Communication/Coordination Between ACAT and Partner Positive or Meaningful Collaboration with ACAT Staff ACAT’s Interventions Positively Contribute to Agribusiness Development Fully To Some Extent Fully To Some Extent Fully Unsure BDSPs 3 3 3 MFIs 5 5 5 Business associations 3 3 1 2 NGOs 4 4 3 Government group 7 2 5 2 3 Total 11 11 10 12 11 10 Comments from the various KIIs indicate that participants view ACAT as generally supportive toward addressing the needs of its partners. However, ACAT’s contribution to agribusiness development in Khatlon region was less recognized; this represents an opportunity for the Activity to increase its visibility to demonstrate and share its contributions more effectively with partners. 4.2.2 Conclusions # Conclusion C2.1 From a shaky beginning to Year 3, the ACAT team structure has solidified, with positions strengthened where required, additional appointments made, and roles and responsibilities clarified in line with the tasks to be carried out. The current management structure seems to be adequate to carry out the Activity’s remaining tasks. Although key informants had little knowledge or understanding of these internal staffing arrangements, the ET concluded respondents’ comments, observations, and perceptions suggest there is an acceptable level of goodwill between those who have come in contact with ACAT staff, directly or indirectly, and their respective organizations. C2.2 This goodwill is expected to continue. It is likely to be further enhanced with the planned activities and interventions announced in ACAT’s Year 4 Work Plan. In addition, the COP indicated that ACAT will make significant progress in the provision and level of technical support for beneficiaries to improve operating performance and profitability. 20 4.2.3 Recommendations # Recommendation Addressed to Timeframe Priority S/T L/T L M H Recommendations for the current program R2.1 Continue to review and revise the Activity’s organizational chart annually and reorganize staff as needed. Given that delivery of equipment to the grantees is nearly complete and ACAT will concentrate primarily on provision of technical assistance for the reminder of the Activity, consider shifting the M&E Director and Social Inclusion and Communication Specialist to the Bokhtar office. This would enable ACAT to better monitor grantees’ performance and to collect and document the successes and lessons learned from the model piloted by the Activity. Winrock HQ and CO ▲ ▲ R2.2 Consider decentralization of the HR function to the Winrock Country Office to save time on recruitment. To achieve this, give the COP the right to recruit consultants if they are budgeted in the ACAT Work Plan. Recruitment should continue to be conducted by Winrock HQ for those positions which are not budgeted in the Activity’s budget. Winrock HQ ▲ ▲ R2.3 Strengthen ACAT’s M&E system by developing the database of all supported grantees. This will enable the Activity to track grantees’ performance against the grant agreement obligations and roadmap targets. Winrock HQ and CO ▲ ▲ R2.4 Consider allowing the ACAT to provide additional sources of evidence on the indicator EG.3.1-14: (“Value of new USG commitments and private sector investment leveraged by the USG to support food security and nutrition) in addition to MSME financial records, in particular photos and/or engineers’ conclusions/reports on the estimated costs of grantees’ constructed/ upgraded buildings. USAID ▲ ▲ R2.5 Pay more attention to ACAT staff teambuilding in both the Dushanbe and Bokhtar offices through staff retreats or trainings to upgrade staff skills. Winrock CO ▲ ▲ Recommendations for future programming R2.6 Adapt ACAT’s organizational structure in proportion to the size and complexity of the intervention, especially when it comes to the Procurement and Technical teams. Winrock HQ ▲ ▲ R2.7 When selecting staff for future activities in Tajikistan, emphasize candidates’ technical skills over their knowledge of and prior experience with USAID procedures and interventions. This would expand the pool of competent specialists. Also, when selecting COPs for innovative Activities like ACAT, give higher priority to candidates who have—in addition to technical expertise—strong leadership skills, prior experience in target country or region, in-depth knowledge of grant mechanisms, risk management skills, and experience building staff capacity in the technical aspects of the intervention. Winrock HQ and USAID ▲ ▲ 21 # Recommendation Addressed to Timeframe Priority S/T L/T L M H R2.8 For Activities which implement outside of the capital, like ACAT, locate the country office (including leadership, management team, and specialists) in the area which is covered by the intervention. To encourage staff to relocate to the field, consider providing incentives (e.g., housing and travel allowances) for both international and local staff. Winrock HQ and USAID ▲ ▲ R2.9 To ensure proper coordination among Activity departments (PIF, Procurement, Finance, Operations and Admin, Technical, M&E), develop a general guide to coordinate the work of all departments. As roadmap, this guide should show how the grant mechanism processes are organized and the timeline for each step—from the concept note to payment. Winrock HQ ▲ ▲ R2.10 Consider the country context when selecting office space and workplace structure. In Tajikistan, employees are used to a traditional office design with enclosed work areas rather than an open space design. Considering, workplace culture is important because it links company culture with things like employee engagement, happiness, productivity, retention rate, and positive recruitment efforts. Winrock HQ ▲ ▲ R2.11 In grant agreements, include the provision to conduct post-implementation monitoring of the supported businesses to ensure the Activity is able to measure the impact of specific investments and interventions on economic development, employment, and livelihoods. Winrock HQ ▲ ▲ 4.3 EQ 3: MIDWAY, WHAT ABOUT ACAT’S MARKET SYSTEM APPROACH TO IMPROVING AGRIBUSINESS COMPETITIVENESS IN TAJIKISTAN IS WORKING WELL AND WHAT IS NOT? 4.3.1 Findings ACAT’s market system approach as defined in the ACAT Contract essentially mandates Winrock to implement its activities via a market-driven, private-sector-led approach aimed at stimulating economic growth, increasing employment, and improving the livelihoods of its beneficiary agribusiness enterprises by increasing their competitiveness. It will do this in line with the program’s three objectives as elaborated in ACAT’s SOW. Project Design The document review and interviews with ACAT partners, stakeholders, and beneficiaries (N=42) indicate that the overall design of the Activity is quite broad and has ambitious objectives and targets without sufficient country context analysis. ACAT adopts a market-segmentation approach in the implementation of its activities to capitalize on potential market opportunities more effectively. While this approach contributes to improving market system efficiency, increasing the potential for investment and lowering overall operating costs, ACAT’s mandate also includes additional and complementary program activities that support the agribusiness ecosystem in Khatlon by helping develop essential linkages between aggregators, wholesalers and exporters, and local, regional, and international markets. 22 Overall, the Activity’s objectives are complementary but require skilled staff to navigate them properly as Objective 1 and Objective 2 activities are closely linked. Objective 1 focuses on working with enterprises to improve their technological foundation by providing equipment, trainings, and coaching on the technical and technological methods of production. Objective 2 concentrates on linking the enterprises to the market by organizing business-to-business (B2B) meetings, fora, and marketing to enable enterprises to sell the products produced. The initial design of the Activity was logical and was based on the market system approach; however, in the course of the implementation, ACAT’s approach became more modified. This is reflected in the provision of agribusiness equipment, which cannot be described as a market system approach. Rather, the approach consists of relatively direct implementation and provision of grants. The original ACAT design also included components to support the service sector (e.g., BDSPs, KBA, and agro-logistic centers) to provide services to the agribusinesses in the long run, which align more closely with a market system approach. ACAT’s overconcentration on equipment grants and transferring equipment to agribusinesses in the first three years of implementation resulted from the initial project design’s lack of criteria and targets for selecting value chains in the target sectors. As a result, ACAT selected 163 grantees in dairy and horticulture sectors in the Activity’s Zone of Influence (ZOI). In total, ACAT supported 24 grantees in the dairy value chain, 30 nut processing enterprises, 17 horticultural enterprises, and 92 individuals with post-harvest handling (PHH) tools. Figure 9: ACAT Grantees by Value Chain (N=163) F&V (N=109) Dairy (N=24) Nuts (N=30) 84% 8% 3%3%2% PHH Tools F&V Drying and Processing Packaging production Agri-Logistic Centers Canning production 37% 13% 13% 4% 33% MCCP - Milk Aggregation and processing Dairy processing Dairy products Ice cream Qurut Production 77% 3% 20% Nuts processing Peanut Processing Pistachio Processing Source: PIF Dataset The PIF records indicate (Figure 10) that 48 percent of grants were small, up to a maximum of $10,000, 23 percent between $10,000-$40,000, 17 percent between $40,000-$90,000, 11 percent between $90,000-$140,000, and 1 percent over $170,000. The in-kind grants for nut processing enterprises were 23 also small, mainly up to a maximum of $10,000; half of the in-kind grants for dairy were between $10,000- $40,000 and 47 percent of grants in horticulture were between $20,000-90,000. Figure 10: Share of Grantees Per Value Chain by Budget Category (N=71) 24% 17% 87% 6% 33% 3% 12% 17% 18% 18% 17% 7% 12% 4% 3% 12% 8% 4% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% F&V Dairy Nuts up to $10,000 $10,000-$20,000 $20,000-$40,000 $40,000-$70,000 $70,000-$90,000 $90,000-$120,000 $120,000-$140,000 over $170,000 Note: The disaggregation under F&V value chain does not include 92 PHH tools. Source: PIF Dataset Recognizing that the market systems approach utilized did not produce systematic change, ACAT has shifted its approach in the fourth year of implementation to the lead firms model. Over the last year, the Activity has been in transition. The ACAT team is working with all enterprises which received equipment grants to identify which of these firms can produce long-term impact and serve as lead firms and how ACAT can invest in those firms to ensure they succeed and become the business leaders—which would lift the rest of the sector as well. Although this approach is appropriate and correct, it would be more sustainable if the lead firms would facilitate the value chains themselves, rather than by ACAT as is done currently; this shift would enable lead firms to feel more ownership over the change process. Experience has shown that, unless value chain actors take responsibility and ownership of the management and operations of the competitiveness process, long-term success is unlikely. ACAT’s design also covers both competitiveness and employment creation under the framework of one Activity. It aims to: facilitate a 25 percent increase in sales among project participants, create 18,112 jobs, and leverage $5 million in investment. The private sector plays an important role in contributing to the employment creation and poverty reduction. However, creation of jobs is very a complex task and the logic between investment in companies’ growth for improved competitiveness and job creation is not always linear. There is also a long lag time between when a company grows (by adding a new product or reaching a new market) and when it has enough experience to start adding new jobs. Additionally, in most cases, equipment makes the work more efficient through automatization, resulting in the company needing a smaller labor force. Seen in this context, the Activity’s job creation target was ambitious. Furthermore, ACAT’s job creation efforts focus on only one region of Tajikistan, which makes the job creation target even more challenging. The market system would be more effective if there will be nationwide coverage. ACAT’s focus on improving sales both domestically and internationally was also ambitious given the location of the Khatlon region, the level of development of the private sector in the region, and the region’s limited opportunity to sell abroad. The Khatlon region is very limited in terms of market access and sales as the majority of products are sold locally or, secondarily, purchased by businesses from other 24 regions of Tajikistan (particularly the Sughd region). Therefore, being able to compete effectively for the local market is the starting point, and then the export market. Business Development Service Providers In terms of BDSPs, ACAT focuses on expanding services provided by BDSPs to deliver training, advisory services, coaching and mentoring, extension services, demonstrations, and information. This is done through grants and TA to local service providers (NGOs, associations, and individual consultants) to adopt private sector-oriented approaches to coaching and mentoring and by providing trainings and consultations to target enterprises on topics including production management, marketing, budgeting, financial management, planning, and financial forecasting. ACAT partnered with local service providers (PO Nuri Khatlon, PO Sarchashmai Hayot, PO Taqviyat va Tavonmandi, and PO Rushdi Diyor) to address gaps in the post-harvest stone fruit handling. The purpose of these trainings was to introduce higher harvest standards and practices that enabled women and youth labor groups to offer improved, fee-based pre- and post-harvest activities. The project involved short￾term technical assistance (STTA) to support enterprise certification, build the capacity of processors in both the dairy and horticulture sectors, and improve technological processes in target value chains. In addition, ACAT supported the establishment of the KBA. ACAT is building the business capacity of KBA and its members by supporting registration, access to market information, export promotion, production of high-value and value-added products to meet market requirements, and implementation of initiatives to grow professionally in the sector, including certification and traceability. From KIIs with stakeholders (ACAT, USAID, and international donor organizations, N=10), at the stage of the project design, Winrock did not adequately consider the level of development of the service providers in Khatlon region. When implementation started, it became apparent that the BDSPs’ market was not developed in the south in Tajikistan. NGOs, which are more donor-driven and provide theoretical, not practical, trainings, were the only active players. These NGOs lack practical experience in the private sector and do not understand the market system and value chain concepts. BDSPs are developed only in the north in the Sughd region. However, they are neither prepared nor willing to work with small enterprises in the south. In the south, businesses are on a lower level of development and have different needs compared to those in north; therefore, BDSPs from the north cannot advise those in the south effectively. To resolve those challenges, the ACAT team took a new approach; it started to attract consultants from the private sector (both national and international) with solid practical experience in agribusiness development to build the capacity of ACAT grantees. Access to Finance ACAT plans to improve increase access to finance in the horticulture and dairy value chains through 1) provision of targeted loan products and lower interest rates, 2) increasing MSME awareness of access to credit opportunities, and 3) building the capacity of FIs to better serve MSMEs. To develop a relevant financial inclusion strategy, ACAT conducted an Access to Finance Assessment in November 2018, which analyzed the demand- and supply-side constraints impeding various market system actors from accessing finance. Through a competitive process, ACAT has selected four local MFIs (Microcredit Fund [MCF] Sarvati Vakhsh, Microcredit Organization [MCO] Mehnatobod, MCF Chiluchor Chashma, MCF Imkoniyat)13 to offer specialized loans for small enterprises willing to borrow to invest in new or expanded businesses. The loan duration is for up to 24 months at an interest rate of 16-18 percent with a grace period of up to 9 months, depending on the sector (dairy or horticulture). ACAT has provided technical support. 13 Note: ACAT signed the grant agreements with MCF Sarvati Vakhsh and MCO Mehnatobod in October 2019, and with MCF Chiluchor Chashma and MCF Imkoniyat in June 2020. 25 As of December 2021, 392 new loans with a total value of $657,515 were issued through ACAT’s partnership with FIs with a late payment rate of approximately 3 percent and a 100 percent repayment rate. Loans were used for greenhouse construction, the purchase of milking cows, and small-scale dairy and horticulture processing. These loans leveraged $146,846 in private sector investment and generated $429,252 in additional sales for MSME borrowers after loan repayment. ACAT-supported loan products have also created new jobs for 581 workers (346 female, 191 under the age of 29) since the beginning of the Activity. Small enterprise owners from the loan recipient FGD (four female, four male) reported that they received the loans from ACAT-contracted MFIs in the USAID ZOI for extension of their business, mainly canning vegetables, greenhouses, and cake and ice-cream making. They mentioned that receiving loans from MFIs has improved because they have simplified the process of landing loans and the MFIs are geographically close to the beneficiaries. They said they have benefited from the 18 percent subsidiary rate and will continue expansion of their businesses. “I received 10,000 TJS from MFI Chiluchor Chashma for one year at the subsidiary rate of 18 percent, preserved 2,000 jars (2 liters) and sold 1,800 jars per 15 TJS and got 27,000 TJS. After two months I returned the received loan to the bank and [with] the income I paid the tuition fee of my daughter.” – Female, Loan Recipient FGD, Shahrituz district In the course of mid-term evaluation, the surveyed loan recipients indicated their main purposes for receiving the loans were expansion of the current business (44 percent of entrepreneurs) or modernization of the business (20 percent). Outcomes listed in Figure 11 could be used as actual achievements resulting from ACAT-supported loans, i.e., 73 percent of loan recipients experienced production growth, 66 percent experienced sales growth, and 64 percent experienced profit growth, and 68 percent reported an increase in the number of new clients. On average, production increased by 28 percent, sales by 25 percent, and profit by 23 percent, and the average number of new customers was 31 people per enterprise. Figure 11: Assessment of the Impact of the Loans for Business (N=77) Source: Mini-Survey among loan recipients, November 2021 Overall, most of surveyed loan recipients believed that the loan product developed and provided by the ACAT-supported MFIs was beneficial. However, during interviews (KII=9), some enterprises indicated that the loan amount is quite small and the duration is rather short; meanwhile, the interviewed MFIs (KII=4) mentioned that ACAT’s grant amount to MFIs (just up to $100,000) is not big given the high demand for low-interest loans from value chain actors and businesses from other ZOI districts. At the same time, interviewed MFIs emphasized that collaboration with ACAT is very valuable as, in contrast to banks, Tajik MFIs have issues attracting resources as the development banks like EBRD or the World Bank 26 can invest only if their portfolio is $1.5 million. Due to constant inflation, MFIs are not able to meet this ceiling and start collaboration with development banks. The document review, interviews, and mini-survey results illustrate that ACAT’s approach to the access to finance component benefits recipients (vulnerable and small-scale enterprises) as they receive access to funding at lower interest rates (16-18 percent), but it does not create systemic change, i.e., at the FI level. This approach is not also fully sustainable as it only offers temporary solutions because the FIs most likely would continue the rate in the longer run, as market interest rates for loans run between 25- 40 percent depending on the type of FI (commercial bank or MFI). Women and Youth In terms of inclusion, ACAT aims to ensure that women and youth (15-29 years old) are meaningfully engaged within target value chains (dairy and horticulture), creating profitable business opportunities and facilitating access to finance to stimulate investment in on- and off-farm enterprises. ACAT developed a Gender and Youth Inclusion Strategy in August 2018 based on a rapid gender and youth inclusion analysis conducted by MCI to ensure more meaningful participation of women and youth within the Khatlon agribusiness sector. It was planned to be achieved by targeting women- and youth-led firms for TA, supporting workforce development, facilitating study tours for women and youth to the Sughd region, and working with the education and TVET system to provide high-quality training in agriculture. ACAT chose the right approach for designing both mainstreaming and targeting activities that benefit women and youth in addition to the inclusion targets for women and youth for ACAT interventions. The FGD with young female entrepreneurs indicated that ACAT helped extend their businesses through the establishment of a milk producers’ women’s group linked to the MCCP. KIIs with internship program participants showed that such interventions are effective and enable youth to gain the real work experience. ACAT’s work with women and youth groups in skilled commercial services (pruning, grafting, postharvest handling) was also highly relevant. The market was strong and favored local workers; therefore, it became a good entry point into the labor market for women and youth. Loans to milk producers through ACAT-supported MFIs allowed community to recognize better the role played by women in this value chain. The only inclusion challenge noted by the ET was the lack of integrated approach between Winrock and MCI activities in the first two years of implementation. MCI was trying to work with youth and women not connected to ACAT’s enterprises and value chains. This approach was revised in the third year of implementation through the Women and Youth Inclusion Audit, complete revision of the MCI work plan, MCI’s hiring of a long-term consultant to provide on-the-ground implementation support, and the ACAT Communication and Social Inclusion Specialist’s assignment to manage the contract with MCI. As of December 2021, ACAT has achieved the following key results under its three objectives: • ACAT trained a total of 20,806 individuals and 136 enterprises to perform improved production or business practices. • ACAT supported the development of 62 new high-value and added-value products and six new services to value chains, including milk aggregation and cooling; vegetable aggregation, sorting, cooling, packing, and selling; financial services; quality assurance enhancement through training; certification preparation; and box/packaging services. • Sixty-six (66) ACAT-supported firms have met regional and international trade standards and certification requirements. • As a result of ACAT interventions, 9,363 individuals now have full-time, part-time or seasonal work, and 40 percent the positions were filled by women. • Annual sales by firms (MSMEs and BDSPs) receiving ACAT’s assistance totaled $643,900. • Through 179 grants and five subcontracts, ACAT invested an aggregated value of $3,105,453 in improved systems, new technologies, or increased technical capacity. 27 • ACAT facilitated the formation of 50 public-private partnerships (PPPs). The review of ACAT’s PMP (see Annex 5) demonstrates that the Activity has already overachieved 35 percent of the end-of-project targets, while others are in progress of their achievement. The level of awareness about ACAT and its activities is quite high among different categories of stakeholders interviewed with the exception of local/district authorities. Table 6: Summary of Findings from Direct KIIs (Stakeholders) – EQ 3 KIIs Are you aware of ACAT and its activities in Khatlon? Group Yes No Central government (Dushanbe) 1 Local/district authorities (Khatlon Region) 4 3 Women/youth associations 1 NGOs 3 International donors/organizations 4 1 Business associations 4 BDSPs 3 Total 20 4 In FGDs questions related to EQ 3 focused on participants’ satisfaction levels with ACAT’s interventions, and whether they had an effect or impact on their respective businesses. These were categorized as high, medium, and low. None of the participants reported a high level of satisfaction. With the exception of horticulture and PHH, five of the seven groups reported medium satisfaction with the in-kind grants. Milk suppliers were the least happy with TA leading to productivity improvement, and more than half were unhappy with sales being increased through the development of linkages (see Table 7). Table 7: Grant Status by FGD by Value Chain – EQ 3 # Category of FGD by Value Chain EQ 3: ACAT’s Market System Approach – What’s Working Well and What Isn’t? ACAT Interventions In-Kind Grants Machinery/ Equipment – Delivery & Installation TA & Capacity-Building – Productivity Improved Developing Linkages – Domestic & International Sales Increased High Medium Low High Medium Low High Medium Low 1 Milk suppliers 11 11 2 Loan recipients 8 8 8 3 BSDPs 6 6 6 4 Horticulture 8 8 8 5 Dairy value chain 8 8 8 6 PHH 8 8 8 7 Enterprises selected – no grants yet 8 8 8 Total - 30 16 - 22 35 - 25 32 Dairy – Milk Suppliers Milk and dairy products are one of the primary sources of income for FGD participants. All 11 FGD participants were women who received ACAT-contracted partner/service provider training sessions on 28 livestock management, feeding, and hygiene. They did not receive grants. During discussions, milk suppliers indicated the training sessions were useful and that, to some extent, they were able to adopt what they learned. Their main challenge was how to get connected to a stable market with fewer losses. The milk suppliers used to make “qurut” and sour butter (chakka) to sell in the district market, which is located 7- 10 kilometers (km) away. “We used to pay 8 TJS (approx. $0.80) to take the milk and other dairy products to the district central market to sell, but after this milk collection center was established and started operation it saved our time and financial resources.” – FGD participant, female, milk supplier, Yovon District. “We couldn’t take the milk and other dairy products to the district central market to sell on a daily basis because it couldn’t cover all the travel expenses and we had to collect two days’ milk, which sometimes resulted in the milk turning sour or spoilt.” – FGD participant, female, milk supplier, Yovon District. Milk producers have almost no pasture, and lack of irrigation water prevents them from improving the feeding ration of their dairy cows. Milk suppliers are afraid to get loans to develop their businesses because of the unavailability of pasture for their cows. Within the available and accessible opportunities/resources, they manage the livestock and get 2-4 liters of milk per cow per day. All FGD participants stated the creation of the milk collection centers (MCCs) was an important and timely initiative, which succeeded in establishing an important linkage between milk producers and the dairy sector. The MCCs give milk producers easier access to the market. The FGD with milk suppliers falls in Dairy Group 1. Eleven (11) women participated in this FGD and described the training sessions provided by the ACAT-contracted local service provider on feeding, hygiene, veterinarian service, and livestock management as useful. However, the majority of FGD participants also mentioned that the training sessions are not enough to improve the milk quality and productivity. Dairy Enterprises ACAT established and equipped MCCs and milk processing enterprises with up-to-date equipment. Within almost one year of operation, the MCCs were able to establish a linkage with the milk producers, who are basically household women and milk processing enterprises. As one of the MCCs’ owners said: “The selling price in summer is 2.00 TJS (approx. $0.18) per 1 liter and 2.20 – 2.50TJS (approx. $0.19–$0.22) in winter.” He stated the collected milk is partially processed within the MCC, and the rest is sold to the dairy enterprises during summer but, in winter, the whole collected milk is sold to them. The dairy enterprises, which received equipment and TA, agreed ACAT’s financial support was necessary for the development of the dairy sector in their districts. In fact, the limited liability corporation (LLC) Tijorati Sorbon, which is linked with one of ACAT’s established MCCs, produces a variety of dairy products, including pasteurized milk, sour cream, and butter. Tijorati Sorbon also installed booths in the district where only its own dairy products are sold. All eight FGD participants mentioned they received training sessions from the ACAT-contracted partners and applied the knowledge they received in their businesses. FGD participants mentioned that despite ACAT’s provision of equipment for full operation of their businesses, the dairy enterprises still needed some equipment for processing lines. They pointed out this issue was discussed with ACAT during the contract stage, but for some reason was never acted upon. One MCC owner, who has worked in Russia before and knows the business very well, mentioned that instead of a 1,000-liter capacity tank for milk storage, ACAT could instead buy two 500-liter capacity tanks. This is because one tank is for collecting morning milk and the other one is for evening milk as mixing morning and evening collected milk would affect its quality. In principle, participants indicated the MCCs operated in a supportive way, covering more milk suppliers across the districts where they are located. Sometimes, however, they malfunction due to the lack of independent power supplies. The ET 29 spoke with the ACAT dairy specialist, who mentioned milk suppliers requested bucket and milk separators but later in the process ACAT decided not to provide them because of the large number of beneficiaries and the cost involved. Many of the grantees also requested generators, which were also later removed from the requested list. ACAT promised to provide tanks for milk transportation, however, if the MCCs could get motorcycles themselves. Horticulture – Post-Harvest Handling A significant finding indicated most of the PHH FGD participants were the beneficiaries of the previous USAID TAWA project, which was implemented by Chemonics International, and already had some experience in pruning and grafting. ACAT provided tools such as secators, diesel sprayers, and 2-meter length diesel saws. The Activity also provided orchard management tool training sessions. Expansion of small businesses through capacity-building increased the skills and knowledge of some individuals on pruning and grafting, largely women beneficiaries. However, FGD participants informed the ET that some recipients of pruning and grafting tools have not been able to use those tools, either because they had no skills or they received tools purely by chance and have nothing to do with the project. Also, as noted by the Agribusiness Value Chain Specialist, ACAT conducted training sessions on pruning and grafting before distribution of the tools. The main constraints for the region’s horticulture sector businesses were lack of knowledge and experience on horticulture management, especially integrated pest management (IPM), fertilizers and pruning, and nurseries for fruit trees seedling (grafting apricot, peach, and plum). Participants explained ACAT was unable to meet their needs in improving the quality and increasing the yield (through complete renovation of existing orchards) and marketing (export) through diversification or upgrading orchards. To develop and/or strengthen the horticulture sector, an agriculture expert on IMP, fertilizer management, and grafting is needed. Importing rootstock and stone fruit seeds from developed countries would also be effective. Horticulture Enterprises Findings from the horticulture enterprises FGD indicated four out of eight participants signed contracts to receive equipment for horticulture business development; the rest either did not sign a contract or are waiting for contracts to be signed. The four participants who are expecting equipment from ACAT already have operational businesses and ACAT’s support will help extend them further. Although they have skilled employees, these participants said they need ACAT’s support to operate the equipment in the form of TA regarding professional processing technologies, e.g., a highly qualified technologist who will teach and transfer new technologies to diversify products and the proper maintenance of equipment. In order to improve the horticulture sector, stone fruit farmers particularly need the best grafting inputs (rootstock and seeds of stone fruit from developed countries), which will improve the quality of fruit seedlings. Business Development Service Providers Participants in the BDSP FGD stated they are public organizations and fully depend on donor-funded projects. They have a long history in the region with the support of international communities. The BDSPs’ employees are not specialized in the agribusiness sector, although geographically they are closer to the beneficiaries and understand what is needed to improve the agribusiness sector. In order to improve the efficiency of the BDSPs’ work, providers need to identify the demands of local small entrepreneurs first and develop appropriate plans and actions. BDSP technical staff need to be trained and provided with TA. During the FGD discussion, the ET found BDSP staff are still not active in the development of SMEs in the dairy and horticulture sectors, along with the market system demand in the region. BDSPs should act as engines of progress for SMEs in the agribusiness marketing system. The FGD with partners, who have received various capacity-building programs but not (as of yet) grants, falls under the Horticulture Group II. All eight FGD participants (six male, two female) agreed that distributed tools for orchard management were necessary for them to properly manage the pruning and 30 grafting processes. Most pointed out that regular visits from horticulture experts would add value to their work. One mentioned: “Previously I was involved in grafting but due to the ACAT intervention I learned new grafting technologies, which made me more famous as an experienced grafter among the orchard farmers in the region.” – FGD, Horticulture Group II, female, Balkhi district It should be noted that almost all of these participants were beneficiaries of the previous USAID TAWA Project. The results of the mini-survey among ACAT grantees and beneficiaries (Figure 12) indicate that ACAT interventions contributes toward overall market competitiveness in Tajikistan, and particularly in Khatlon. Figure 12: Perception of Grantees on the Main Outcomes Achieved Thus Far as a Result of Participation in ACAT (N=92) Source: Mini-Survey among ACAT grantees and beneficiaries, November 2021 Overall, ACAT is successful in development of businesses of the participating enterprises: • 70% of participants increased overall production volume as a result of involvement in ACAT. • 65% of participants increased overall business sales as a result of participation. • 59% of participants found new business or trade partners. • 59% of participants invested additional money into their businesses. • 53% of participants either started exporting goods/services or increased export sales. • 38% of participants have developed new products or services. • 20% of participants have established completely new businesses. • 19% of participants hired new staff as a result of participation in the ACAT. 31 4.3.2 Conclusions # Conclusion C3.1 ACAT should be seen as a complex intervention with ambitious targets; apart from strengthening competitiveness of the agribusiness using a market-driven approach, it also has a social responsibility in terms of jobs creation. This is difficult to achieve when the Activity focuses on only one region of the country and has a limited timeframe. The value chains selected were appropriate, but ACAT did not implement activities strategically for the first two years and engaged with hundreds of enterprises without looking at their potential to contribute to strengthening the value chain. As a result, at the midpoint of implementation, ACAT has to adapt its implementation strategy to empower lead firms as drivers of value chain innovation. C3.2 The ET was also able to draw the following conclusions from feedback gathered relative to separate value chains: • Dairy: The milk suppliers need training to develop their business skills to convert traditional farming to commercial farming (e.g., PHH, bookkeeping/accounting, quality standards, and packaging and branding). ACAT-supported dairy enterprises actively operate the processing lines and cooling systems. As FGD participants noted, and the ET observed, ACAT’s interventions in the dairy sector are visible. • Horticulture: The F&V yields in Khatlon Region are similar to those in the rest of the country and, in some cases, are even higher. However, for key products in the region like peaches and/or apricots, where traditional methods are still used, there is still a substantial lag. Large dissemination and financial support for modern intensive orchards or greenhouses could massively increase output. The production sector is comprised of a large number of SMEs. Cooperation between enterprises in these agribusiness value chains would offer a strategic alternative for such enterprises to react to changing competitive conditions if there was a catalyst such as ACAT to provide guidance on how to do this. • BDSPs: ACAT’s BDSP partners refer to a wide range of non-financial services used by entrepreneurs to help them improve: the performance of their businesses, access to markets, and their ability to compete. The types of services generally in use include business advisers, consultants, and training programs. The women interviewed raised many issues relating to the quality of business support available to them. The core responsibility of a business developer is growing a business. In most cases, BDSPs need experience in sales, marketing, and business management. Certain skills are also vital to entrepreneurs’ success in this field, including those in sales, communication, negotiation, marketing, and data analysis and business management, as expressed during the FGD with BDSPs and small enterprises. ACAT BDSP partners mostly focused on training in technology (e.g., pruning and grafting) of growing agriculture crops which, although extremely useful, appeared to be insufficient to help ACAT’s beneficiaries move forward with any urgency while still lacking strategic management direction. 4.3.3 Recommendations # Recommendation Addressed to Timeframe Priority S/T L/T L M H Recommendations for current program R3.1 ACAT needs to focus on operationalization of the lead firms model, work with the suppliers, and build the capacity of its grantees, particularly on developing their soft skills. The linkages within the value chains should also be explored and facilitated along with further work with MFIs, BDSPs, and agro-logistic centers on coaching the borrowers on how to use and manage the credit for their business. Winrock CO ▲ ▲ Recommendations for future programming 32 # Recommendation Addressed to Timeframe Priority S/T L/T L M H R3.2 Clarity on the market system approach is needed. In this regard building staff capacity in market system development through STTA is critical. It is important to have clear criteria on selection of sub-sectors with a defined approach for facilitating the selected value chains. Conduct in-depth situation analysis, especially with regard to the BDSPs’ availability and capacity on the ground to ensure design of relevant strategies to strengthen the service sector for the value chains. Winrock HQ ▲ ▲ R3.3 Focus on strengthening the competitiveness of the private sector and employment creation market system approach by expanding the geographic coverage and supporting national-level projects. USAID ▲ ▲ R3.4 To ensure better sustainability of access to finance work and facilitate system-level changes, consider and explore the possibility of working directly with banks using an impact investment funding approach. This approach focuses on establishing partnerships with large regular banks or pools of banks and helping them learn how to generate income by providing direct services to businesses to complement their interest rates. This approach was successfully piloted by FinDev Canada. 14 USAID ▲ ▲ 4.4 EQ 4: WHAT SHOULD BE IMPROVED TO ENSURE SCALING UP AND SUSTAINABILITY OF THE MARKET SYSTEM APPROACH? 4.4.1 Findings In market driven interventions, scale and sustainability are closely related. Sustainability is achieved when the private sector takes ownership of the upgrading process and can respond to dynamic market trends. Scale is reached when the entire industry shares in and benefits from this increased capacity. Scaling up has two dimensions: 1) the proportion of stakeholders embracing behavior that increases competitiveness and 2) the depth of their commitment. Enhancing the commitment to the new behaviors of multiple functions in a value chain, and enabling them to act on this commitment, will allow these behaviors to become entrenched. As new behaviors shift from being the exception to becoming the industry norm, they create competitive environment, requiring other stakeholders to likewise innovate and upgrade. The document review revealed that ACAT does not have a scaling up and sustainability plan for the Activity and some description of the approach is mentioned in the annual workplans under different objectives. To maximize value chain transformation in the dairy and horticulture sectors in ACAT’s ZOI, project staff plans to undertake the following initiatives: 1) narrow the suite of activities in each sector to those initiatives that allow for the scale-up of proven innovations; 2) replicate successful business models; and 3) foster integrated value chain business relationships. Lead Firms During the first half of the project, ACAT worked with partner enterprises to assess various products with potential to be competitive in the local and/or international markets. By the project midpoint, ACAT has identified a select group of products that have the potential to scale up and penetrate new markets. ACAT identified 20 lead firms in the project’s priority value chains (fruit drying, nut processing, F&V 14 https://www.findevcanada.ca/en/what-we-do/our-approach 33 processing and value-addition, horticulture commodities aggregations, milk collection services, and dairy processing) to anchor their respective value chains. In view of the interviewed stakeholders (ACAT, business associations, international donors, and other USAID projects, N=18), this approach should ensure effective value chain management through strengthened linkages between value chain players. The available evidence on the market system approach15 suggests that lead firms are more capable of developing supplier and market linkages that create opportunities for downstream and upstream firms, creating a pathway that can benefit other firms. ACAT also plans to connect the lead firms with other donors for attracting additional funding, e.g., EBRD GEFF, the World Bank Consolidation of Agriculture Sector Project, and KfW Agri Value Chain Project. Therefore, such an approach should result in scaling up. Nevertheless, as the lead firms were selected in the first quarter of FY 4, the question arises whether the project has sufficient time and resources to ensure sustainability of the piloted approach, specifically 1) capacitate the lead firms, and 2) get them to the point where they can be providing services and pulling other smaller firms in the marketplace in the remaining timeframe. Interviews with lead firms (N=3) show that there is a different level of maturity of the enterprises in target value chains; dairy enterprises are much more developed compared with horticulture enterprises. Consequently, different levels of effort and time will be needed to ensure the sustainability of lead firms in both value chains. Therefore, it is highly unlikely that it can be achieved within the remaining 15 months of the project implementation. In FY 3 ACAT established the diagnostic analyses or roadmaps for individual enterprises across the value chains to determine the level and extent of TA required to enhance the performance of individual businesses and, by extension, the agribusiness market ecosystem in the Khatlon region through the end of the Activity. To ensure the sustainability of ACAT’s investments in equipment, ACAT is supporting agribusinesses’ technical staff by strengthening their long-term skills and market knowledge. ACAT plans to offer technical trainings focused on efficient equipment use, safety, and equipment maintenance, which is a highly relevant approach towards sustainability. However, as was evident from ACAT’s records, to date, 51 of 162 grantees are still waiting for their equipment, which supposed to arrive by the end of April￾May 2022. Considering the change in a geopolitical situation at the end of February 2022 due to the start of Russian-Ukrainian conflict, there is a high likelihood that ACAT might experience further delays in the delivery and installation of the equipment, which in turn might impact the sustainability of the results. In view of the most of interviewed ACAT partners and stakeholders (see Table 8), it is still too early to assess whether the ACAT-supported agribusiness enterprises in Khatlon became more competitive and market-driven. Table 8: Summary of Findings from Direct KIIs (Stakeholders) – EQ 4 KIIs Do you believe that agribusiness enterprises in Khatlon are more competitive with donor support and ACAT in particular? Are agribusinesses now more market-driven leading to the likelihood of increased sales and jobs? Group Yes No Unsure Yes No Unsure Central government (Dushanbe) 1 1 Local/district authorities (Khatlon Region) 2 2 3 1 2 4 Women/youth associations 1 1 NGOs 3 3 International donors/organizations 4 1 4 1 Business associations 2 2 1 3 BDSPs 2 1 2 1 Total 12 2 10 9 2 13 15 https://www.marketlinks.org/search?keywords=lead+firms 34 Findings from the PHH FGD indicate most of the enterprises are former members of the state regional research horticulture station located in Khatlon. They have significant experience in horticulture management, but they are not commercially-oriented enterprises. Feedback from the Dairy Group II FGD shows there is no motivation from the MCC side, nor is there an appropriate coordination between MCCs and milk suppliers. Business Development Service Providers One pillar of ACAT’s market system approach requires BDSPs to establish commercialized services to businesses so that service provision moves away from being donor dependent to become more private sector focused. The project is engaging local organizations (for profit and non-profit) to develop a cadre of BDSPs as well as local businesspeople, food technologists, and academics to provide business services. The ACAT-supported BDSPs could be clustered into five groups based on the types of services provided: 1) market access (market linkages, market research, trade fairs and product exhibitions, and demo sites), 2) infrastructure (storage and warehousing), 3) training and technical assistance (technical training and counseling/advisory services), 4) technology and product development (design services and quality assurance), and 5) input supply (linking MSMEs to input suppliers). According to the United Nations Conference on Trade and Development (UNCTAD), one of the most important tasks for developing SMEs is creating effective BDSPs that possess several core principles used to achieve best practice in the implementation. One of the core principles is sustainability of services, which should be assessed at two levels: the level of impact and the level of BDSP. Based interviews and FGDs, the sustainable BDSPs are the NGOs that were attracted by ACAT, such as the NGOs partnered with the project in post-harvest processing and providing training and agricultural services in grafting, pruning, protecting plants from pests and diseases, and post-harvest activities—Nuri Khatlon, Taqviyat va Tavonmandi, Sarchashmai Hayot, and Rushdi Diyor. The BDSP FGD illustrated that ACAT-supported BDSPs have extensive experience working with the community in different fields, including the agribusiness sector. However, they are also non-commercial and donor-oriented, which calls their sustainability into question when there is no donor support. One male BDSP FGD participant from Nosiri Khisrav district observed, “We are public and non-profit organizations and, therefore, always look for funds from donors to partially implement the donors’ funded activities in partnership.” Moreover, they have limited in the technical knowledge required to advise on scaling up at enterprise level. International experience16 indicates that the most financially sustainable BDSPs are for-profit organizations and membership organizations. ACAT supported the setup of the Khatlon Business Association (KBA), which is a membership organization, by assisting with the development of a three-year operational and financial strategic plan and website and identifying business services to offer on behalf of members and non￾members. Interviews (N=4) indicated that the most demanded services of KBA are training and technical advice on Hazard Analysis and Critical Control Points (HACCP) and International Organization for Standardization (ISO) certifications. However, the main problem of KBA is its still weak institutional and governance structure which requires further strengthening. Consequently, further support with organizational development of the association is needed. ACAT has also worked on strengthening the input supply network which remains limited and operates inefficiently, especially at the district level, and restricts the availability of agri-inputs during the growing season for the horticultural sector in Khatlon. The project partnered with Neksigol, one of Tajikistan’s largest agri-input suppliers, to develop a network of input shops. This form of embedded service delivery is a relatively new concept in Khatlon but using a commercial approach to supply high-quality inputs contributes to sustainability because it is anchored in the private sector. The scale up of this approach should be further explored. ACAT has also supported the establishment of agri-logistic centers as very few functioning consolidation points exist for horticultural produce in Khatlon. The project partnered 16 Business Development Services: Review of International Experience by Levitsky, J., (ed.). ITDG Publishing, 2000. 35 with three local partners through TA and grant assistance to develop a purpose-built facility to handle, store, sort, grade, and export horticulture produce to be aggregated in a coordinated manner. The interviews with firms (N=4) show that these agri-logistic centers are in high demand among producers. This model should be further reinforced by ACAT. Access to Finance Interviews with ACAT grantees and MFI partners demonstrated a high interest among businesses in discounted loans. In response, ACAT has scaled up its work with MFIs. The project engaged Bank Arvand, which had the capacity to handle larger inflows of funds and customers, under a revolving fund mechanism to develop a new loan product for the nut, stone fruit, and dairy value chains. ACAT grant funds are used to subsidize loan interest rates and Bank Arvand provides financial resources in the amount of $256,709 in loanable funds to agribusinesses. Bank Arvand offers loans with higher value, longer duration, and reduced collateral requirements17 than ACAT-supported MFIs. Loans are provided to customers in ACAT’s ZOI districts for up to 36 months with various types of collateral and flexible repayment schedules. Loan amounts vary from 250,000 Tajik Somoni ($25,000) for individuals to 450,000 Tajik Somoni ($45,000) for legal entities. As the loan value is higher, it could be of interest for the ACAT’s lead firms. As of February 2022, Arvand extended 20 loans valued at 433,000 Tajik Somoni. However, the ET found that Arvand would not be able to offer the loans with the duration of 36 months to the borrowers as their agreement with ACAT concludes in March 2023. At the current agreement arrangements, the duration of the loan could be just 12 months. Due to such short duration, as of now, there is no demand for this loan product offered by Arvand Bank. Therefore, the planned scale up of the discounted loans through Arvand is under threat. At the same time, existing ACAT-supported loan products in four MFIs (MCF Sarvati Vakhsh, MCO Mehnatobod, MCF Chiluchor Chashma, and MCF Imkoniyat) are revolving funds, meaning that as loans are repaid, new loans are issued to new clients. ACAT has selected a good approach for ensuring the sustainability of the MFIs’ loan products. The signed agreements with MFIs covered three years and have a provision that MFIs would offer the loans at a 16-18 percent interest rate for an additional three years after the end of ACAT. The ACAT team also held a meeting with the National Bank of Tajikistan during which the bank assured the team that it would oversee implementation of the above agreement through its annual monitoring of FIs. In addition, innovative ideas and solutions do not feature highly in the ACAT’s activities, although mention is made in the Activity’s FY 4 work plan under Objective 3, which states, “Pilot innovative ideas, such as a dairy enterprise that serves as a model and center of excellence for others. Peer to peer learning will be used to raise interest for innovative ideas.” Although limited in detail, this recognition is important in that keeping abreast of innovative measures in agriculture worldwide benefits ACAT’s clients, and BSDPs, through adopting or replicating initiatives that reduce or remove constraints to agribusiness development in Tajikistan. The successful and sustainable adoption of agricultural innovations depends on several factors: the relative advantage of the innovation for the user, its compatibility with the user’s setting, and its complexity, trialability, and observability. ACAT beneficiaries suggested the project needs to consider 1) successfully deploying pilot-tested innovations to reach and benefit large numbers of beneficiaries, and 2) using innovations as an entry point to systematic change in the agricultural and processing sectors. Furthermore, ACAT beneficiaries are not able to scale up due to low incomes and high operating expenses (e.g., fuel, fertilizer, and pesticides). Overall, the revision of ACAT’s PMP demonstrates that the Activity is on track and in principle should be able to meet its quantitative targets by June 2023. However, to ensure achievement of qualitative changes 17 Note: If a potential client does not have enough security/collateral for business loans over $2,000, there is also the possibility of providing a guarantee by Bank Arvand’s partner company Credit Guarantee Fund Tajikistan GmbH. Such guarantees range from 60-80 percent of the principal loan amount to Bank Arvand, which greatly reduces the client collateral requirement. 36 (changes in the mindset, behavioral and institutional changes) which are the building blocks of sustainability of supported enterprises and lasting change that is owned and driven by value chain actors, more time is needed. As the goal of ACAT is not limited to increase in sales but includes a strengthened market system and strengthened private sector competitiveness, actors must first strengthen themselves before they can strengthen the linkages between other actors in the value chain. 4.4.2 Conclusions # Conclusion C4.1 ACAT’s effort to strengthen the capacity of horticulture and dairy enterprises to ensure scaling up is productive and cooperative. The sustainability and scale up of SMEs depend on post-ACAT activities, which include employing highly qualified processing technologists, digesting over five years of information on local and export markets, improving packaging, upgrading equipment, and updating the knowledge and skills of employees. Considering the significant delays in selecting grantees, forming the pool of dynamic lead firms (which occurred in early 2022), and providing equipment to the most grantees (which did not begin until the second half of 2021 and with many firms still waiting on receipt), ACAT is running out of time to ensure the scaling up and sustainability of the market system approach. At least one to two full harvest seasons after the initial investment are required to start to observe the results of the ACAT assistance among the supported value chain actors. Thus far, ACAT has only been able to establish a fragile model of a market system in agribusiness and much work remains with enterprises to help them to become confident and more sustainable. 4.4.3 Recommendations # Recommendation Addressed to Timeframe Priority S/T L/T L M H Recommendations for the current Activity R4.1 Consider the extension of ACAT for at least two more harvest seasons to allow full rollout of the lead firm model and promote better sustainability of BDSPs. Based on budget analysis, ACAT will spend all but $400,000 out of $18 million by the end of June 2023. This small left-over amount will be insufficient to ensure a proper scale up. Therefore, consider a cost-extension of ACAT. Additional time would also allow USAID to better monitor and see the impact on these businesses as well as seeing the impact of the Activity. USAID ▲ ▲ R4.2 Concentrate on building up the sustainability of local BDSPs. To accomplish this: a. Stop using NGOs as BDSPs; b. Continue work to strengthen industry associations, like the KBA and Khatlon Association of Business Women (KABW), by further improving their organizational and functional management as well as designing and promoting a range of fee-based services for the horticultural sector. Winrock CO ▲ ▲ R4.3 Work on strengthening, operationalizing, and marketing the agri-logistic centers to expand access to professional and efficient logistic solutions. Winrock CO ▲ ▲ R4.4 Continue support for the agri-inputs network to ensure its transition into FSCs capable of offering high-quality agri￾inputs and professional and efficient related services at the district level in Khatlon. Winrock CO ▲ ▲ Recommendations for future programming 37 # Recommendation Addressed to Timeframe Priority S/T L/T L M H 4.5 Consider usage of a multi-phased approach for market￾driven interventions in Tajikistan. Market system projects take time to get going as in-depth market research and identification of willing partners are needed. The interventions also take time to mature, once started. The phased approach provides an opportunity to set clear goals while allowing for longer-term piloting and learning. It also ensures that lessons could be learned, and the approach adapted. USAID ▲ ▲ 38 ANNEXES 39 ANNEX 1: STATEMENT OF WORK SECTION C – DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK Agribusiness Competitiveness Activity in Tajikistan Midterm Evaluation. PURPOSE OF THE EVALUATION USAID-funded Agribusiness Competitiveness Activity in Tajikistan (ACAT), implemented by Winrock International (Winrock), runs from July 2018 through June 2023. ACAT was designed to strengthen Tajik agribusinesses and transform agricultural production in horticulture and dairy value chains from subsistence farming to a commercial orientation. The activity is implemented in thirteen districts in southwestern Khatlon region of Tajikistan, 12 of which are included in the Feed the Future (FTF) zone of influence. Tajikistan has been an FTF aligned country since 2018, prior to which it was an FtF focused country. The purpose of this midterm evaluation is to assess the progress of Agribusiness Competitiveness Activity in Tajikistan (ACAT) toward projected outcomes and objectives to enhance learning and provide accountability. The evaluation will outline which interventions are working well, which are not, and why. It will also provide recommendations for modifications and mid-course corrections to help guide the program through the second half of implementation. The evaluation should provide pertinent information, statistics, and judgments that assist USAID to learn what has been accomplished against the intended results of the ACAT contract, including any management, financial, and cost-efficiency findings. The evaluation will help USAID, Winrock and related stakeholders to better understand the initial results and contributions of the activity and help refocus and strengthen it. BACKGROUND/COUNTRY CONTEXT AND PROBLEM ADDRESSED Agriculture contributes to 21 percent of Tajikistan's Gross Domestic Product (GDP) and employs 51 percent of the labor force. While it is the largest sector of Tajikistan’s economy, continued reliance on a few major crops like cotton, consistently poor crop yields and the absence of a vibrant agribusiness sub￾sector restricts agriculture sector growth as a whole. Particularly, the horticulture and dairy sub-sectors are characterized by inefficient production and marketing systems, high rates of post-production losses, low level of value addition and the inability of value chain actors to comply with export market quality standards. Despite excess production, Tajikistan imports fruits and vegetables from Pakistan and India to cover domestic market demand outside the harvest season. Additionally, local supermarkets sell mainly imported fruit juices, canned tomatoes, milk, and milk products (such as cheese and yogurt) that could be produced locally and for which the country has a comparative advantage. Price volatility and spoilage are acute especially in the Khatlon region, where approximately 30–40 percent of fruit and vegetables produced are wasted because the market cannot absorb excess production at peak season when there are limited opportunities for storage, processing and export. Home to 3.2 million people, Khatlon is a fertile agricultural region spanning 25 districts. Khatlon’s economy depends more heavily on agriculture than other regions in the country and faces multiple challenges. Low wages drive male labor migration to urban areas and abroad, which has exposed women to higher poverty and resulted in many women-owned farms and agribusinesses. A more competitive agribusiness sector would catalyze higher wages which can lead to agricultural commercialization, export-led growth, food import substitution, enhanced food security, employment creation and poverty reduction. The horticulture and dairy sectors in particular demonstrate the potential for increased employment and income generation through diversification into high value production systems, product and process innovation, and greater market orientation. ACAT was designed based on the potential of the agriculture sector in Tajikistan, owing to its size, structure and relevance to poverty reduction. Winrock Intl, the implementing partner for ACAT, uses a market-driven approach, led by the private sector, to stimulate economic growth, increase employment 40 opportunities and improve livelihoods by enhancing the competitiveness of Tajikistan’s agribusinesses in dairy and horticulture value chains in 13 districts in southwestern Khatlon region of Tajikistan (Khuroson, Abdurahmoni Jomi, Yovon, Kushoniyon, Vakhsh, Levakand, Jaloliddini Balkhi, Jaihun, Dusti, Kubodiyon, Shahritus, Nosiri Khisrav and Panj). Making Cents International, is a sub-contractor under Winrock Intl. Building upon previous USAID projects, ACAT provides demand-driven assistance to agricultural associations, cooperatives, market integrators, and small and medium enterprises (SMEs) to take advantage of domestic, regional and international market opportunities. By working through market actors, ACAT will promote innovations and enhance adoption of new and existing technologies in the market system. It will support lead firms, including entrepreneurs, Business Service Development Providers (BDSPs), and financial institutions, to seize on scalable and replicable opportunities, enable opportunities for women and youth, and build stronger value chain relationships to catalyze systemic change. All these interventions will enable Khalton agribusinesses to utilize an opportunity to take advantage of its different seasonal calendar and greater supply of arable land. Project Goal and Development Hypothesis The overall goal of ACAT is “Sustained economic growth through interventions aimed at improving competitiveness of Tajikistan’s agribusiness enterprises leading to increased economic development, employment, and livelihoods”. By achieving its goal, ACAT will contribute to the USAID/Central Asia Regional Development Cooperation Strategy Development Objective (DO) 1: Expanded diverse and competitive trade and markets. The intermediate results (IRs) applicable to ACAT include: IR 1.1: More diverse and competitive private sector; and IR 1.2: Enhanced agricultural competitiveness and food security. The project pursues its goal through three objectives: 1) Diversify agricultural sector that includes higher value commodities and value-added products; 2) Strengthen linkages between aggregators, wholesalers, exporters, and international and domestic end markets; and 3) catalyze systemic change in the agribusiness sector through partnerships and innovation and technology to diversify production to higher-value resources. If ACAT is successful in meeting these objectives, a competitive, market driven agricultural system will be established and farmers and agribusiness in the selected value chains should become more competitive, increase sales to domestic and foreign markets, increase their profitability and contribute to broad based economic growth in the sector. This will lead to increases in employment and income and thus improve economic status of the focus populations in Tajikistan. ACAT’s theory of change relies on identifying and working through dynamic firms and BDSPs to catalyze market system improvements by providing embedded services and leadership to strengthen market system relationships. As articulated in the below figure, ACAT will establish a competitive, market-driven agricultural system by 1) improving value chain actors’ ability to respond to market signals and diversify product offerings in response, 2) improving actors’ awareness of, access to and adoption of technologies, and services, 3) strengthening agribusinesses linkages, and 4) stimulating investment in technology. Based on the Collaborating, Learning and Adapting (CLA) session conducted in June 2020, Winrock confirmed that its theory of change is still valid to apply in Khatlon’s conditions. ACAT Theory of Change Approach, Implementation and Results to Date To meet the goal and objectives laid out in the theory of change, ACAT embraces the principles and techniques of USAID’s market systems development approach. Specifically, instead of providing direct services to enterprises, the ACAT team will facilitate improved business relationships and services. It will convene, coordinate, build capacity, demonstrate, and introduce new technologies, agricultural and management practices, and methods for organizing value chain actors in a manner that builds trust and efficiency. Through linking, coaching, and mentoring, ACAT will foster inclusive, mutually beneficial value chain relationships, support embedded services in private sector partners to demonstrate and replicate 41 sustainable business models and maximize opportunities to impact multiple value chains. The goal is to foster systemic change, more specifically, changes in the way business is conducted that achieve increased efficiency and profitability for buyers, producers, and service providers. It will identify the market opportunities and work back toward the producer, with a strategic focus on finding partners that drive upgrades in infrastructure and processes that catalyze market changes and provide strong market signals to stimulate further investments along horticulture and dairy value chains. For example, ACAT will work with lead firms to demonstrate the commercial viability of technologies, upgrading quality standards, and investing in their supply chain. Through its opportunity mapping assessment in year 1, ACAT identified an initial group of lead agribusiness and service firms in the dairy and horticultural value chains with the ability to influence downstream and upstream value chain dynamics. To better plan field activities, ACAT divided 13 target districts into four clusters based on the geographical locations and capacity of horticulture and dairy products. ACAT implements its activities through 1) technical assistance for capacity building and program support, and 2) grant component under its Partnership and Investment Fund (PIF). The technical assistance and grants to agribusinesses aim to increase agricultural production and promote products (e.g., through value addition) and process (e.g., new technologies and practices, certifications, market access) in selected horticulture and dairy value chains. The grant mechanism activity mobilizes private sector investment in agribusinesses and related enterprises geared to developing value chains that effectively create employment. ACAT will also build the capacities of local BDSPs to provide agribusiness support services. Implementation is carried out in collaboration with international and local partner organizations, including other USAID activities with relevant experience in market/enterprise and rural development. For results achieved to date, please see ATTACHMENT J.3 ACAT achievements to date. EVALUATION RATIONALE AND OBJECTIVE Despite progress, ACAT has been facing challenges in handling grants under its Partnership and Investment Fund (PIF), resulting in delays. Delays in the PIF have slowed down provision of technical assistance to agribusinesses because most technical assistance activities should be realized as complementary to PIF. Because of this, ACAT has built up a large pipeline. Winrock has also experienced significant staff turnover, which has affected project performance. The Khatlon region is considered a challenging operating environment because of its developing agribusiness sector. In addition, ACAT is the first USAID agribusiness project in Tajikistan to employ a market system approach, requiring greater cooperation from agribusiness partners compared to traditional approaches. Hence, USAID/Tajikistan wishes to conduct the midterm evaluation of ACAT to assess the progress that was made until now, implementation challenges, and opportunities and progress made toward anticipated outcomes and objectives. A midterm evaluation will provide timely evidence needed to ensure efficient and effective implementation halfway through the activity. Results and findings will be used to adjust interventions to ensure that they are sustainable and institutionalized to achieve goals and objectives of the U.S. Strategy for Central Asia, and to better support Tajikistan on its way to self-reliance. Lessons learned will be incorporated into USAID’s new designs and future work in these sectors, as applicable. EVALUATION QUESTIONS 1. Given the performance of ACAT’s grants component to date, what should be changed to ensure that the PIF achieves key program objectives e.g., increasing production, productivity and competitiveness of agribusinesses? 2. What improvements to the management structure might be needed to ensure that ACAT achieves the activity goal and objectives? 42 3. Midway, what about ACAT’s market system approach to improving agribusiness competitiveness in Tajikistan is working well, and what is not? 4. What should be improved to ensure scaling up and sustainability of the market system approach? DATA ANALYSIS METHODS The evaluation team should identify appropriate quantitative and qualitative aspects of the project in order to address the evaluation questions raised above. The evaluation team is expected to base their conclusion on empirical evidence gathered from a variety of sources which should include both primary and secondary data sources. The evaluation team should use a sound methodological approach that includes but is not limited to field visits to project areas, focus group interviews with relevant stakeholders such as central and local government officials, farmers associations, input suppliers and private sector representatives. The evaluation team is expected to meet with USAID/Tajikistan Mission and ACAT staff including subcontractors and grantees, program clients and beneficiaries, key donors working on agribusiness, and relevant stakeholders at project sites in order to gain first person perspective of ACAT activities and the experience and perceptions of the various stakeholders. However, USAID expects a detailed evaluation design and plan to be presented to the USAID/Tajikistan Mission before the start of evaluation for review and approval. The contractor will have flexibility in proposing the most appropriation methodology to accomplish the tasks outlined in the most efficient way. However, the contractor will address the following: Task 1: Desk review The contractor will conduct a desk review to provide a general overview of the agribusiness sector in Tajikistan and the current status of ACAT activities in Khatlon region, and will help identify areas that need closer attention once the team begins its fieldwork. Reading materials will be available to the team prior to their arrival in Tajikistan. The contractor shall also review other relevant non-project related documents such as peer-review articles and other publications and incorporate the review findings into the data collection design and data analysis and interpretation. The evaluation team shall consult a broad range of background documents apart from project documents provided by USAID/Tajikistan Mission. USAID and ACAT will provide the evaluation team with a package of briefing materials, including: • USAID Central Asia Regional Development Cooperation Strategy (2015-2019) • Tajikistan Feed the Future Multi-Year Strategy (2011-2015) • ACAT work plans, 1-3 years of the activity • ACAT Annual Reports, 2019 -2020, and 2021 (when available) • ACAT Quarterly Reports, 2019-2020, and 2021 (when available) • ACAT Financial Reports, 2018-2020, and 2021 (when available) • ACAT Annual Surveys, 2019-2020 • ACAT Baseline Survey, 2019 • ACAT CLA Session Report, 2020 • ACAT Access to Finance Assessment Report, 2019 • ACAT Dairy Assessment Report, 2019 • ACAT Gender Strategy Report, 2018 • ACAT Horticulture and Dairy Value Chain Assessment Report, 2018 • ACAT Nursery Commercialization Report, 2020 • Feed the Future Tajikistan Baseline and Midline Survey Reports • Government of Tajikistan National Development Strategy up to 2030 • USAID Tajikistan job diagnostic analysis • USAID Tajikistan Feed the Future Gender Analyses 43 • ADB Tajikistan Country Gender Assessment • Other documents as required The final evaluation report shall incorporate information obtained from the pre-field work desk review as well as information obtained from key informants and observations in the field. Task 2: Draft description of overall methodology and work plan In conjunction with USAID/Tajikistan Mission, the contractor will develop a plan for conducting their fieldwork using information from the desk review and this SOW. The work plan will clearly outline the methodological approach the team will use. The work plan shall be submitted for review to the USAID/Tajikistan Mission COR for approval prior to arrival in the country. The work plan shall include a field work schedule, including a preliminary list of all proposed districts for travel and key organizations the evaluation team plans to meet. Task 3: Conduct a field data collection The in-country data collection will expand upon the analysis in the desk review. The tasks will include facilitated discussions through interviews, focus groups of sub-grantees/end-users, the Government of Tajikistan, business associations, other private sector and certification entities and conduct field visits using other evaluation methods. The evaluation team will spend at least three weeks in-country to conduct the evaluation. The identification of key stakeholders and informants and the scheduling of in-person interviews/focus groups must be arranged in advance as much as possible to maximize efficient use of time while in the field. USAID/Tajikistan Mission will provide introductions and contact information to facilitate the data collection. The team will travel outside the capital to the districts as stated in the approved work plan to collect data and/or obtain information from ACAT’s beneficiaries (e.g., male and female smallholder farmers, extension providers, district and government officials, agribusiness enterprises, etc.). Data collection will be conducted in all 13 districts in Khatlon (Khuroson, Abdurahmoni Jomi, Yovon, Kushoniyon, Vakhsh, Levakand, Jaloliddini Balkhi, Jaihun, Dusti, Kubodiyon, Shahritus, Nosiri Khisrav and Panj). All target districts terrains are flat, allowing the evaluation team to have smooth travels. 44 ANNEX 2: LIST OF MEETINGS AND INTERVIEWS a. First Round KIIs Participants (October 19 to November 5, 2021) KIIs Contact Person Position Organization Location Date 1 Task Order COR USAID/Tajikistan Remote 10/19/21 2 A&A Specialist USAID/Tajikistan Remote 10/19/21 3 Task Order CO USAID/Tajikistan Remote 10/19/21 4 Alternate Task Order COR USAID/Tajikistan Remote 10/19/21 5 COR of ACAT USAID/Tajikistan Remote 10/19/21 6 COP ACAT Project Face-to-face Dushanbe 10/19/21 7 MEL Director ACAT Project Face-to-face Dushanbe 10/19/21 8 M&E Consultant IFAD Remote 10/20/21 9 Deputy Director Tajik Export Agency Face-to-face Dushanbe 10/20/21 10 Head of Department of Export of Industrial Products Tajik Export Agency Face-to-face Dushanbe 10/20/21 11 “Thrive” Deputy Program Manager Aga Khan Foundation Remote 10/20/21 12 Director AMFOT Face-to-face Dushanbe 10/21/21 13 Project Manager GEFF – EU Project Remote 10/21/21 14 Deputy Director GEFF – EU Project Remote 10/21/21 15 Chairman of Management Board NGO “Sarob” Remote/Face-to-face (Dushanbe 10/21/21 16 First Deputy of District Governor Government of Yovon Region Face-to-face – Yovon Region 10/22/21 17 Head of Agriculture Department Government of Yovon Region Face-to-face – Yovon Region 10/22/21 18 Director NGO “Nuri Khatlon” Bokhtar 10/23/21 19 Director NGO “Sadoi Kuhsor” Bokhtar 10/23/21 20 Executive Director National Association of Business Women Remote 10/25/21 21 Business Development Officer National Association of Business Women Remote 10/25/21 22 Head of Agro-Industry Department Khatlon Government Bokhtar 10/25/21 45 KIIs Contact Person Position Organization Location Date 23 Head of Chamber & Industry Khatlon Region Bokhtar 10/25/21 24 Dairy Specialist ACAT Bokhtar 10/25/21 25 Area Manager UNDP/Tajikistan Khatlon Office 10/26/21 26 Horticulture VCD Specialist ACAT Bokhtar 10/26/21 27 FAO Representative in Tajikistan FAO Remote 10/27/21 28 Project Manager FAO Remote 10/27/21 29 National Project Coordinator FAO Remote 10/27/21 30 Project Manager FAO Remote 10/27/21 31 Director Neksigol (BDSP) Remote 10/28/21 32 Chairman Khatlon Business Association Face-to-face – Bokhtar 10/28/21 33 Head of Agriculture Department Government of Nosiri Khisrav District Face-to-face 10/29/21 34 First Deputy of District Governor Government of Nosiri Khisrav District Face-to-face 10/29/21 35 Director MFI “Chiluchor Chashma” Face-to-face – Bokhtar 10/29/21 36 Head of Business Development MFI “Chiluchor Chashma” Face-to-face – Bokhtar 10/29/21 37 Director Khatlon State Business Incubator, BDSP Face-to-face – Bokhtar 10/30/21 38 Country Manager – Tajikistan USAID’s Future Growth Initiative Project (FGI) Remote 11/1/21 39 Country Director USAID/DAI Agriculture Land Governance Project Remote 11/1/21 40 First Deputy Head Jaihun District Government Face-to-face 11/1/21 41 Product Design & Marketing Advisor ACAT Staff Face-to-face – Bokhtar 11/1/21 42 Enterprise Development Specialist ACAT Staff Face-to-face – Bokhtar 11/1/21 43 PIF Manager ACAT – PIF Remote 11/1/21 44 PIF Specialist ACAT – PIF Face-to-face – Bokhtar 11/1/21 45 PIF Specialist ACAT – PIF Face-to-face – Bokhtar 11/1/21 46 KIIs Contact Person Position Organization Location Date 46 Dep. Head of District Government of Kushoniyon District Face-to-face – Kushoniyon 11/2/21 47 Director MFI “Sarvati Vakhsh” Face-to-face 11/3/21 48 Director MFI “Imkoniyat” Face-to-face 11/3/21 49 Director MFI “Mehnatobod” Face-to-face 11/3/21 50 DCOP ACAT Staff Face-to-face 11/3/21 51 Access to Finance Advisor ACAT Staff Face-to-face 11/3/21 52 MEL Director ACAT Staff Remote 11/3/21 53 Inclusion Agribusiness Specialist Making Cents International Face-to-face – Bokhtar 11/4/21 54 First Deputy of MoA Ministry of Agriculture Remote 11/4/21 55 Executive Director Agribusiness Association of Tajikistan Phone Call 11/4/21 56 Project Management Specialist USAID Remote 11/5/21 b. Second Round KIIs Participants, all remote KIIs (February 7 to March 2, 2022) KIIs Stakeholder Group Name Position Organization 1 ACAT COP Winrock 2 ACAT DCOP Winrock 3 ACAT Operations Director Winrock 4 ACAT Finance Director Winrock 5 ACAT PIF Director Winrock 6 ACAT PIF Specialist Winrock 7 ACAT Social Inclusion and Communication Manager Winrock 8 ACAT MSME Access to Finance Advisor Winrock 9 ACAT Enterprise Development Specialist Winrock 10 ACAT Dairy Specialist Winrock 11 ACAT Produce Design and Marketing Manager Winrock 12 ACAT MEL Specialist Winrock MEL Assistant Winrock 13 ACAT Admin Manager Winrock 14 ACAT Sr. Program Office Winrock HQ Director Winrock HQ 15 MCI MCI Inclusion Specialist MCI 47 KIIs Stakeholder Group Name Position Organization 16 USAID Project Management Specialist, General Development Office USAID Tajikistan 17 USAID Project Management Specialist, General Development Office USAID Tajikistan 18 USAID Director of General Development Office USAID Tajikistan 19 MFIs Chairman of the Board MCF Sarvati Vakhsh 20 MFIs Intern MCF Imkoniyat 21 MFIs Chaimen of Arvand Arvand 22 Other partners Director Khatlon State Business Incubator 23 Other partners Expo organizing director under Chamber of Commerce Chamber of Commerce 24 Business Associations Head of Association National Assosiation of Small & Medium Business of the Republic of Tajikistan 25 BDSPs/NGOs Director BDSP Al Mar Consulting LLC 26 BDSPs/NGOs Director NGO Nuri Khatlon 27 Lead firms Director Tijorati Sorbon 28 Lead firms Director Osio-M 29 Lead firms Deputy Director Salvati Mahmud 30 Lead firms Director OJSC Kombinati Hurokvori 31 Micro-Enterpises Director MCCP Nosiri Khisrav 32 Micro-Enterpises Director MCCP Sorbon 2013 33 Micro-Enterpises Director PE Tojiddin-Pistachio Processing 34 Other Development partners/Donors Country Director USAID CTJ/TRA 35 Other Development partners/Donors DCOP USAID Agriculture and Land Governance (ALG) Activity 36 Other Development partners/Donors DCOP Trive Tajikistan, Aga Khan Foundation (AKF) c. FGDs and GDs Participants (October 19 to November 5, 2021) FGDs Category M F T Position Organization Location Date 1 Milk Suppliers 5 6 11 Owners Private Yovon District 10/22/21 2 Loan Recipients 4 4 8 Individual Entrepreneurs Private Khatlon Districts (Various) 10/26/21 3 Horticulture 6 2 8 Head of Enterprise Private Khatlon Districts (Various) 10/27/21 48 FGDs Category M F T Position Organization Location Date 4 Dairy Value Chain 7 1 8 Head of Enterprise Private Khatlon Districts (Various) 10/28/21 5 Post-Harvest Handling – In-Kind Grants 6 2 8 Head of Enterprise Private Khatlon Districts (Various) 11/2/21 6 BDSP 4 2 6 Directors Private Khatlon Districts (Various) 11/2/21 7 Enterprises Selected – No Grants Yet 6 2 8 Owners Private Khatlon Districts (Various) 11/2/21 GDs Category No. Position Organization Location Date 1 Young Women 3 Internship Program Students Only 1 working Khatlon Districts (Various) 10/27/21 2 Youth & Young Women 5 Milk Producers Private Nosiri Khisrav 10/29/21 3 Young Woman and Young Man 2 Individual Farmers Private Bokhtar 11/4/21 49 ANNEX 3: LIST OF DOCUMENTS CONSULTED Title of Document 1. USAID Central Asia Regional Development Cooperation Strategy (2015–2019) 2. Tajikistan Feed the Future Multi-Year Strategy (2011–2015) 3. ACAT work plans, 1–3 years of the activity 4. ACAT Annual Report – Year 1 (October 2019) 5. ACAT Quarterly Reports: Q. Report – October 2018; Q.1 - January 2019; Q.2 – May 2019; Q.3 – July 2019; Q.1 – February 2020; Q.2 – May 2020; Q.3 – August 2020; Q.4 – February 2021; Q.1 – February 2121; Q.2 – May 2121; Q.3 – August 2121 6. ACAT Financial Reports 7. ACAT Annual Surveys, 2019–2020 8. ACAT Baseline Survey, 2019 9. ACAT CLA Session Report, 2020 10. ACAT Access to Finance Assessment Report, 2019 11. ACAT Dairy Assessment Report, 2019 12. ACAT Gender Strategy Report, 2018 13. ACAT Horticulture and Dairy Value Chain Assessment Report, 2018 14. ACAT Nursery Commercialization Report, 2020 15. Feed the Future Tajikistan Baseline and Midline Survey Reports 16. Government of Tajikistan National Development Strategy up to 2030 17. USAID Tajikistan job diagnostic analysis 18. USAID Tajikistan Feed the Future Gender Analyses 19. Signed contract USAID / Winrock – 19 June 2018 20. Contract Modification No.1 – USAID / Winrock – 19 June 2018 21. Contract Modification No.2 – USAID / Winrock – 19 June 2018 22. Contract Modification No.3 – USAID / Winrock – 19 June 2018 23. Contract Modification No.4 – USAID / Winrock – 19 June 2018 24. Contract Modification No.5 – USAID / Winrock – 19 June 2018 25. World Bank – Ease of Doing Business in Tajikistan 2008–2019 26. World Bank – Tajikistan: Country Economic Update – Fall 2020 27. U.S. State Department: 2021 Investment Climate Statements Tajikistan – Economic Slowdown Amid the Pandemic 28. European Bank for Reconstruction and Development (EBRD) – Tajikistan: Transition Report 2020-21 – Country Assessment 29. International Monetary Fund (IMF) – Tajikistan Country Report No.20/151 30. Bertelsmann Transformation Index (BTI) – Tajikistan Country Report 2020 31. Food & Agricultural Organization of the United Nations (FAO) – Tajikistan Country Brief, May 2021 32. United Nations (UN) – Tajikistan: Sustainable Development Goal: “Decent Work and Economic Growth” 2021 33. Asian Development Bank (ADB) Fact Sheet Tajikistan, June 2021 34. Caspian Policy Center – Strategies for Inclusive Economic Growth: Women’s Entrepreneurship in Tajikistan, Aug. 2021 35. ADB Tajikistan Country Gender Assessment 36. ACAT Accrual Financial Reports – 2018 - 2021 37. ACAT Annual Survey Report – Year 1 – August 2019 50 Title of Document 38. ACAT Annual Survey Report – Year 2 (no date) 39. ACAT Access to Finance Assessment – September 2018 40. ACAT Dairy Processing Enterprise Assessment – June 2109 41. ACAT Dairy Technical Assistance Report – May 2020 42. ACAT Gender and Youth Inclusion Strategy – August 2018 43. ACAT Horticulture Status and Recommendations – May 2019 44. ACAT Horticulture and Dairy Value Chain Assessment – August 2018 45. ACAT Horticultural Status and Recommendations – May 2020 46. Thrive Tajikistan – Partnership for Socio-Economic Development – USAID / Aga Khan 47. IFAD Country Strategic Opportunities Programme 2019-2024 48. Women Empowerment in Tajikistan – UNDP: 2016 49. Barriers to Female Entrepreneurship – UNDP / Ministry of Foreign Affairs Finland 51 ANNEX 4: EVALUATION MATRIX ACAT – Evaluation Matrix No. Evaluation Question Data Sources Data Collection Method Data Analysis EQ1 Given the performance of ACAT’s grants component to date, what should be changed to ensure that the PIF achieves key program objectives e.g., increasing production, productivity, and competitiveness of agribusinesses? [Effectiveness: Quantitative + Qualitative] • ACAT documents: annual and quarterly reports; M&E reports, assessments, studies, and surveys • Documents related to ACAT’s direct beneficiaries: agricultural associations, cooperatives, SMEs, entrepreneurs, BSDPs, financial institutions, i.e., related to grants, TA and training • USAID Mission staff • ACAT key management personnel • Pre-identified “stakeholders” – individuals/organizations with direct/indirect knowledge of ACAT including women and youth organizations • Government line ministries: Ministry of Agriculture; Ministry of Energy and Industry among others • Other relevant sources including publications for the international donor community • Document search and review sourced from USAID and ACAT work plans, studies, assessments, M&E data, and information • Secondary data search and analysis from publications in the public domain – government, donors, private organizations, and NGOs, among others • KIIs in Dushanbe and in ACAT’s target districts in Khatlon with direct beneficiaries and other stakeholders including local authority representatives • FGDs conducted with sub-grantees, Government of Tajikistan, business and agriculture associations, and key private sector operators in the region’s value chain network • Mini-surveys of beneficiaries/ stakeholders, partners, and loan recipients and direct observations where and when deemed possible within the constraints of COVID-19 prevailing at the time of the evaluation • Verification of ACAT reporting • Analysis of ACAT and secondary data • Content analysis of KII and FGD notes from team members and assimilation into the Draft Evaluation Report • Synthesis and triangulation of data gathered to inform findings, conclusions, and recommendations for the Draft Evaluation Report • Quantitative data: descriptive analysis, summary tables, and infographics (disaggregated by value chain, geographic location, gender, etc.) • Qualitative data: content/thematic analysis, i.e., grouping, finding patterns 52 ACAT – Evaluation Matrix No. Evaluation Question Data Sources Data Collection Method Data Analysis EQ2 What improvements to the management structure might be needed to ensure that ACAT achieves the activity goal and objectives? [Efficiency: Qualitative] • ACAT documents: annual and quarterly reports; M&E reports, assessments, studies, and surveys • Documents related to ACAT’s direct beneficiaries: agricultural associations, cooperatives, SMEs, entrepreneurs, BSDPs, financial institutions, i.e., related to grants, TA, and training • USAID Mission staff • ACAT key management personnel • Document search and review sourced from USAID and ACAT work plans, studies, assessments, M&E data, and information • Secondary data search and analysis from publications in the public domain – government, donors, private organizations, and NGOs, among others • KIIs in Dushanbe and in ACAT’s target districts in Khatlon with direct beneficiaries and other stakeholders including local authority representatives • FGDs conducted with sub-grantees, Government of Tajikistan, business and agriculture associations, and key private sector operators in the region’s value chain network • Qualitative thematic analysis around the EQs • Comparative analysis of ACAT performance comparing target vs. actual (by quarter and year) EQ3 Midway, what about ACAT’s market system approach to improving agribusiness competitiveness in Tajikistan is working well, and what is not? [Effectiveness: Qualitative mainly + some Quantitative aspects] • Beneficiary groups and stakeholders, ACAT M&E data, reports, assessments, studies, etc. • Desk review of project documents and reports • KIIs and FGDs with ACAT staff and stakeholders, including from the USAID Mission • Systematic document review • Qualitative thematic analysis around the EQs • Comparative analysis of target vs. actual results (disaggregated by region, value chain, beneficiary type, etc.) 53 ACAT – Evaluation Matrix No. Evaluation Question Data Sources Data Collection Method Data Analysis EQ4 What should be improved to ensure scaling up and sustainability of the market system approach? [Sustainability + Effectiveness: Qualitative mainly + some Quantitative aspects] • Document search and review sourced from USAID and ACAT work plans, studies, assessments, M&E data, and information • Secondary data search and analysis from publications in the public domain – government, donors, private organizations, and NGOs, among others • KIIs in Dushanbe and in ACAT’s target districts in Khatlon with direct beneficiaries and other stakeholders including local authority representatives • FGDs conducted with sub￾grantees, Government of Tajikistan, business and agriculture associations, and key private sector operators in the region’s value chain network • Other donor activity including the World Bank, ADB, and IFAD, among others • Document search & review sourced from USAID & ACAT (work plans, studies, assessments, M & E data and information • Secondary data search & analysis from publications in the public domain – government, donors, private organizations, NGOs among others • KIIs in Dushanbe and in ACAT’s target districts in Khatlon with direct beneficiaries and other stakeholders including local authority representatives • FGDs conducted with sub-grantees, Government of Tajikistan, business and agriculture associations, and key private sector operators in the region’s value chain network • Mini-surveys of beneficiaries/ stakeholders, partners, and loan recipients and direct observations where and when deemed possible within the constraints of COVID-19 prevailing at the time of the evaluation • Systematic document review • Qualitative thematic analysis around the EQs • Comparative analysis of target vs. actual results (disaggregated by region, value chain, beneficiary type, etc.) • Analysis of ongoing or planned international development projects to assess the level of any synergy between their work and that of ACAT 54 ANNEX 5: LEVEL OF ACHIEVEMENT OF SET TARGETS BY ACAT AS OF THE END OF FY3 Objective Indicator Definition Baseline LOP Target LOP Results as of Y3 % of LOP Target Achieved as of the End of Y3 Current Status of Achievement of the End-of-Program Target O1 Percentage increase in sales from value addition and higher value products 0% 25% 64% 156% overachieved O1 Number of individuals in the agriculture system who have applied improved management practices or technologies with USG assistance 0 18,525 16,751 90% partly achieved O1 Quantity (in metric tons) of products meeting regional and international trade standards and certification requirements (Horticulture) 1,417 10% average increase 39.64% increase and 18.32% average increase 23.64% partly achieved O1 Quantity (in metric tons) of products meeting regional and international trade standards and certification requirements (Dairy) 231 6% average increase 117% increase and 58.5% average increase 124% overachieved O1 Number of individuals participating in USG food security program 0 28,500 20,609 72% partly achieved O1 Number of for-profit private enterprises, producers, organizations, women’s groups, youth groups, trade, and business associations and CBOs receiving USG food security related organizational development assistance 0 1,400 798 57% partly achieved O1 Number of high-value and added-value products developed due to ACAT facilitation and support 0 70 62 89% partly achieved O2 Number of public-private partnerships formed as a result of USG assistance 0 40 50 125% overachieved O2 Number of firms receiving USG-funded technical assistance to export 0 175 146 83% partly achieved O2 Number of organizations with increased performance improvement with USG assistance 0 356 120 34% partly achieved O2 Percent of USG-assisted organizations with improved performance [IM-level] 0% 65% 32% 49% partly achieved O2 Number of firms meeting regional and international trade standards and certification requirements 32 125 66 53% partly achieved O2 Number of services available in targeted value chains 17 25 27 104% overachieved 55 Objective Indicator Definition Baseline LOP Target LOP Results as of Y3 % of LOP Target Achieved as of the End of Y3 Current Status of Achievement of the End-of-Program Target O3 Number of individuals in full time, part-time, and seasonal labor 15,349 18,112 9,290 51% partly achieved O3 Value of new USG commitments and private sector investment leveraged by the USG to support food security and nutrition 0 $5,000,000 $2,851,319 57% partly achieved O3 Value of agriculture-related financing accessed as a result of USG assistance 0 $500,000 $842,857 169% overachieved Cross￾Cutting Percentage of female participants in USG￾assisted programs designed to increase access to productive economic resources (assets, credit, income, or employment) 0% 40% 104% 104% overachieved Cross￾Cutting Percentage of participants in USG-assisted programs designed to increase access to productive economic resources who are youth (15-29) [IM-level] 0% 20% 40% 40% overachieved 56 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523