This publication was produced at the request of the United States Department of Agriculture. It was prepared independently by Cynosure Consultants. Foreign Agricultural Service, United States Department of Agriculture reign Agricultural Service, United States Department Source: Winrock International Horticulture Takes Root in Pakistan’s Agriculture Sector, Pakistan Agriculture Development (PAD) (https://www.winrock.org/project/pad/) PAKISTAN FOOD FOR PROGRESS Agriculture Development Project (PAD) MID-TERM EVALUATION JULY 2020 2 Mid-Term Evaluation of Pakistan Agriculture Development (PAD) Project The Pakistan Agriculture Development (PAD) Project is a five year program (September 2016 to September 2021) implemented by Winrock International and funded by the Food for Progress Program of the United States Department of Agriculture (USDA). The agreement was signed under the Food for Progress Act (FFPr), to address the issues which are prevailing in four agricultural crops, namely red chili, tomato, date and banana. Following the monetization of crude degummed soybean oil and USDA approval of the project baseline study, implementation of Pakistan Agriculture Development (PAD) project began in April 2018. Project Duration: September 2016 to September 2021 Implemented by: Winrock International Agreement Number: FCC-391-2016/003-00 Evaluation Authored by: Cynosure Consultants Lead Evaluator Umm e Zia Contributors Ehtesham ul Hassan Wajahat Hussain Khan Hamda Arif 3 TABLE OF CONTENTS 1. INTRODUCTION & BACKGROUND ......................................................................................... - 16 - 1.1. COUNTRY CONTEXT ........................................................................................................ - 16 - 1.2. ABOUT PAD...................................................................................................................... - 16 - 1.2.1. PROJECT FRAMEWORK & OBJECTIVES .................................................................. - 17 - 1.2.2. SCOPE OF THE PROJECT.......................................................................................... - 18 - 1.2.3. STAKEHOLDERS/PARTNERS .................................................................................... - 18 - 1.2.4. BENEFICIARY GROUPS............................................................................................. - 18 - 2. ABOUT THE EVALUATION....................................................................................................... - 20 - 2.1. PURPOSE & CRITERIA ...................................................................................................... - 20 - 2.2. METHODOLOGY............................................................................................................... - 21 - 2.2.1. DATA COLLECTION TOOLS...................................................................................... - 21 - 2.2.2. SAMPLING METHODOLOGY.................................................................................... - 21 - 2.2.3. PROCESS ................................................................................................................... - 22 - 2.2.4. CHALLENGES ............................................................................................................ - 23 - 3. EVALUATION FINDINGS.......................................................................................................... - 25 - 3.1. RELEVANCE ...................................................................................................................... - 25 - 3.1.1. PROJECT DESIGN ..................................................................................................... - 26 - 3.1.2. DESIGN CHANGES.................................................................................................... - 27 - 3.2. EFFICIENCY....................................................................................................................... - 28 - 3.2.1. TIMELINESS ............................................................................................................... - 28 - 3.2.2. PROJECT MANAGEMENT ........................................................................................ - 28 - 3.2.3. MONITORING........................................................................................................... - 29 - 3.2.4. FINANCE ................................................................................................................... - 30 - 3.2.5. STAKEHOLDER PERFORMANCE .............................................................................. - 31 - 3.2.5.1. PERFORMANCE OF PRIVATE SECTOR PARTNERS ............................................. - 31 - 3.2.5.2. PERFORMANCE OF PUBLIC SECTOR PARTNERS................................................ - 32 - 3.3. EFFECTIVENESS................................................................................................................. - 33 - 3.3.1. ACTIVITY I – CREDIT ................................................................................................ - 33 - 3.3.2. ACTIVITY II – INFRASTRUCTURE GRANTS.............................................................. - 34 - 3.3.3. ACTIVITY III – INPUT GRANTS................................................................................. - 35 - 3.3.4. ACTIVITY IV – MARKET ACCESS ............................................................................. - 37 - 3.3.5. ACTIVITY V – PUBLIC INFORMATION CAMPAIGN................................................ - 38 - 4 3.3.6. ACTIVITY VI AND VII – TRAINING IN IMPROVED AGRICULTURE PRODUCTION TECHNIQUES AND POST-HARVEST HANDLING AND PROCESSING..................................... - 39 - 3.3.6.1. TIER I ..................................................................................................................... - 40 - 3.3.6.2. TIER II .................................................................................................................... - 41 - 3.3.7. ACTIVITY VIII – CAPACITY BUILDING OF PRODUCER GROUPS ........................... - 41 - 3.3.8. ACTIVITY IX – AGRO-DEALERS TRAINING............................................................. - 42 - 3.3.9. ACTIVITY X – TRAINING IN SANITARY AND PHYTOSANITARY .......................... - 42 - 3.4. IMPACT ............................................................................................................................. - 43 - 3.5. SUSTAINABILITY............................................................................................................... - 44 - 4. CONCLUSION & RECOMMENDATIONS .................................................................................. - 46 - 4.1. CONCLUSION ................................................................................................................... - 46 - 4.2. RECOMMENDATIONS ...................................................................................................... - 46 - 4.2.1. APPROACH TO IMPLEMENTATION......................................................................... - 46 - 4.2.1.1. MODIFICATION OF ACTIVITIES........................................................................... - 46 - 4.2.1.2. STAKEHOLDER COLLABORATION ...................................................................... - 49 - 4.2.1.3. PROJECT MANAGEMENT .................................................................................... - 49 - 4.2.2. PROJECT DESIGN ..................................................................................................... - 50 - 5 LIST OF TABLES Table 1: Geographic Scope of PAD Project............................................................................ - 18 - Table 2: List of Active Sub-Awardees and Stakeholders ................................................................ - 18 - Table 3: Summary of Field Activity ................................................................................................ - 21 - Table 4: Project Expenditure at Mid-Term...................................................................................... - 31 - LIST OF FIGURES Figure 1: PAD FFPr Result Framework – Strategic Objective I – Increased Agricultural Productivity .............................................................................................................................................................. 48 Figure 2: PAD FFPr Result Framework – Strategic Objective II – Improved Trade of Agricultural Products ................................................................................................................................................ 48 COVER PHOTO CREDITS: First Cover Page: Date Farming in Pakistan; Photo Courtesy: Syed M. Rafiq, Flickr Second Cover Page: Tomato Cultivation; Photo Courtesy: Technology Times Third Cover Page: Banana growers using organic fertiliser to get better yields; Photo Courtesy: The News International Fourth Cover Page: Spice Value Chain Development; Photo Courtesy: Asian Development Bank 6 ABBREVIATIONS & ACRONYMS AAG Ali Akbar Group BV Bureau Veritas CI Custom Indicators CBOs Community-Based Organizations COP Chief of Party CDSO Crude Degummed Soybean Oil DAC/OECD Development Assistance Committee of the Economic Cooperation and Development DCOP Deputy Chief of Party DPRI Date Palm Research Institute EOI Expression of Interest FFPr Food for Progress FGD Focus Group Discussion GAP Good Agricultural Practices GDP Gross Domestic Product GOP Government of Pakistan HACCP Hazard Analysis and Critical Control Points IDI In-Depth Interview IVR Interactive Voice Response KII Key Informant Interview LGF Loan Guarantee Fund LOP Life of Project MFI/MFBs Microfinance Institutions/Banks MT Metric Ton MTE Mid-Term Evaluation O&M Operations and Management PARC Pakistan Agricultural Research Council PMP Performance Monitoring Plan 7 QA Quality Assurance RFA Request for Application SBDs Solar Bubble Dryers SBL Sprouts Biotech Laboratories SI Standard Indicators SMS Short Message Service SPS Sanitary and Phytosanitary TIC Testing, Inspection, and Certification USDA United Stated Department of Agriculture WI Winrock International 8 LIST OF ANNEXES Annex 01: Projects Results Framework Annex 02: Evaluation Matrix Annex 03: Data Collection Tools Annex 04: Sampling Methodology Annex 05: List of Key Informants Met/Interviewed Annex 06: Documents Reviewed Annex 07: Pad Organogram Annex 08: Activity Wise Financial Expenditure Annex 09: Effectiveness and Impact Annex 10: Performance Indicator Annex 9 EXECUTIVE SUMMARY Funded by the USDA Food for Progress (FFPr) Act, PAD project was signed in September 2016 based on a cooperative agreement between the United States Department of Agriculture (USDA) and Winrock International (WI), and its full-scale implementation started in mid￾2018. The project duration being five years, PAD is scheduled to close in September 2021. A mid-term evaluation (MTE) of the project was conducted between January and April 2020, using the DAC/OECD criteria of Relevance, Effectiveness, Efficiency, Impact1 , and Sustainability. The project is financed through monetization of 25,000 metric tons (MT) of crude degummed soybean oil (CDSO), with a funding of USD 17,898,779 available for implementation. PAD focuses on strategic value chain development of four horticulture crops, including dates, banana, tomato, and red chili in 12 priority districts of the Sindh and Punjab provinces of Pakistan. The project’s two strategic objectives, aligned with the USDA’s Food for Progress (FFPr) program, are: i) Increased agricultural productivity; and ii) Expand trade of agricultural products. As of March 2020, PAD has reached approximately 24,000 direct beneficiaries that include: 22,875 producers, 1,129 farm workers (496 male, 633 female), 64 agro dealers and suppliers, 124 traders, and 02 private sector partners namely Haji Sons and Sprouts Biotech laboratories. The MTE was carried out in a consultative manner, using both qualitative and quantitative tools. Following a Desk Review of key PAD project documents, project stakeholders for all four target crops were interviewed using Key Informant Interviews, In-Depth Interviews, and Focus Group Discussions, covering 112 respondents. In addition, a survey of 834 sampled beneficiaries across all activities was undertaken to gather quantitative data. Analyzed data from all sources was triangulated and reported in accordance with the evaluation criteria of Relevance, Efficiency, Effectiveness, Impact, and Sustainability. Major limitations faced during the MTE exercise included the refusal of some beneficiaries (agro-dealers) to participate in the survey due to their dissatisfaction with the project interventions; and the onset of country-wide lockdown due to COVID-19 towards the end of the data collection activity in mid-March. However, the MTE team kept a significant margin in the sampled number of respondents and the actual sample size for each cluster in order to meet the survey target. Whereas, in light of COVID-19, the interview methodology for KIIs was modified by conducting online interviews using video/phone calls. The MTE team found that with respect to relevance, the project’s activities and objectives were in line with the Government of Pakistan’s (GoP) policy measures at the Federal and Provincial levels, as well as the USDA’s Food for Progress (FFPr) results framework and strategic objectives. Similarly, the project design is also responsive to the needs of all direct 1 Since this is not an impact evaluation with a counterfactual, the project’s contributions towards its objectives were analyzed to inform analysis on impact. 10 beneficiary groups, including small and medium-sized farmers, agro-dealers and traders residing in the target districts of Punjab and Sindh provinces. A review of the project design and results framework showed that all activities were inter￾related and responsive to the beneficiary needs. However, in response to the implementation context, the project made subsequent changes to some activities after the initial design. Of these, major changes include: engagement of additional stakeholders (Sprouts Biotech Laboratories (SBL) for provision of tissue cultured banana plants and Ali Akbar Group (AAG) for establishment of demonstration plots at farmer fields), dropping of the Loan Guarantee Facility (LGF) at the time of project signing, and alternative approaches to formation of producer groups (due to policy implications of the GOP) and procurement of inputs (to establish private linkages). The project’s efficiency was assessed by assessing operational factors including timeliness, finance, project management, monitoring, and stakeholder performance. While the project management team was found to be well staffed, the lean structure of field teams engaged by both WI and sub-grantees affects the project’s ability to deliver at the farm-level. With respect to timeliness, the project faced nearly 18 months delay at the start due to approvals from USDA and the GOP, while further delays are expected because of the current uncertain COVID-19 crisis. In total, the project is expected to operate at reduced capacity for two years, i.e. 40% of the planned five-year duration. Of the total funds available for project implementation USD 17,898,779, a total of 5,898,963 (29%) has been spent, reflecting both direct and indirect costs. The major reasons for low financial delivery have been a) Devaluation in the PKR since project approval; b) Delays in project start; and c) Difficulties in activity implementation, including Credit, Farmer Groups, and uptake of Infrastructure Grants. Project monitoring is carried out at multiple levels by the PAD project and sub-grantees, in accordance with the monitoring plan laid out in the Cooperative Agreement. While the project utilizes efficient monitoring systems and bases planning decisions on regularly collected monitoring data, there is a critical need to track the quality of sub-grantee progress. Finally, private sector sub-grantees and stakeholders have made significant contributions to the project by creating synergies through existing programs and allowing the project to reach a larger audience in a cost-effective manner. However, public sector stakeholders have remained uninvolved primarily due to hurdles caused by bureaucratic red tape. The project’s effectiveness was measured in terms of achieving its set targets against all activities as well as implications for production, marketing, and income. Under Activity I – Agricultural Credit, against the life of project (LOP) target of 6,800 individuals obtaining agricultural credit worth USD 3.4 million, the project has only been able to link 126 farmers to formal credit amounting to USD 39,842 thereby achieving 2% and 1.2% of its LOP targets, respectively. The major reasons for this low performance is cancellation/dropping of the LGF that was envisioned in the initial project design, the high cost 11 of formal borrowing, existing farmer-contractor relationships which are difficult to break away from, and cultural/religious beliefs of farmers which reflect negatively on interest/usury. Under Activity II, the project aimed to provide in-kind grants for equipment and infrastructure worth USD 4.2 Million. Thus far, the project has delivered grants to 33 grantees worth USD 379,421, thereby meeting 9% of its LOP goal. However, grants worth USD 1,788,206 for various projects, including 39 irrigation grants, 10 GrainPro products and 11 other infrastructure grants are at various stages of completion in addition to a further around USD 2,680,000 in the pipeline and are expected to be delivered during 2020. Since most of the delivered infrastructure have been set up only recently, their effectiveness will become clear after one or two crop cycles. Of the grants delivered thus far, date storage and drip irrigation have the highest potential for effectiveness based on productivity improvement and enterprise profitability. The project is well behind its LOP target of leveraging investment of USD 1.5 million under this activity. Activity III –Under this activity, the project aimed to provide grants for inputs worth USD 761,951 during the life of the project. Thus far, two grantees have been provided support who have in turn delivered inputs to 2,118 farmers (2,055 tomato and red chili farmers provided seedling trays and peat moss; and 63 farmers provided banana tissue cultured plants), while utilizing USD 84,790 (11%) of the total allocated funds. In the interest of establishing sustainable supply chains, the original approach of procure-deliver was changed to supplier￾farmer linkages. However, the lack of suppliers accessible to smallholders has affected progress and also restricted the scope of activity to only planting material for banana, tomato, and red chili. While the outcome of banana plants has been greatly appreciated and resulted in tremendous demand, the supply of tissue culture plants is limited. Similarly, most chili farmers were new to nursery raising in seed trays and plan to continue this method in the future. However, being conditional upon the purchase of tomato/chili seed from PAD sub-awardee, the structure of grants related to seed trays was found to be non-conducive to beneficiary interest. Activity IV – The project aimed to assist 20 agreements during LOP in order to facilitate buyer￾seller relationships and private partnerships. Overall, one banana and 8 date agreements (i.e. 45% of the target) have been linked to higher paying wholesale/retail markets, having resulted in the sale of 567 MT bananas (value USD 62,016) and 158 MT dates (value USD 143,100). Further, these linkages have been operational for only one year and rely mostly on one or two clients. Therefore, the MTE team believes that it is premature to determine their long-term effectiveness at this point. Activity V – Thus far, activities have included a Public Awareness Campaign through Telenor-Pakistan, a cellular company; and text messages focusing on Good Agricultural Practices (GAP) disseminated to PAD trained farmers through an SMS portal, and WhatsApp groups of farmers who are interested in exchanging information and experiences. Since its launch in September 2018, the campaign has gained 900,233 subscribers not only in the project districts but also across Punjab, resulting in achievement of 226% LOP project target. 12 Activity VI and VII – Against an LOP target of 35,820, a total of 20,060 individuals (56%) have been trained in improved agriculture production techniques, and post-harvest methods. As a result of applying the trainings, 65% respondents have witnessed an increase in income. While the MTE team concludes that the trainings have been effective, they also found a certain level of disappointment among the beneficiaries for failing to receive any input grants. This issue can play a role in long-term effectiveness, since the surveyed Master Trainers said that the inputs recommended during the trainings are not available (24%) or are expensive (19%). Also, for this activity, another 139 female workers have been provided awareness on GAP at the time of nursery raising and transplanting of plants; and another 494 on harvesting and field handling techniques. Activity VIII – Farmer groups could not be formed as originally designed due to restrictions imposed on Community-Based Organizations (CBOs) due to government policies. Consequently, the project organized 133 informal groups against the LOP target of 170 groups (78% of target). Although these groups have facilitated outreach to individual farmers under various activities, including training, marketing, and credit, they have not been able to attain their original purpose of collective action such as procurement and marketing. Activity IX – The project has trained 64 agro-dealers (116% of LOP target) through three one-day seminars in Sindh and one in Punjab. Seed storage, selection, and germination was found to be the most valued component of the training by most participants. However, the training was largely ineffective with regards to adoption of other improved technologies and methods, such as seeds and seed trays by these dealers. The major reason for this was the preference of dealers to have business relations with suppliers other than the sub-contractor. Activity X – The activity has primarily focused on farmers, and 2,143 individuals (52% of LOP target) have been trained in Sanitary and Phytosanitary (SPS) measures. As a result of applying these practices, 65% have been able to market higher volumes, 42% experienced lower product wastage, and 16% reported getting access to new markets. Consequently, 73% of those who applied the training have reported an increase in income. The MTE team found that despite a late start, the PAD project has had significant contributions in the form of trainings, infrastructure grants, input grants, and marketing. Conversely, credit, agro-dealers training, and producer groups have not shown positive impact. Having said that, the project’s influence is spread thinly due to the broad range of activities undertaken by the project, thereby weakening its capacity for transformational change in any given crop or cluster. Furthermore, it is important to note that the worldwide lockdown from COVID-19 is likely to limit the contribution at least during the period March to July, 2020. Sustainability of the PAD project has been assessed with reference to continuation of activities by existing beneficiaries as well as through upscaling and replication. In general, it was observed that while the project’s effectiveness and contribution have been high for selected activities, this contribution may be short lived unless market readiness and linkages are prioritized. Based on the above assessment, it is recommended that going forward the project should modify focus of all activities by gearing them towards development of sustainable market 13 linkages, while utilizing the 71% of the budget remaining. In particular, it is recommended that: - Credit: Due to the limited time available and financial sector dynamics in the country, this activity be cancelled, and funds reallocated to other activities with more significant contribution; - Infrastructure Grants: The project should continue to provide infrastructure grants with the intention to develop product-based clusters. Moreover, the grant to strengthen tissue culture facility at PARC is cancelled and funds are reallocated to other potential organizations producing tissue culture plants. - The project should continue promoting banana tissue cultured plants. Also, farmer should not be compelled to buy seed from a certain source in order to access inputs for chili and tomato nursery raising. And, the list and categories of tools and implements under this activity should be broadened. - All marketing activities being carried out should continue and be up scaled. Also, the option of establishing a public sector company for promotion of the date sector should be explored. - PAD should support Telenor to collaborate with other mobile phone companies for delivery of messages or directly use networks of other companies for message delivery to reach a broader audience. - The project should switch its approach to the proven methods of Farmer Field School (FFS) and demonstration plots. Local agro-dealers should be provided hands on training. And, interested producers with potential should be encouraged and supported to get quality certifications. - Furthermore, current modality of agro-dealers training should be revamped in case continued, including selection criteria and training content, etc. Small and medium scale local agro-dealers should also be contacted through a general announcement/advertisement process and selected to participate in the training following an established set of eligibility criteria. Training should be provided by a qualified company/department through close collaboration and monitoring of PAD project. It is recommended that this training must focus on practical methods of seed selection, storage, and germination; plant diseases, and fertilizer regimes, etc. - In view of the performance gaps observed in the partnership with one of the sub-awardees, it is recommended that in case the activities are continued, the project replaces the company with alternative partner(s). In the interest of transparency, these partner(s) should be selected based on competitive bids and their performance should be closely monitored by the PAD project team/third party monitors. - Future Strategy for PAD: Further, to ensure that the project has sufficient time to implement these measures, it is strongly recommended that the project is granted a no-cost extension of 12 – 18 months, the budget be revised for various activities in order to align them with the newly emerged priorities post MTE, and staffing structure should be reviewed in accordance with these changes. Moreover, it is recommended that future project design incorporate the following elements: i) The time taken for key procedural activities required to start a project, e.g. country-level registrations or approval of baseline activity by the USDA should not be counted towards 14 project duration; ii) A well-articulated exit strategy is required; iii) Future projects must design activities which are inclusive of both men and women farmers. 15 16 1. INTRODUCTION & BACKGROUND 1.1. COUNTRY CONTEXT Agriculture is inarguably the largest sector of Pakistan’s economy, accounting for 24% of the GDP, and together with agro-based products fetches 80% of the country’s total export earnings2 . In addition, half of the country’s labor force is engaged in this sector3 . Within agriculture, horticulture is an important sector accounting for exports worth more than USD 600 million in fiscal year 2015-164 . Pakistan is the fourth largest producer of dates in the world5 . Whereas, banana production despite having tremendous potential, particularly in Sindh can hardly meet the demand of the domestic market thus limiting export opportunities. On the other hand, red chili is also a major crop produced in Pakistan, making the country its fourth largest producer in the world. However, in 2019 there was a 40% drop in the production of red chili owing to issues resulting from climate change and water shortage6 . Tomato is another horticulture crop of significant economic value in Pakistan since it is a relatively short duration crop and gives a high yield. Yet, there are prevalent issues with tomato yield and produce quality often rendering exports uneconomical. Notably, approximately 25% to 40% of horticultural crops are lost due to mishandling, spoilage, and pest infestation. There are issues relating to compliance, certification according to international standards, traceability, farm management, perishability, cold chain, storage, and marketing. Banks credit to this sector is only 4-5%. The disbursement to horticulture was only around PKR 6.5 billion in 2006-07 against total agriculture disbursement of PKR 169 billion during the period7 . In order to promote the growth of this industry, it is essential that special attention be given to encourage and facilitate the producers and processors to utilize new technologies and techniques; develop and implement export marketing strategies; create an export oriented environment facilitated through producers and quality standard through regulations and incentive schemes; attract local and foreign investment; facilitate in setting up of necessary material and quality infrastructure; develop linkages and networking with relevant institutions; commercial linkages with local and international companies; technology transfer; and sub￾contracting. 1.2. ABOUT PAD Acknowledging these gaps and opportunities in the sector, Winrock International Institute for Agricultural Development (WI) signed a five-year Cooperative Agreement with the U.S 2 Source: http://www.agripunjab.gov.pk 3 Source: http://www.pbs.gov.pk/content/agriculture-statistics 4 Source: https://tribune.com.pk/story/1409953/horticulture-exports-can-grow-7b-rd-support/ 5 Source: http://www.sbp.org.pk/Guidelines/Horticulture/Horticulture_Guideline.pdf 6 Source: https://www.trtworld.com/business/climate-change-threatens-red-chilli-harvest-in-pakistan-s-sindh-province￾33436 7 Source: http://www.sbp.org.pk/Guidelines/Horticulture/Horticulture_Guideline.pdf 17 Department of Agriculture (USDA), under the Food for Progress Act, to address the issues in selected agricultural crops through the Pakistan Agriculture Development (PAD) project. The project operations started in October 2016; however full-scale field activities started in April 2018 following the approval of the baseline study by USDA. The project is implemented in collaboration with sub-awardees Haji Sons and Bureau Veritas (BV), as well as local banks, mobile phone companies, and other private sector partners. The project primarily addresses the problems concerning productivity and post-harvest losses in four horticulture value chains. 1.2.1. PROJECT FRAMEWORK & OBJECTIVES WI has carried out the monetization proceeds of 25,000 MT of CDSO to carry out the following activities through the PAD project: Activity I: Financial Services: Facilitate Agriculture Lending Activity II: In-kind Grants: Equipment and Infrastructure Activity III: In-kind Grants: Inputs Activity IV: Market Access: Facilitate Buyer Seller Relationships and Private Partnerships Activity V: Public Information Campaign: Disperse Improved Market Information Activity VI: Training: Improved Agriculture Production Techniques Activity VII: Training: Post Harvest Handling and Processing Activity VIII: Capacity Building: Producer Groups/Cooperatives/Trade Associations Activity IX: Inputs: Develop Agro Dealers/Input Suppliers and Equipment Suppliers Activity X: Training: Sanitary and Phytosanitary Standards. These project activities are categorized under the following two USDA FFPr Strategic Objectives, including:  Strategic Objective I: Increased agricultural productivity by building the capacity of producers in improved productivity and profitability; training producers in improved production techniques; post-harvest handling, marketing, sanitary and phytosanitary practices; and providing grants and loans for equipment and agriculture inputs.  Strategic Objective II: Expanded trade of agricultural products by developing both domestic and export market linkages; facilitating trade relationships, researching export opportunities; promoting a coordinated and strategic approach to building market share by associations and promoting food safety issues and requirements. The project’s results framework under these two strategic objectives is provided in Annex 01. 18 1.2.2. SCOPE OF THE PROJECT The provinces of Punjab and Sindh were selected for PAD because they are both major agricultural producers in each of the target value chains of banana, dates, tomato, and red chili. PAD focuses on 12 districts and identified target communities where farmers are geographically clustered so that activities can be concentrated to ensure greater efficiency and economies of scale (Table 1). Table 1: Geographic Scope of PAD Project Province Target Districts Sindh Mirpurkhas Umerkot Khairpur Thatta Tando Allah Yar Hyderabad /Matiari Punjab Multan Khanewal Muzaffargarh Lodhran Bahawalpur Sheikhpura PAD has reached approximately 24,000 direct beneficiaries through March 2020 that include: 22,875 producers, 1,129 farm workers (496 male, 633 female), 64 agro dealers and suppliers, and 124 traders, and 02 private sector partners namely Haji Sons and Sprouts Biotech Laboratories. 1.2.3. STAKEHOLDERS/PARTNERS PAD implementation is also assisted by sub-awardees and collaborators, of which the most active ones are listed in Table 2 as follows: Table 2: List of Active Sub-Awardees and Stakeholders Stakeholder/Partner Type Organization Role in the Project Input Supplier/Sub￾Awardee Haji Sons Provides technical trainings to PAD farmers (chili and tomato value chains); Promotes module-based seedling production for red chili and tomato crops Sub-Awardee Bureau Veritas Trainings in sanitary, phytosanitary and HACCP standards to farmers in all four value chains Input Supplier Sprout Biotech Labs Provides banana tissue cultured plants to farmers at subsidized rates Input Supplier Ali Akbar Group (AAG) To establish demonstration plots at farmer fields with the goal of further adoption of GAP Private Sector/Telecom Telenor Pakistan Disseminates digital content related to agricultural extension advice to farmers 1.2.4. PROJECT BENEFICIARIES Beneficiaries include small and medium-sized farmers (individuals and groups), agro-dealers, and traders in Punjab and Sindh provinces. Small farmers are important participants because they constitute the largest group of agricultural producers in the target areas. 19 20 2. ABOUT THE EVALUATION 3. 2.1. PURPOSE & CRITERIA The purpose of the MTE is to assess the effectiveness of the programmatic approaches of PAD activities. The evaluation is based on the criteria of DAC/OECD which includes relevance, efficiency, effectiveness, impact and sustainability. The services of Cynosure Consultants (Pvt.) Ltd.8 were hired to undertake the MTE of the PAD project. While undertaking the research study, a consultative and participatory approach was adopted together with employing mixed methodologies, combining qualitative and quantitative data to capture information relating to evaluation objectives. Data collection for the evaluation was undertaken during March-April, 2020. The cut-off date for project achievements/progress related data provided in the report is March 31, 2020. The evaluation collectively focuses on the assessment of the following key evaluation components: (i) Assess and recommend any successful (pilot) interventions that can be scaled up for greater impact in the future across the sector; (ii) discuss the obstacles and challenges PAD is facing, and how these are currently being addressed and recommendations for how to better address these issues through the remaining life of the project; (iii) progress on the implementation of activities and key interventions; (iv) project performance with respect to intermediate results and indicator targets to date; (v) key implementation gaps if they exist and what causes them; (vi) evidence of where project human, financial, and material resources are being deployed efficiently and effectively and where adjustments may be appropriate; and (vii) unforeseen opportunities or challenges that may require revisions of the project strategy or redirection of project resources. The findings and recommendations are based on qualitative and quantitative data compiled by the MTE team using primary and secondary sources. An evaluation matrix comprising of key evaluation questions and data collection and analysis methods is presented in Annex 02. The target audience and associated utility for the mid-term evaluation include: i) Project staff and management will use the MTE to determine whether the project is successfully reaching its targets and achieving outcomes and assess which interventions need to be continued or modified for the remaining life of the project; ii) USDA will use the MTE to assess whether the project is on track in accordance with the established objectives and targets, and contribution to the Department’s own institutional research and learning; and iii) Other stakeholders, including beneficiaries, implementing partner organizations, Government of 8 The core MTE team consisted of 7 members, including: Ms. Umm e Zia (Team Lead), Ms. Munazza Zia (Gender & Evaluation Expert), Mr. Ehtesham ul Hassan (Value Chain Expert – Punjab), Mr. Majid Khan (Value Chain Expert – Sindh), Mr. Wajahat Hussain Khan (Value Chain Expert/Senior Facilitator – Sindh), Mr. M. Ali Raza (Statistical Analysis Expert), and Mr. Sajjad A. Khan (Survey Expert/Supervisor). 21 Pakistan, and other US government assisted projects in Pakistan, etc. to learn from the project results for incorporation in future programming. 2.2. METHODOLOGY 2.2.1. DATA COLLECTION TOOLS The data collection tools developed by the Consultant are based on the principles of participatory techniques. The tools comprised of i) Key Informant Interviews (KIIs); ii) In￾Depth Interview (IDIs); iii) Focus Group Discussion (FGD) and iv) Beneficiary Survey. The above-mentioned tools are provided in Annex 03 and designed as such to be user friendly and provide a combination of qualitative and quantitative information. The data collection tools were translated into Urdu language for ease of comprehension. A summary of the planned field activity and sample sizes is provided below (Table 3). Table 3: Summary of Field Activity Data Collection Tool Respondent Type Activity Numbers No. of Respondents/Groups Beneficiary Survey Men and Women Beneficiaries of Various Activities Against Each Commodity I,III,VI,VII,X 834 FGD Men and Women Beneficiaries of Various Activities III,VI,VII,VIII,X ~90 / 15groups IDI Men and Women Beneficiaries of Various Activities II,IV,IX 15 KII Institutional Stakeholders/Partners V 7 Total No. of Respondents 946 2.2.2. SAMPLING METHODOLOGY In order to assess the quantitative contribution of different interventions, a beneficiary survey was conducted in 09 selected target districts of both provinces with 796 beneficiaries. The survey sample size and geographic scope were determined through an in-depth analysis of the project data. The survey was aimed to reach Tier-1 and Tier-29 farmers corresponding PAD activities carried out with the most significant number of recipients, i.e. Activity I, Activity VI, Activity VII, and Activity X. Since the project beneficiaries of these activities are classified according to their respective crops/commodities, therefore, stratification was done crop wise and then further classified into clusters at district level. Detailed sampling methodology is presented in Annex 04. 9 Farmers directly trained by PAD represent Tier-1 and are referred to as Master Trainers. Some of these Master Trainers transfer knowledge to other farmers in their respective localities through on-farm training sessions and experiential learning. Those farmers trained by Master Trainers are considered Tier-II farmer beneficiaries. 22 In-depth interviews (IDIs) were aimed at beneficiaries of PAD Activities II, IV, and IX as these were individual farmers and agro-dealers which were fewer in number as compared to number of beneficiaries of other project activities. Focus group discussions (FGDs) were aimed at beneficiaries of PAD Activities III and VIII. Although activity III beneficiaries were also aimed to be covered in the Survey, however, to avoid missing its beneficiaries during simple random sampling due to its smaller sample, FGDs were specifically held with selected beneficiaries. In addition, under activity VIII the farmers were first formed into informal groups and then introduced to other project activities. Therefore, FGDs were conducted with these groups in order to assess the functionality, cohesion, and sustainability. Key Informant Interviews (KIIs) were held with stakeholders with direct or indirect involvement in both the formulation and implementation of the PAD project in the last two years. In addition to meeting with the PAD project staff and management, KIIs were held with private sector partners, financial institutions, and sub-awardees with which activities have been initiated. List of key informants met/interviewed is attached in Annex 05. 2.2.3. PROCESS Upon the award of a contract, a kick-off meeting was held between the Consultant and the PAD team to discuss expectations about key deliverables and other associated responsibilities of the Consultant and PAD staff, as well as agree on coordination mechanisms of the assignment. The evaluation process started10 with a comprehensive desk review of available data and documents on the program. The desk review facilitated a clear understanding of the project and enabled an effective evaluation design. The documents reviewed are shared in Annex 06. Taking into account the extensive reach of the stakeholders in a large geographic area, a detailed schedule for field visits was devised so that stakeholders participating in the assignment were given advance notice of the Consultant’s visit. WI was requested to help organize the interaction with stakeholders and extend all reasonable support necessary to facilitate data collection activities uninterrupted. Upon approval of the sampling plan, a local field team of 2 Survey Coordinators, and 19 Field Enumerators collected data and conducted a survey in target districts. The field activities were supervised and monitored by a Survey Supervisor. During the data collection, one Survey Coordinator was assigned to each province to guide the enumeration teams and maintain liaison with the project team. Moreover, Value Chain Experts from the Consultant’s team conducted the IDIs and FGDs, while KIIs with senior management were conducted by the Evaluation Team Leader. Prior to initiating the field activity and survey, a two-day training session and one-day pre￾testing was held in each district to familiarize the investigators and supervisors with the 10 The evaluation team members signed conflict of interest forms as part of the contract. 23 survey’s questionnaire and discuss probing and recording techniques as well as ethical considerations. Data collected via the questionnaires was analysed according to key objectives of the assignment and through inductivity thematic analysis11. Data collected from the field (KII, FGD, IDI and beneficiary survey) was entered in the computer using Microsoft MS Excel and CS PRO as specialized data entry and analysis software. The quantitative analysis included percentages, comparisons, averages, etc. Data will be presented in graphical form so that trends can be clearly read and correlations drawn. Quantitative data analysis was undertaken using SPSS and MS Excel. Qualitative data gathered during the course of the assignment was transcribed and categorized according to the various themes and topics explored with clear conclusions drawn. 2.2.4. CHALLENGES The MTE team faced a few limitations while conducting field work. Firstly, in a few instances, due to dissatisfaction with support provided by the project, beneficiaries refused to respond during the survey as well as in IDIs (agro-dealers). This limitation was overcome by continuous follow-up by the MTE team with the PAD project team and district Community Organizers (COs) who helped in reaching out to other farmers in order to meet the survey targets. Also, the MTE team kept a significant margin in the sampled number of respondents and the actual sample size for each cluster in order to meet the survey target. The data collection process for the evaluation included obtaining data at institutional level through KIIs. As per the initial activity plan, face-to-face KIIs were planned with key stakeholders towards the end of the month of March. However, due to the lockdown situation across the country owing to the COVID-19 emergency, the strategy had to be improvised and virtual KIIs were held instead over phone/video calls. 11 Qualitative data gathered though FGDs, KIIs, and IDIs was analyzed using the inductive thematic analysis approach. This approach to analysis is a commonly used method to interpret qualitative data through pinpointing, examining, and recording patterns (themes) within the collected data. Accordingly, responses from the three qualitative tools were entered systematically into MS Excel for an ease of review and analysis. The data was then analyzed in accordance with the major themes of the evaluation, namely Relevance, Efficiency, Effectiveness, Sustainability, and Impact; and resultant analysis was documented in relevant sections of the report. 24 25 3. EVALUATION FINDINGS 3.1. RELEVANCE The PAD project is highly relevant to the agriculture sector in Pakistan and is also in line with the development priorities of key stakeholders, including: GOP, USDA, WI and project beneficiaries. Agriculture is central to economic growth and the overall development of Pakistan. The agriculture sector accounts for 24% GDP and employs half of the country’s labor force.12 Resultantly, more than 60% of the country either directly or indirectly relies on agriculture as a major source of livelihood.13 However, the country’s agriculture sector suffers from constrained productivity levels and limited market access. To overcome these problems, the Federal and Provincial governments have devised various policies and strategies. At the Federal level, the National Food Security Policy 2017 goals include: i) achieving 4% per annum agriculture growth; ii) use of bio-remediation technology for safe food production; iii) farm mechanization and processing technologies to reduce production costs; iv) enhance food and horticulture exports by up to 10-20% percent; and v) enhance food storage capacity.14 On the Provincial level, Punjab Agriculture Policy 2018 emphasizes increasing food production through higher yields and better crop mix, improving farmer profitability and enhancing the competitive position of the agriculture sector in line with global and domestic market demands.15 Similarly, major policy objectives of the Sindh Agriculture Policy 2018 – 2030 include raising overall growth in the sector to 4-5% and reducing rural poverty through increased agriculture productivity.16 Hence, the project’s activities and objectives with regard to enhancement of agriculture productivity, profitability and trade are in line with the Government of Pakistan’s policy measures at the Federal and Provincial levels. Moreover, the project design is consistent with the USDA’s FFPr results framework and strategic objectives, in particular Strategic Objective 1 – Increased Agriculture Productivity and Strategic Objective 2 – Expanded Trade of Agricultural Products. Accordingly, the project’s aim is to enhance incomes through introduction of modern techniques and technologies, access to markets and increased availability of credit and financial services.17 The project is also responsive to the needs of all direct beneficiary groups, including small and medium-sized farmers, agro-dealers and traders residing in the target communities of Punjab and Sindh provinces. Small farmers are important participants because they constitute the largest group of agricultural producers. Furthermore, selection of provinces of Punjab and Sindh were relevant because they are both major agricultural producers in each of the target 12 Source: http://www.agripunjab.gov.pk 13 Source: National Food Security Policy 2017 14 Source: Government of Pakistan – National Food Security Policy 2017 15 Source: Punjab Agriculture Policy 2018 16 Source: Sindh Agriculture Policy 2018 - 2030 17 Source: https://www.winrock.org/wp-content/uploads/2017/01/PAD-Handout-1.pdf 26 value chains of banana, date, tomato, and chili. 18 Additionally, these rural areas have significant development potential with respect to horticulture production, harvest, and post-harvest practices19. Furthermore, it was also observed that selected value chains were relevant to the broader objectives of the project as well as other stakeholders. In conclusion, it was determined that the two main objectives the project are Highly Relevant to the priorities of major stakeholders and needs of targeted beneficiaries. 3.1.1. PROJECT DESIGN The PAD project was designed by WI staff in the Headquarters with support from colleagues in the Pakistan office. Moreover, lessons learned from other Food for Progress (FFPr) projects implemented by WI elsewhere were also incorporated into the setting of PMP targets. An analysis of the design revealed that while planned activities outlined in the design served both FFPr Strategic Objectives 1 and 2, market linkage activity directly contribute to SO2. On the other hand, while directly addressing productivity (SO 1), activities such as increasing the capacity of farmers, facilitating them in adoption of better farm inputs through grants, developing their linkages with formal banking sector for seeking loans and mass communication activities have the potential to in-directly affect marketability (SO2) as well. Similarly, addressed at reducing post-harvest losses in the value chain, can affect the quality of product, thereby having an impact on marketability. A review of the project design and results framework showed that all activities were inter￾related and responsive to the beneficiary needs. Moreover, based on the project’s achievements so far in the given implementation context, with the exception of Activity I - Agriculture Lending and Activities VI, VII, and X (Training in GAP, harvest and post-harvest, and SPS), the life of project (LOP) targets for all other activities were realistic. In particular, the LOP targets for training-related activities (particularly Tier II) are ambitious and not likely to be achieved during the project lifetime. According to the design, the Master Trainers trained under the project are expected to train other farmers without the provision of any incentives to the Master Trainers from the project. Moreover, the LGF, a keystone element of Activity I: Facilitation of Agriculture Lending, was dropped during negotiations with the USDA due to potential difficulties in getting approvals from the state regulators in Pakistan. However, the activity was included in the cooperative agreement. Further, activity targets were set according to previous project experiences at the proposal stage. However, during implementation, the project has achieved more numbers against the LOP targets for some indicators, therefore demonstrating that the assumptions are not consistent with the outcome being seen at the field level. Similarly, a risk analysis and accompanying mitigation measures are absent from key project documents, thereby lacking guidance on course correction when things do not go as planned. 18 Source: PAD Workplan Narrative 2019 - 2020 19 Source: PAD Project Baseline Report 27 Furthermore, the PAD project was designed to operate in a high potential, extensively underserved area. This aspect along with targeting farmers and traders of different levels according to set criteria outlined in the project design document contributed to the project’s effectiveness. However, despite their active contribution to production and post-harvest, the project mostly overlooked women. In particular, having focused on landholders20, the beneficiary selection criteria generally applied to men farmers and mostly overlooked women, as the latter have negligible or non-existent property rights but play an active role in production and post-harvest, especially in tomato and chili crops. Similarly, nearly all trade or market￾related operations are carried out by men while women may work as laborers in activities such as harvesting, sorting, and packaging, etc. 3.1.2. DESIGN CHANGES In light of the implementation context, the project made changes to some activities since the design. Of these, major activities include the engagement of additional stakeholders and altering the approach to formation of producer groups under Activity VIII. Moreover, the approach to procurement of inputs under Activity III was also changed in order to develop sustainable input linkages. At the time of design, a number of public and private sector entities had agreed to collaborate with the project. However, once the project was launched the situation of some of these organizations was changed. In particular, among the identified sub-recipients, Tameer Microfinance Bank, Asim Agriculture Farms, and Sindh Abadgar Foundation could not participate. While Tameer had suspended its activities due to operational issues, the Sindh Abadgar Foundation could not participate due to GOP’s restrictive regulations on partnership between development agencies and local organizations. Similarly, Asim Agriculture Farms did not participate due to their expectations of international expert engagement by PAD project to facilitate their operations, a factor that is outside the project scope. Therefore, PAD either directly took on the role of some of these stakeholders (e.g. training of banana and date farmers) or included alternative stakeholders to support implementation. In particular, Sprouts Biotech Laboratories (SBL), a private sector entity, has been selected to supply high yielding cultivars of banana plants. In addition, awareness on agricultural lending was created by including specific sessions on formal lending in the farmers trainings and by arranging meetings of other microfinance bank representatives with farmer groups. Moreover, under Activity VIII, the project was to form registered producer groups and cooperatives. However, restrictive regulations by the GOP as part of the National Action Plan 2015 on registration of local organizations forced the project to form informal organizations instead. 20 Selection criteria for farmers included: 70% farmers with landholding up to 12.5 acres; and 30% farmers with landholding up to 30 acres (tomato and date) /50 acres (chili), etc. 28 Furthermore, while the project was designed to use the procurement and distribution approach to Input Grants (Activity III), in the interest of sustainability, this approach was changed in 201821 to developing direct linkages between suppliers and grantees. Finally, while the criteria for beneficiary selection laid out in the project design generally favors men’s participation, the project has also started supporting women workers who mostly work as laborers in tomato and chili production. As outlined in the section on Effectiveness, these changes had varying positive and adverse implications for achieving activity-level targets. 3.2. EFFICIENCY The project’s efficiency was assessed by analyzing the operational processes and methods. Key aspects analyzed include: timeliness, adaptive management, project management, monitoring, finance, and stakeholder performance. 3.2.1. TIMELINESS The cooperative agreement between the USDA and WI for the PAD project was signed in September 2016. The project duration being five years, the project is accordingly set to close in September 2021. However, the project faced some delays at the start due to approvals from the USDA and the GOP. Upon start of the project, a team was hired and the baseline survey was conducted in May 2017 and submitted to USDA in September 2017. However, USDA approval of the baseline survey did not come through until April 2018. Moreover, the application for registration submitted by WI to the GOP was also not approved until later in 2017. Since both approvals were the precondition to initiating field activities, thereby the project lost 18 critical months of programming, particularly affecting the achievement of LOP targets for farmer trainings and input grants. Furthermore, the global emergency resulting from the COVID-19 is likely to cause further delays of up to six months. Sindh being the hardest hit province went into a province-wide lockdown around mid-March, with the rest of the country following suit. During this time while office staff has continued to work from home, field activities have come to a virtual halt. In summary, the project is expected to operate at reduced capacity for two years, i.e. 40% of the planned five-year duration due to issues of late start up and the COVID-19 emergency. 3.2.2. PROJECT MANAGEMENT As stipulated in the cooperative agreement, the PAD project is being implemented by 44 full￾time, in-country personnel and backstopped by two headquarters staff. Annex 07 provides an organogram of the project staffing structure. In addition, staff hired by sub-grantees, Haji Sons and Bureau Veritas, also contribute to project implementation. 21 2018-2019 Work Plan 29 Led by an experienced Chief of Party (COP) and Deputy Chief of Party (DCOP) in USDA and WI programming, the project has an office in Lahore (Punjab) and Hyderabad (Sindh), each. The mid-term review team determined that the in-country coordination and processes are led and executed by the team in Pakistan, while being supported by the backstopping team in the headquarters. In general, while staff at WI is sufficient at planning and program management, the field mobilization staff were found to be stretched due to the high number of farmers and groups to be dealt with across a broad geographic area. Moreover, the final decision making for critical interventions such as the approach to distribution of input grants or approval of grant infrastructure grant applications are made at the WI headquarters. While this practice introduces an extra layer of scrutiny and due diligence, it also risks leading to remote and centralized decision making, thereby slowing down progress. For instance, while the backstopping team directed to change the approach to input grants from project-led procurement-delivery to establishment of direct linkages between suppliers and grantees, in reality the later approach has faced challenges due to absence or lack of locally available suppliers in the project areas. Furthermore, the staff hired by the two sub-grantees are mostly engaged in delivery and follow up of trainings. However, considering the high targets and lost time due to later start up, based on discussions with beneficiaries, the MTE team found the team from Haji Sons to be understaffed. In conclusion, while the project management team at WI is well staffed, that of one of the two sub-grantees needs to enhance the number of staff engaged in the project. Moreover, WI is delivering the project through close coordination between the in-country and HQ teams, with field teams designing technical interventions and the team at HQ providing review and concurrence, as well as guidance on overall strategy. However, the remote activity-level decision making at the headquarters, contributes to a slow pace of delivery at times. 3.2.3. MONITORING Project monitoring is carried out at multiple levels by the PAD project and sub-grantees, including Haji Sons and Bureau Veritas. The PAD project team follows a robust monitoring plan laid out in the Cooperative Agreement and uses several tools, including a Results Framework, a Performance Monitoring Plan (PMP), and a detailed Evaluation Plan, all approved by USDA. In addition, the baseline survey carried out in 2017 helped set the baseline indicators in the project log frame, including benchmarks for monitoring FFPr Standard Indicators (SI) and Custom Indicators (CI). The MTE mission found that the baseline survey had substantial utility for project monitoring and impact/contribution assessment purposes. Moreover, the baseline survey also confirmed the major needs identified by the project design of the four targeted value chains. Data from the field is primarily collected using data formats, quarterly reporting from sub￾grantees, monitoring logs of PAD team, and bi-annual follow up surveys with beneficiaries. Monitoring data is stored in DevResults, a custom built database which acts as a reliable and centralized system for recording and organizing quantitative data collected by all stakeholders. 30 The system allows comparison between indicators, disaggregates, activities, locations, and reporting periods. Moreover, to improve accuracy of data collected from the field, a mobile data collection tool was introduced in 2019. With regards to project staffing, PAD has a well-managed M&E team that includes the M&E Manager, who is based in Lahore and reports directly to the PAD COP, and two M&E officers based in Hyderabad and one in Lahore. The M&E staff remain in contact with beneficiaries through monitoring visits to track activity progress and using telephonic follow-ups. In addition, the M&E staff are responsible for monitoring the activities of project team and sub￾grantees through spot checks and participation in trainings delivered by them. Moreover, PAD bi-annually collects information via a structured questionnaire completed by beneficiaries on work related to their farms. Findings from this follow up monitoring activity are fed into the bi-annual reports submitted to the USDA. Moreover, the backstopping team in WI headquarters conducts quality assurance (QA) of all progress reports and other monitoring-related documents shared with USDA. At the sub-grantee level, the field staff of Haji Sons and Bureau Veritas carry out monitoring and reports are submitted to WI on a quarterly basis. While attendance sheets and training assessments are shared with PAD on a monthly basis, which are accordingly digitized by PAD M&E team on timely bases for up to date reporting. The MTE team found the monitoring systems at WI to be satisfactory with regards to collection of quantitative progress data. Moreover, planning decisions are also based on this data, for instance, equipment for a date storage grant was not delivered until the building to be built by the grantee was completed according to design specifications. However, it was noted that the quality of services delivered by Haji Sons require a closer follow up. A case in point is tomato farmers in Thatta22 who were provided low quality seed and received no response from the company to their complaints. Another issue with the same sub￾grantee was noticed with regards to selection of agro-dealers for training (Activity IX), where the MTE team found issues with the quality of training content and trainees selection, as detailed in the section on Effectiveness. It is pertinent to mention that PAD team already took notice in 2019 of the gap in service delivery to tomato farmers and devised corrective measures for 2020 in the form of: a) requiring Haji Sons to increase the number of their field staff from four to ten; and b) creating a direct reporting line between Haji Sons field staff and PAD value chain specialists. 3.2.4. FINANCE The project is financed through monetization of 25,000 MT of CDSO, translated into a federal award amount of USD 22,875,000 at the time of cooperative agreement, including USD 19,375,000 for project implementation. However, the total funds available for project implementation are USD 17,898,779, 7.62% lower than planned. These funds consist of USD 16,696,455 monetization proceeds, USD 1,000,000 of CCC funds, and USD 202,324 interest earned. The reason for this lower than expected sum was the delay in monetization of second 22 FGD conducted by Cynosure 31 tranche to November 2019 by which time global CDSO prices had come down. As of March 31, 2020, a total of 6,332,228 (31%) has been spent which reflects both direct costs and indirect costs. A summary of the budget allocation per head and actual expenditure as of March 31, 2020 are provided in Table 4. Table 4: Project Expenditure at Mid-Term Budget Expenditure (March 2020) Variance Percentage Expenditure Percentage Variance Project Management 6,182,365 3,209,438 2,972,927 52% 48% Project Activities 9,149,020 1,372,776 7,776,244 15% 85% ICR 5,063,286 1,804,013 3,259,274 36% 64% Total 20,394,671 6,386,227 14,008,444 31% 69% As shown in Table 4, by March 2020, the project had utilized 31% of the total budget, with the proportion of expenditure being lowest (15%) for the project activities pertaining to FFPr SO 1 and SO 2. An activity-level expenditure analysis revealed that Activity VI – Improved Production Techniques and Activity VII – Post Harvest Methods have the highest burn rates (35% and 25% respectively), followed by Activity V – Information Campaign (24%). Annex 08 provides a brief summary of the activity-wise financial expenditure. The major reasons for low financial delivery have been a) devaluation in the PKR since project approval; b) delays in project start; and c) difficulties in activity implementation, including Credit, Farmer Groups, and uptake of Infrastructure Grants. At the time of project approval, the average PKR to USD conversion rate was PKR 104 = USD 1. However, the PKR has sharply fallen in value since then, with the current conversion rate being PKR 167 = USD 1. Although this resulted in a slower burn rate, it also made additional funds available for the project’s use. Furthermore, delays in project start of nearly 18 months due to lengthy approval processes of GOP and the USDA, as well as restriction on staff movement and activity implementation due to recently introduced GOP policies under the National Action Plan (NAP) 2015 have also affected financial delivery. 3.2.5. STAKEHOLDER PERFORMANCE Among the 7 potential sub-recipients and four public sector partner agencies identified at the time of design, only two private sector partners, namely Haji Sons and Bureau Veritas have actively contributed to project implementation since signing contracts with PAD in April 2018. 3.2.5.1. PERFORMANCE OF PRIVATE SECTOR PARTNERS Haji Sons has had the most extensive role by providing support to red chili and tomato farmers in the form of: 1) delivering pre and post-harvest training to Master Trainers (Activity VI and VII); 2) Distributing input grants (Activity III); and 3) information on formal credit (Activity I); and 4) Training to agro-dealers (Activity IX). Similarly, BV has been providing training in SPS to Master Trainers, and SBL has been providing tissue cultured plants of banana. 32 Moreover, Telenor disseminates project-developed public awareness messages to farmers over mobile phone network (Activity V). In general, Haji Sons has been delivering services in accordance with the work plan agreed with WI. Thus far, the company has trained 963 red chili, and 752 tomato farmers, and 64 agro￾dealers; while also providing seedling trays and peat moss for nursery raising to 2,055 farmers. However, the evaluators observed some gaps in delivery, including limited staff numbers and issues with selection of agro-dealers. Bureau Veritas, the company specialized in training, certification, and inspection has provided training to 102 batches attended by 2,143 farmers, generally receiving good response as detailed in the section on Effectiveness. While SBL, brought onboard in early 2019, is a unique private sector startup in the country engaged in production of banana tissue cultured plants. SBL is tasked with providing 200,000 banana tissue cultured plants at a three way cost share of: WI (25%), SBL (25%), and beneficiary farmer (50%). By March, 2020, the company has provided 62,800 plants, thereby meeting 31% of its LOP target. While it is too early to see the full benefits of these plants, interviewed farmers have been highly satisfied with the services received from SBL. Telenor has integrated PAD’s public service messages in its existing mAgri program and reached over one million producers across the four crops, both in the target districts as well as in other parts of Punjab and Sindh. Furthermore, in addition to WI’s role as elaborated in the section on Project Management, project staff (Value Chain Specialists) are also responsible for training date and banana in pre and post-harvest management. Thus far, the project has trained 733 banana and 982 date farmers. Moreover, the project is responsible for overall communications with beneficiaries and monitoring of activities. In general, beneficiaries and partners have a positive view of the PAD team, finding them easy to work with and responsive. 3.2.5.2. PERFORMANCE OF PUBLIC SECTOR PARTNERS The performance of public stakeholders has remained less than optimal. The Date Palm Research Institute (DPRI) at Shah Abdul Latif University (Sindh) has failed to keep its tissue culture facility operations for production of disease free date palm plants. Similarly, while the PAD project approved a grant back in March 2019 for the Pakistan Agricultural Research Council (PARC) for strengthening tissue culture lab at Thatta for farmers to access to disease free banana plants, despite reported follow-ups with concerned officials by PAD staff, the grant has not been countersigned by the PARC due to red-tape. Overall, the MTE team found PAD’s strategy of working with implementing partners/sub grantees to be highly efficient, thereby enabling the project to have a broad outreach using existing organizational structures. While the opportunity for provision of tissue cultured date 33 plants has been lost due to the issues with DPRI, the project has rescued the aspect of providing tissue cultured banana plants by engaging Sprout Biotech Laboratories. 3.3. EFFECTIVENESS The project’s effectiveness has been measured in terms of achieving its set LOP targets against all activities. In addition, Annex 09 provides a summary of activity-wise achievements against LOP targets. 3.3.1. ACTIVITY I – CREDIT This activity focuses on creating farmer awareness on institutional credit in relation to: i) informal borrowing, ii) promoting lender-borrower linkages and iii) increasing farmers’ and producers’ financial literacy by the lending institutions. However, against a LOP target of 6,800 individuals obtaining agricultural credit worth USD 3.4 million, the project has only been able to link 126 farmers to formal credit amounting to USD 39,842, thereby achieving 2% and 1.2% of its targets, respectively. The major reasons for this low performance are the absence of the LGF that was envisioned in the initial project design, the high cost of formal borrowing, existing farmer-contractor relationships which are difficult to break away from, and cultural/religious beliefs of farmers that reflect negatively on interest/usury. Initially, a LGF of USD 1.7 million was included in the design with the intention to encouraging banks through lessening risk when lending to project beneficiaries. Hence, the targets for this activity were set with the LGF in mind. However, the LGF was dropped during negotiations with the USDA, but the activity and associated lending targets somehow remained a part of the cooperative agreement. Therefore, the MTE team determined that in the absence of a LGF, Microfinance Banks (MFBs) have no incentive to lend to PAD beneficiary farmers. Hence, the targets set in the project for this activity are unachievable during the project lifetime. Moreover, an assessment carried out by the project in 2019 determined that there are critical demand and supply side challenges to agriculture lending, mostly arising from risk aversion but also from other factors such as lack of collateral, etc. However, the project continues to develop linkages with additional banks while also raising awareness among farmers regarding the benefits of formal borrowing. This is done through conducting sessions on formal lending in the pre and post-harvest trainings under Activities VI and VII, and additional awareness raising sessions by the project’s Microfinance Officer and invited bank representatives. A survey of randomly selected beneficiaries of credit showed that this was the first time for 92% of them to borrow from a formal source. Khushali MFB accounts for the highest proportion of loans (66%) followed by U MFB (14%), and Khushali Bank (12%). The loans were taken by tomato (60%) and red chili (30%) farmers, while 10% also obtained the loan for other crops, primarily (91%) for the purpose of land preparation. Most loans (82%) were issued as group lending with a group comprising of 4 to 8 individuals. The highest number (71%) of 34 borrowers reported having obtained a loan of PKR 30,000 – 40,000, followed by 8% borrowing PKR 50,000 – 60,000. Prior to borrowing from this source, only 39% farmers had borrowed before, mostly from informal lenders, including commission agents, agro-input dealers, and other market intermediaries, while 6% had also borrowed from Thardeep Rural Development Program (TRDP), a local NGO. 3.3.2. ACTIVITY II – INFRASTRUCTURE GRANTS Under Activity II, the project aimed to provide in-kind grants for equipment and infrastructure worth USD 4.2 million to support improved post-harvest management in the four targeted products. Thus far, the project has delivered grants to 33 grantees worth USD 379,421, thereby meeting 9% of its goal. However, grants worth USD 1,788,206 for various projects are at various stages of completion in addition to a further around USD 2,680,000 in the pipeline and are expected to be delivered during 2020. Approved grants in process include drip irrigation, GrainPro products, cold stores, dates pitting, and banana ripening unit and pack-houses. These technologies were identified based on assessment at various stages of the project, and grantees were selected using competitive methods, including EOIs and RFAs published in May 2019. Grants are delivered and installed by vendors who are also responsible for training grantees in the Operations and Maintenance (O&M), while activities are tracked by the project’s grants and monitoring team. Since most of the delivered infrastructure have been set up only recently, their effectiveness will become clear after one or two crop cycles. However, the MTE team determined that of the grants delivered thus far, date storage and drip irrigation have the highest potential for effectiveness. In particular, rehabilitation by PAD of ten solar dryers designed to store approximately 10 MT each, established under an earlier USAID project, have shown encouraging results including: uniform drying, 40% decrease in drying time as compared to conventional methods, and a 25% increase in market price through increase in holding power and improved product quality. The project till March 2020 has provided five cold stores23 of 300 MT capacity each, while further five cold stores are approved24 and four are in pipeline25, thereby adding total storage capacity of 5,700 MT at a total cost of USD 4,166,583, including PAD contribution of USD 2,039,519 and grantee share of USD 2,127,063 (51%). Meeting with two date cold store grantees with operational facility revealed that this is a unique contribution of the project, since in the absence of project support the grantees would have continued using sub-standard rented space. In addition to the savings on rent, the date’s storage facilities are expected to result in better quality product, thereby potentially generating 50% higher prices. In fact, it was reported that other traders in the area are also interested in establishing similar date storage facilities. 23 Of these, two cold stores are completed and handed over to the grantees, while three are completed and awaiting hand over to grantee. The stores are valued at USD 869,292 24 Approved stores at a value of USD 2,338,733 25 Stores in pipeline at a value of USD 1,454,405 35 This high demand has also been confirmed by the PAD project team who has gotten additional requests for support. Furthermore, drip irrigation is expected to be another effective activity. Although the systems have not started functioning yet, interviews with farmers revealed that in comparison to their current practice of flood irrigation, they expect drip technology to result in time saving (30 minutes down from 3 to 4 hours for a 5 acre plot), 60% reduction in water usage, 50% reduction in labor, lower incidence of disease, and facilitate timely harvesting. Due to the untapped potential of the date sector, demand for date storage is high. Similarly, drip irrigation was found to be high in the project areas due to their water stressed nature. In particular, the MTE team found that in Punjab, the project has targeted an underdeveloped area with regard to basic production and post-harvest technology, as beneficiary farmers reported lack of any prior knowledge of drip irrigation technique. On the other hand, despite 50% reduction in drying time, farmers were generally seen to be less receptive in general to Solar Bubble Dryers (SBDs) due to the high cost and reluctance to invest in untested technology. Major challenges faced with grantee selection include low-quality grant applications, especially with regard to developing business plans. Since potential grantees have to develop these plans for presentation to WI, despite having sound technical capacity to absorb the grant in many cases, they lack the skill to document good business plans in spite of guidance from the project. The MTE team therefore believes that this aspect would have likely discouraged a number of well qualified grantees from going through with the application. While WI team has suggested the recruitment of expert grant writers to such individuals at their own expense, they are often unwilling to use this option in the absence of guaranteed success. Moreover, project management challenges with regard to disbursement of selected grants have been reluctance of potential beneficiaries to contribute cost share and/or inadequate or slow pace of progress when completing their share. These aspects have at times slowed down or delayed delivery. On the other hand, some grantees of date stores and tunnels complained of slow response and poor planning on behalf of the vendors. 3.3.3. ACTIVITY III – INPUT GRANTS Under this activity, the project aimed to provide basic inputs and equipment to farmers worth USD 761,951. Thus far, two grantees have been provided support who have in turn delivered inputs to 2,118 farmers (2,055 tomato and chili farmers provided seedling trays and peat moss; and 63 farmers provided banana tissue cultured plants), while utilizing USD 84,790 (11%) of the allocated funds. While the initial delivery model at design stage was that of procurement￾distribution, in the interest of sustainability, this approach was changed in mid-2018 to supplier-farmer linkages. Resultantly, instead of relying on the project, farmers were to directly procure inputs of their choice from local vendors from a prioritized list of inputs, including: a) tomato and chili seedling trays and planting medium, b) chili and tomato harvesting tools, c) tomato packaging materials, d) improved variety banana plants, and e) banana packaging materials. However, when the project set out to implement the alternative approach, it was found that the supply chain for such inputs was generally characterized by centralized suppliers 36 concentrated in large cities with no distribution networks available in production centres. This limited availability restricted the scope of the Input Grants activity to only planting material, including tissue cultured plants of banana and tomato and chili seedling trays using a supplier￾based subsidy program. Banana Plants: For provision of disease free tissue cultured banana plants, Sprouts Biotech Labs (SBL), a private sector company based in Karachi was selected in March 2019. The company was tasked to provide 200,000 tissue cultured banana plants of Grande Naine (G 9) variety at PKR 80 per plant on cost share basis, with the cost shared between WI (25%), SBL (25%), and Grantee (50%). Each beneficiary is entitled to 1,000 plants to be planted over one acre. Thus far, the company has delivered 62,800 plants to 63 farmers, thereby meeting 31% of its LOP target. Currently, banana is dominated by Cavendish variety which has smaller sized fruit (finger) and low yield. Therefore, new but not widely known varieties such as G-9 produce large sized fruit (finger) and more yield. Interviews with beneficiaries revealed that this activity has been highly effective, having resulted in almost 40% less time to mature (10 months vs. 16 months), double yields of 40,000 kg (40 MT) per acre as compared to 20,000 kg (20 MT), lower post￾harvest losses, and almost 60% price increase (PKR 1.4 million vs. PKR 0.6 million per acre). These results have convinced existing beneficiaries to purchase additional plants and others to invest in the new variety. In fact, an FGD with banana farmers in Matiari revealed that prior to project support they were unaware of tissue cultured plants but are now motivated to entirely replace the current plantations with these plants. However, a valid concern shared by both the project team and beneficiaries is the lack of availability of tissue cultured plants in the country due to limited production. In addition, farmers were also satisfied with the services received from SBL, including initial demonstration and timely replacement of dead plants. Similarly, they reported receiving periodic visits from PAD staff to guide farmers on plants management in the field. Chili and Tomato Seed Trays and Peat Moss: For this input, Haji Sons, one of the two sub￾awardees, was tasked with providing chili and tomato farmers with plug trays and peat moss to demonstrate the benefit of this technology compared to raising seedlings in soil beds. The package included 30 plug trays and half a bag of peat moss free of cost to be provided with purchase of improved seed for plantation on half acre from Haji Sons or their associated dealers. The cost of seed translates into grantee share of 46%. In addition, through their extension staff, Haji Sons have to provide guidance to grantees on nursery raising in trays. Interviews with tomato farmers in Sindh revealed that although they were already aware of such nursery raising technique, PAD facilitated these farmers to adopt this technology through free availability of trays and peat moss. While chili farmers said that they were new to raising seedlings in plug trays. However, in both cases, while the general response to trays and peat moss was positive, most farmers were critical of the seed quality (e.g. lack of germination, low germination rates, soft fruit, etc.) and lack of follow up support provided by Haji Sons. 37 However, the use of plug trays and peat moss for nursery raising was also introduced through trainings (Activity VI) and has been considered highly effective, as detailed in the section on Training. For instance, farmers interviewed in Thatta reported that they were able to save 2 to 3 packets of seed equivalent to PKR 16,000 to 24,000 per acre using the nursery raising techniques demonstrated by the project. Consequently, both tomato and chili farmers shared that while they will continue using seed trays and peat moss for nursery raising, both items being easily available in the market, they will not purchase seed from Haji Sons in the future due to the issues encountered with these inputs provided by the company under Activity III. Since this activity was delivered in partnership with two private sector sub-grantees/partners, an assessment of relative effectiveness showed that PAD provided both suppliers a platform for expanding their market base in exchange for sharing their expertise. Further, SBL had to offer a niche product at subsidized rates which also proved to be highly beneficial to farmers and created significant demand. On the other hand, in order for farmers to access plug trays and peat moss, Haji Sons provided a rather generic input (seeds) at full price but with ineffective results. This issue got further aggravated because of the lack of effective monitoring and follow-up by this sub-grantee. 3.3.4. ACTIVITY IV – MARKET ACCESS Under this component, the project aimed to assist 20 farmers in order to facilitate buyer-seller relationships and private partnerships. Thus far, a total of nine agreements with six farmers (45% of the LOP target), have been signed in Sindh. These farmers have been linked to higher paying markets (mostly retailers), including one banana (one agreement) and five date farmers (eight agreements), thereby resulting in 45% achievement against LOP targets. While efforts to connect more farmers across all four products are also in process. In order to form buyer-seller linkages, the project has been working with several commercial buyers, such as processors and retailers, including prominent industry names such as Unilever Pakistan, Shan Foods, Shezan International, Mitchell’s Fruit Farms, Premium Choice, Al Twabul Foods, and Metro Cash and Carry. These efforts have been in the form of meetings to assess demand, delivery of samples, exploring the possibility of contract farming with small growers, negotiating contracts, and in some cases finalizing buyer-seller agreements. Interviews with selected farmers revealed that the linkages established thus far have been effective and could not have been established without the project’s support. In particular, date farmers linked with markets reported being able to grow sales volumes, improve packaging practices, and increase revenue of up to 20% per 40 kg (20% per 0.04 MT). Similarly, a banana farmer who was earlier exploited by middlemen through low prices and delayed payments is now linked to major markets in Punjab for advance payments and PKR 2.5/kg (PKR 2,500/MT) higher rate. Overall, farmers linked by the project have been able to sell 725 MTs of produce to higher paying markets, including 567 MTs of bananas by one beneficiary and 158 MTs of dates by five beneficiaries. Moreover, in addition to the nine farmers linked to markets, which are of medium to large size, several smaller farmers in the area are also expected to benefit by acting 38 as suppliers if market demand exceeds the ability of these directly assisted farmers. One such date farmer, having demand for 200 MT, has already started exploring such relationships. Similarly, a banana farmer linked to markets for sale of 300-400 MT is purchasing produce from another 40 farmers in the area. However, these linkages have been operational for only one year while mostly relying on one or two buyers. Therefore, the MTE team believes that it is premature to determine their long term effectiveness at this point. Instead, it will be important to assess whether the relationships can continue over the coming harvest seasons. This was evident in the case of one date farmer who despite getting good indications early on has had a breakdown in communication with the two clients after just one season. Moreover, in order to exploit the full market potential, the grantees will need to include expanding their customer base through participation in activities such as trade fairs. Further, sustainable market linkages will require industry-level investment in post-harvest facilities such as production of improved varieties, construction of pack houses, introducing packaging materials, and value addition and processing. 3.3.5. ACTIVITY V – PUBLIC INFORMATION CAMPAIGN Under this activity, the project aimed to disseminate market information focused on improving farm productivity and increasing awareness on food handling and safety to 395,000 people through electronic media. Thus far, activities have included a public awareness campaign through Telenor-Pakistan, a cellular company; and text messages focusing on GAP disseminated to PAD trained farmers through an SMS portal, and a WhatsApp group of farmers who are interested in exchanging information and experiences. Since its launch in September 2018, the campaign has gained 900,23326 subscribers not only in the project districts but also across Punjab, resulting in achievement of 228% of the project target. As elaborated in the section on Efficiency, this high achievement has been due to bundling the messages with Telenor’s existing mAgri program. Based on customer usage tracking, Telenor believes that this has been a useful service. In fact, the content provided by the project is considered to be of better quality than the existing messages under the company’s ongoing initiative of mAgri, thereby leading to adoption of similar approach when developing messages for its other initiatives on agriculture-related information. Moreover, the MTE team was able to assess the effectiveness of the messages and information under this activity through a survey of the project direct beneficiaries, and determined that 67% of Tier I are aware of the services and 49% of Tier 2 beneficiaries are aware of the services. Of these, 71% from Tier I and 47% from Tier II have utilized the services, with all users reporting some degree of satisfaction. 26 The count for subscribers is regularly provided by Telenor to PAD on a monthly basis. An Activity Insight report is shared, which includes the data on numbers of users disaggregated by crop and month. It also includes daily usage reports and some info graphics to give a better sense of the demographics of the users. 39 However, of the 29% in Tier I and 53% in Tier II who are not utilizing the services, the majority, 54% and 33% respectively, do not either have access to a mobile phone or are subscribers of phone companies other than Telenor. Moreover, 8% and 26% of those not using the services from Tier I and Tier II, respectively also said that they are not interested. However, only the minority, 14% and 4% said that they do not use the services because they do not find it helpful. In conclusion, while the information and advisory service have been found helpful by most users, in the case of Interactive Voice Response (IVR), their use is restricted only to those who have a mobile phone and also subscribe to Telenor. Therefore, in order to improve the outreach of its public service messages the project should work on exploring alternate channels of dissemination. 3.3.6. ACTIVITY VI AND VII – TRAINING IN IMPROVED AGRICULTURE PRODUCTION TECHNIQUES AND POST-HARVEST HANDLING AND PROCESSING The project aimed to provide training in GAP (Activity VI) and post-harvest handling and processing with the aim to improve yields and quality. These include Tier I trainings to train Master Trainers in the community conducted by Haji Sons (chili and tomato) and PAD Value Chain Specialists (banana and date), Tier II trainings to be conducted by the trained Master Trainers of fellow farmers in the area, and female work force trainings. Thus far, against an LOP target of 35,820, a total of 20,060 individuals (56%) have been trained in production techniques and post-harvest methods. Apart from these, another 139 female workers have been provided awareness on GAP at the time of nursery raising and transplanting of plants; and another 494 on harvesting and field handling techniques. 40 3.3.6.1. TIER I Overall, Tier I farmers interviewed across the four value chains having received trainings in production, harvest, post-harvest, and marketing practices found the trainings to be effective. As a result of applying the trainings to their crops, 65% respondents have witnessed an increase in income. In particular, 85% reported having applied production techniques learned during the various trainings. These include the adoption of improved planting material (65%), land preparation (65%), nutrient management (37%), pest management (22%), and irrigation techniques (23%). Encouragingly, all but 3% of trainees reported having production benefited from participation in the training, including higher yields (60%), improved quality produce (60%), reduced disease or pest attacks (30%), lower cost of production (17%), and reduced wastage (15%). While 14% of those reported that although their first crop after the training is still in the field, they are foreseeing similar benefits towards the end. Similarly, harvest and post-harvest training was applied by 70% of trainees, and included better picking and harvesting techniques (50%), post-harvest handling of produce (50%), better packaging material (6%), and improved storage (7%). These trainees reported benefits including: higher price (42%), larger quantity marketed (21%), and access to new markets (11%). Product-wise effectiveness of the trainings through FGDs revealed differences in benefits accrued, mostly due to the nature of product. For instance, while banana farmers have reported enjoying one kg extra weight per bunch and increased price of PKR 1.25-3.75 per kg (PKR 1,250-3,750 per MT), date farmers said that despite appreciating the harvest and post-harvest methods taught in the training, they are not able to apply the techniques due to difficulty in organizing human, financial, and logistical resources during the very short harvest season. Moreover, availability and cost of inputs such as harvesting ladders and baskets as well as trained labor were also cited a hurdle by date farmers. Whereas, due to similarities in some production processes, chili and tomato farmers found the training to be particularly beneficial with regards to nursery raising and reported average cost savings of PKR 35,000 and PKR 22,000 per acre, respectively. Moreover, chili farmers also said that their yields of dry chili have doubled from 2 to 4 MT per acre in some cases; while some tomato farmers also reported yield increase from 2.4 MT to 4 MT per acre. While the MTE team concludes that the trainings have been effective, they also found a certain level of disappointment among the beneficiaries for failing to receive any input grants. As detailed under the effectiveness on Activity III, the project has fallen behind in the delivery of inputs due to a change in delivery approach. This issue can play a role in long-term effectiveness, since the surveyed Master Trainers said that the inputs recommended during the trainings are not available (24%) or are expensive (19%). Moreover, a number of farmers reported not being given any training material for later referencing and believed that the availability of such material could help them. 41 3.3.6.2. TIER II Moreover, the MTE team also assessed the effectiveness of Tier II trainings and determined that, 53% reported an increase in income and 26% were able to establish new market linkages because of applying techniques learned during the training. In particular, these benefits have been derived due to the large majority (86%) using the disseminated improved production techniques, including improved planting material (29%), land preparation (29%), nutrient management (12%), irrigation (9%), and pest management (7%). Furthermore, 70% reported adopting improved harvest and post-harvest handling techniques, including better picking methods (36%), post-harvest handling (30%), and others such as packaging material and storage (3%). In view of these immediate results, the MTE team found the Tier II training to be effective as well. However, based on their experience, 43% of the interviewed Tier II men and women trainees27 have found it difficult to apply the training due to various reasons, including: lack of availability or high cost of recommended inputs, and difficulty to follow the training due to limited education. These factors are likely to affect the long-term effectiveness of this activity. 3.3.7. ACTIVITY VIII – CAPACITY BUILDING OF PRODUCER GROUPS This activity was designed to organize farmers into groups and strengthen existing associations to benefit from collective procurement, joint marketing, improved competitiveness, adoption of quality standards and practices, and other associated opportunities. However, farmer groups could not be formed as originally designed due to restrictions imposed on CBOs in Pakistan via government policies that prevented any efforts of similar organizations being used to fund terrorist activities under the guise of development work. Resultantly, PAD could not formally register the CBOs and, by September 2018, moved towards organizing informal groups. By March 30, 2020, the project has organized 133 groups against the LOP target of 170 groups (78% of target). In their current informal status, these groups have facilitated outreach to individual farmers under various activities, including training, marketing, and credit, etc. However, the groups have not been able to attain their original purpose of collective action such as procurement and marketing. Further, while group members found most of the support by PAD project to be effective, as detailed elsewhere in this report, the expressed disappointment at not receiving the various input grants that were planned under Activity III. The MTE team found that while the producer groups have been effective in helping the project reach out to farmers, their long-term effectiveness cannot be expected due to the lack of any common ongoing business interests among members, e.g. lack of collective procurement or marketing opportunities, etc. In fact, during FGDs it was observed that group membership has been steadily declining. 27 Interestingly, the responses from men and women farmers were the same with regards to issues related to training and application. 42 3.3.8. ACTIVITY IX – AGRO-DEALERS TRAINING Under this activity PAD plans to build the capacity of 50 agro-dealers/input suppliers and 5 equipment suppliers in the areas of business and financial management, financial inventory control, and use of latest practices and technologies such as improved irrigation techniques, fertilizer usage, and pest management regimes. Thus far, the project has trained 64 agro-dealers (116% of LOP target) through three one-day seminars, of which two were conducted in Sindh and one in Punjab. The training primarily focused on traders providing inputs for tomato and chili, and topics included seed selection, storage, and germination; safe disposal of pesticides; inventory management, and customer relationship. Moreover, the training was also used as a platform to promote seeds for tomato, chili, and other vegetables, and other inputs sold by Haji Sons (plug trays, peat moss, and crop nets). Whereas, trainees were major dealers of agriculture inputs in the target districts, selected by Haji Sons. During interviews with trainees, the MTE team learned that seed storage, selection, and germination were found to be the most valued component of the training by most participants. However, when it comes to adoption of other improved technologies and methods by these dealers, with some exceptions in Sindh, the training was largely ineffective. The major reason for this was trainee selection, training methodology, and the pre-existing relationship of Haji Sons with dealers in both Sindh and Punjab. With regards to selection, trainees were major dealers of agriculture inputs in the target districts, selected by Haji Sons. However, during evaluation interviews it was determined that some of the selected trainees were not readily willing to attend the training as they did not see any utility of it for their respective businesses. But they nevertheless showed up as a matter of courtesy to colleagues/friends who persuaded them on behalf of Haji Sons. Furthermore, although the activity is referred to as ‘training’, attendees reported that this was more of a marketing event by Haji Sons (for inputs and seeds supplied by the company), but with some additional focus on knowledge sharing on seed germination and inventory management, etc. In addition, no learning materials were provided to trainees for future referencing. Moreover, the PAD project’s role during the training was not highlighted and participants assumed the activity to be the sole initiative of Haji Sons. Finally, despite admitting that Haji Sons sells some good varieties of tomato, chili, and other vegetable seeds, interviewed agro-dealers who attended the training reported that they generally avoid conducting business with the company due to its difficult payment terms and problematic sales practices. 3.3.9. ACTIVITY X – TRAINING IN SANITARY AND PHYTOSANITARY Under this activity, the project intended to train 4,110 farmers in SPS standards and processes with the aim to improve hygiene, food safety, and quality of product along the value chains of all four targeted products. Moreover, in the final year, the project design stipulates assisting 43 promising farmers with obtaining HACCP certification. This activity has been carried out with BV, a lead international firm in the area of Testing, Inspection, and Certification (TIC). Thus far, the activity has primarily focused on farmers, and 2,143 individuals (52% of LOP target) have been trained. Although, 58% of interviewed trainees said that they had no problem applying the training; 30% also reported that recommended inputs and tools are not readily available, 24% found the recommended inputs and tools to be expensive, and 11% found it difficult to follow training. Despite these difficulties, most (93%) of those interviewed reported having applied the training and experienced positive outcomes. Among those interviewed, 93% have applied the training on crop production and 68% on post￾harvest and handling processes, including better harvesting techniques (52%), improved handling of produce after harvest (47%), adoption of better packaging material (7%), and improved storage methods (4%). As a result of applying these practices, 65% have been able to market higher volumes, 42% experienced lower product wastage, and 16% reported getting access to new markets. Only 5% said they did not benefit from application of trainings. Consequently, 73% of those who applied the training saw an increase in income. Moreover, exchange of experience being a common phenomenon across farming communities, 90% trainees reported that they have shared the newly gained knowledge with other farmers. However, despite its high effectiveness, the MTE team found that the project would need to improve its follow up for this training as well, since 74% interviewed SPS trainees reported having tried to approach project/training staff after the training, but only 34% have received any follow up support. 3.4. IMPACT28 The MTE team found that despite its recent start (mid 2018), the PAD project has had significant contribution towards its stated objectives. As detailed in the preceding section on Effectiveness, a number of activities have shown immediate positive results in the form of improved production, harvest, post-harvest and marketing practices, thereby improving product quality and improving incomes of farmers along the four targeted value chains. Activities with significant positive results include training (in GAP, harvest, post-harvest handling, and SPS), infrastructure grants, input grants, and marketing. Conversely, credit, agro￾dealers training, and producer groups have not shown any significant contribution due to various reasons mentioned above. Having said that, the project’s contribution is spread thinly due to the broad range of activities undertaken by the project, thereby, weakening its capacity for transformational change in any given crop or cluster. Furthermore, there have been disproportionate gains across activities, mostly depending on the type and nature of activity. For instance, while tissue cultured banana plants under input grants have contributed to the improved production and incomes of hundreds 28 Since this is not an impact evaluation with a counterfactual, the project’s contributions towards its objectives were analyzed to inform this section 44 of farmers, date storage has supported only lead farmers, especially in instances where backward linkages with growers were not established. In addition, it is important to note that the worldwide29 lockdown from COVID-19 is likely to limit the project’s contribution, at least during the period March to July, 2020. 3.5. SUSTAINABILITY Sustainability of the PAD project has been assessed with reference to beneficial continuation of activities by existing beneficiaries as well as through upscaling and replication. In general, it was observed that while the project’s effectiveness and impact/contribution have been high for selected activities30, this contribution maybe short lived unless market readiness and linkages (backward and forward) are prioritized going forward. In particular, capacity building activities focused on farmer training and the training of Master Trainers to further spread the newly learned methods and techniques has been helpful in disseminating the message. However, without demonstrating these technologies through initial provision of inputs, or developing links to input providers as well as access to affordable credit, farmers were found reluctant or unable to apply the methods. Moreover, while the infrastructure grants have been received well thus far, it is too early to foretell their sustainability. Various factors that can play a potential role in this regard include market linkages, capacity for operations and maintenance, and replication by others in order to gain the advantages of cluster-based production and marketing. Similarly, since the marketing linkages established thus far are in their infancy, their sustainability will be determined by consistent supply by grantee producers/traders, further investment in post-harvest handling (e.g. packaging, processing, branding, etc.) and also broadening the customer base. Notable, however, the public awareness campaign (Activity V) in collaboration with Telenor has the potential to reach a large number of farmers due to the company’s broad outreach and incorporation of PAD’s activity into an ongoing program (mAgri). In fact, the company is now looking for methods to collaborate with other ‘telcos’ in order to expand the geographic and numeric scope of the farmers using this service. Finally, the sustainability of credit, agro-dealer training, and producer groups is at high risk due to issues of design and delivery; and nursery raising of tomato and chili, one of the two activities under Activity III- Input Grants, is also unsustainable, as detailed in the relevant sections above. 29 In Pakistan, the COVID-19 lockdown started from Sindh in mid-March 2020 and gradually turned into a nationwide lockdown. The situation is likely to persist at least until early May. 30 Training, Input Grants, Infrastructure Grants, and Marketing 45 46 4. CONCLUSION & RECOMMENDATIONS 4.1. CONCLUSION In conclusion, the design of PAD project has been relevant. However, there have been some changes in design governed by endogenous and exogenous factors. These changes have generally had adverse implications for the project in terms of meeting its targets, and include cancellation of the LGF, change in approach to delivery of input grants, and modality of forming producer groups. With regard to efficiency, the project has sound project management and financial procedures in place. However, while the project has a monitoring system that helps track financial progress, monitoring of services delivered by sub-awardees needs improvement. Furthermore, the project has lost two of the five years (40%) of the planned duration due to delays associated with startup, approvals, and the COVID-19 emergency. The evaluation team found that despite its recent start (mid-2018) and significant devaluation in the PKR, the PAD project has been able to utilize 29% of available funds and has also been effective, resulting in positive contribution to improved production and higher farm incomes. In particular, activities with significant positive results include training (in GAP, harvest, post￾harvest handling, and SPS), infrastructure grants, input grants, and marketing. Furthermore, the activities with regards to infrastructure grants and market linkages are only starting to pick up. Conversely, credit, agro-dealers training, and producer groups have not resulted in any contribution due to issues of design and implementation. 4.2. RECOMMENDATIONS Based on the in-depth review of the PAD project, the MTE team provides the following recommendations to improve future project implementation and/or towards relevant project designs and synergies with other USDA or USG projects. 4.2.1. APPROACH TO IMPLEMENTATION The MTE team found that in a period of just 18 months, the project has been able to undertake a large number of low-cost yet effective activities, with a major contribution to capacity building towards improving production quantity and quality. However, going forward, these activities now need to converge to translate into sustainable gains through focus on market linkages and should include measures such as support to post-harvest handling infrastructure, certifications, and market linkages. While keeping this objective and the outstanding 71% budget in mind, the MTE team recommends the following measures to be taken to maximize project impact and sustainability. 4.2.1.1. MODIFICATION OF ACTIVITIES In view of the performance thus far, the following modification is recommended across the project activities. 4.2.1.1.1. ACTIVITY I – CREDIT 47 The absence of a LGF as earlier envisaged in the design has critically affected the performance of this activity under the current scenario of risk-averse financial institutions and borrowers. Furthermore, given the time resources available to the project during its remaining life, it will be counterproductive to modify this activity at this stage, especially given the attention required to maintain the momentum that the project has built in other areas. It is therefore recommended that this activity is cancelled and funds are reallocated to other high impact/contribution activities. 4.2.1.1.2. ACTIVITY II – INFRASTRUCTURE GRANTS i) Moreover, it is recommended that the project continues to provide infrastructure grants with the intention to develop product-based clusters in the areas of date and banana by supporting additional storage units and pack houses. Whereas, with respect to chili and tomatoes, drip irrigation should be the continued focus, while the project should also explore the possibility of supporting commercial nursery development of planting material in the production areas with easy access to producers. ii) A major operational issue proving to be a hurdle in receiving high number of grant applications is the project’s policy of not providing direct assistance to grantees in the form of business development plans. To encourage applicants, it is recommended that the project pay for the development of all grant applications (or at least advice on developing business plans) by a dedicated third party consultant, with the condition to deduct the value of services from the grant funds of only the successful grantees. 4.2.1.1.3. ACTIVITY III – INPUT GRANTS i) Considering the success with banana tissue culture plants, it is recommended that the project should continue promoting these plants and linking producers to reliable suppliers. ii) Conversely, based on beneficiary feedback, the modality of inputs for chili and tomato nursery raising must be changed so that farmers who wish to benefit from the grant can do so without the compulsion to buy seed from a certain vendor. Moreover, as stated earlier, the project should support the development of private sector commercial nurseries. iii) To ensure higher impact and sustainability of capacity building activities, the project is advised to broaden the list and categories of tools and implements to farmers. In case, vendors of such products do not have the capacity to deliver locally, the project should utilize alternative strategies such as the procure-distribute model until such linkages can be established. iv) Moreover, it is important for the project to collaborate and link with existing projects such as the World Bank funded Sindh Agriculture Growth Project (SAGP) for provision of inputs and tools for dates, or the government agriculture extension department for provision of spraying machines to banana farmers. The project should closely coordinate with SAGP and government agriculture extension department at the levels of PAD Provincial Program Coordinator Sindh and Director General Agriculture (Extension) 48 who is also Project Director, SAGP. They should regularly exchange projects related information and discuss activities. This will help in developing synergies, will avoid duplication and provide opportunities to benefit from lessons learned. 4.2.1.1.4. ACTIVITY IV – MARKET ACCESS i) Since it is recommended that the project should concentrate its overall efforts on improving market linkages, all activities being carried out under this component should continue. ii) Moreover, based on experience of the gem and jewelry company established with the support of the USAID-funded Pakistan PISDAC project, the project is recommended to explore the option of establishing a similar company for promotion of the date sector with the purpose to promote investment in the highly underserved but lucrative date sector. 4.2.1.1.5. ACTIVITY V – PUBLIC AWARENESS CAMPAIGN While the service being provided through Telenor is in high demand, it is not accessible to all potential users due to the limitation of being available only to Telenor subscribers. It is therefore recommended that PAD either supports Telenor in collaboration with other mobile phone companies for delivery of messages or directly use networks of other companies’ message delivery. Moreover, other venues such as radio programming can also be explored. 4.2.1.1.6. ACTIVITY VI, VII, IX, X – TRAINING IN GAP, HARVEST AND POST-HARVEST HANDLING, AND SPS; AND CAPACITY BUILDING OF AGRO-DEALERS i) While this activity has been effective, the project has only met 58% of its LOP target. Therefore, in order to reach a larger audience in a short period of time, as well as to make efficient use of funds, it is recommended that the project switches its approach to the proven methods of Farmer Field School (FFS) and demonstration plots. While implementing activities in this manner, the participation of women farm workers should also be ensured. ii) Furthermore, there is a need to revamp the current modality of agro-dealers training in case being continued, including selection criteria and training content, etc. Small and medium scale local agro-dealers should also be contacted through a general announcement/advertisement process and those meeting an established set of eligibility criteria should be selected to participate in the training. In addition, the training should be provided by a qualified company/department through close collaboration and monitoring of PAD project. Based on the trainee feedback thus far, it is recommended that this training must focus on practical methods of seed selection, storage, and germination; plant diseases, and fertilizer regimes, etc. iii) Also, as stipulated in the project design, interested producers with potential should be encouraged and supported to get quality certifications with the help of Bureau Veritas. 49 iv) Finally, it must be ensured that all trainees are provided reference materials which can be used even after the training. 4.2.1.2. STAKEHOLDER COLLABORATION i) Implementation: Despite delivering services often to the same farmers, there is no interaction between sub-grantees or partner organizations when performing their role. For instance, Haji Sons and Bureau Veritas often provide training to the same group of farmers, but on different subjects and different occasions. Similarly, while SBL provides the banana plants, PAD is responsible for training farmers in production techniques. Therefore, joint activities, e.g. planning sessions between Haji Sons and Bureau Veritas and/or combined training sessions delivered by PAD and SBL can potentially improve the quality of this activity due to the variety of knowledge available from both organizations. ii) Monitoring: While the project’s monitoring systems are in place with reference to reporting progress, there is room for improving the quality of follow up. This is particularly important in terms of implementation by sub-grantees. It is therefore recommended that the project devises monitoring mechanisms to directly monitor the quality of services provided, including due diligence on the quality of inputs. iii) Partnership: Sub-grantees, i.e. Haji Sons and Bureau Veritas, were selected at the time of project design based solely on their willingness to participate in the project bid and potential capacity/expertise. However, in view of the performance gaps observed in the partnership with Haji Sons, it is recommended that in case the targets for these activities are revised upward, the project replaces the company with alternative partner(s). In the interest of transparency, these partner(s) should be selected based on competitive bids and their performance should be closely monitored by the PAD project team/third party monitors. 4.2.1.3. PROJECT MANAGEMENT i) No-Cost Extension: The project has lost two years (40%) of the planned duration to various procedural delays and the ongoing COVID-19 emergency. However, as the project activities in critical areas such as infrastructure grants and marketing have just started picking pace, it is highly recommended that the project is granted a no-cost extension of 12 – 18 months31 in order to enable the project to capitalize on this momentum. ii) Budgetary Review: Since a number of modifications have been recommended to the original project design, it will be critical to revise budgetary provision for various activities in order to align them with the newly emerged priorities post MTE. 31 Determined by the time it would take the country to return to normalcy after the COVID-19 outbreak 50 iii) Monitoring: In order for the project to achieve its targets and improve monitoring, it is recommended that the staffing structure is reviewed and additional staff or sub￾grantees/contractors recruited, if and where necessary. Moreover, considering the encouraging contribution to different aspects of the four crops, the project is advised to undertake a detailed product or activity-wise impact assessment to highlight the progress made and potential areas for improvement. 4.2.2. PROJECT DESIGN It is recommended that future project design incorporate the following elements: i) While sub-grantees are selected at the time of project design, in order to remain relevant and efficient, implementing partners should be selected based on competitive bids, especially in the case of private sector companies. Also, in order to ensure transparency, the responsibility for monitoring such activities must lie with the project team and/or third party monitors. ii) The time taken for key procedural activities required to start a project, e.g. country-level registrations or approval of baseline activity by the USDA should not be counted towards project duration, as such delays inevitably affect project results. iii) A well-articulated exit strategy should either be incorporated in the project design or required to be developed at project start, and reviewed at the time of MTE. iv) Considering the important role played by women in farming and associated value chains, future projects must design activities which are inclusive of both men and women farmers, and accordingly assign gender segregated targets. 51 ANNEXES 52 ANNEX 01: PROJECTS RESULT FRAMEWORK Figure 1: PAD FFPr Result Framework – Strategic Objective I – Increased Agricultural Productivity Figure 2: PAD FFPr Result Framework – Strategic Objective II – Improved Trade of Agricultural Products 53 ANNEX 02: EVALUATION MATRIX EVALUATION MATRIX Key Questions32 Sub-Questions33 Areas of Inquiry Data Collection Tools Target Group Relevance The alignment of project interventions to the needs of the project beneficiaries, the country’s agriculture and/or development investment strategy, and with USDA and the USG’s development goals, objectives, and strategies. Are current project activities relevant and well-conceived in the context of the approved project? - Are the activities responsive to the needs to the beneficiaries? - Are activities designed in accordance with the local policy and economic context? - Are the activities aligned with capacities of beneficiaries, partners, and value chain infrastructure, etc.? - Literature Review – Cooperative Agreement between Winrock and USDA; PAD Baseline Report - Focus Group Discussions - In-Depth Interviews - All Project Beneficiaries, including farmers, traders, agro-dealers, and processors, etc. Can the strategy be improved? - What aspects of the project design have hindered progress? - Which activities have not achieved targets? Why? - What are the project operational and management aspects that require modifications in order to improve progress against targets and increase/leverage outreach and impact? - Literature Review – Semi-Annual Performance Reports; Annual Work Plans - Focus Group Discussions - In-Depth Interviews - All Project Beneficiaries, including farmers, traders, agro-dealers, and processors, etc. - Project Staff in Pakistan and HQ - Project Implementing Partners, Sub￾awardees, and Service Providers How accurate have the critical assumptions underpinning the project proven to be in practice? - What have been the changes in the political, social, and economic context since the project design? - Were these anticipated in the project design along with accompanying mitigation measures? - Literature Review – PAD Baseline Report; Semi-Annual Performance Reports; Annual Work Plans - Key Informant Interviews - Project Staff in Pakistan and HQ - Project Implementing Partners, Sub￾awardees, and Service Providers 32 As posed in the PAD MTE TORs 33 Ibid 54 How are project benefits accruing differently to men and women? - Are there any differences in the role of men and women with respect to project focus? - Are any project activities tailored to the different needs of men and women? - What are the differences in the benefits accruing to men and women? - Literature Review – Cooperative Agreement between USDA and Winrock; Publications on the Role of Women in Farming and Agricultural Value Chains in target areas - Beneficiary Survey - Focus Group Discussions - In-depth Interviews - All Project Beneficiaries, including farmers, traders, agro-dealers, and processors, etc. Effectiveness The extent to which the project has achieved its objectives according to the results of program indicators To what extent have project outputs been achieved and are these contributing to impact? - Project achievement of physical targets as outlined in the project’s Performance Monitoring Plan (PMP) - Impact of project activities on agriculture (yield quantity and quality), quantity and income from marketable produce, value and utilization of financial services/loans, application of new practices learned during the training, etc. - Desk Review: Project Annual Workplans; Semi-Annual Performance Reports; DevResults Monitoring Database - Beneficiary Survey - Focus Group Discussions - In-Depth Interviews - All Project Beneficiaries, including farmers, traders, agro-dealers, and processors, etc. - Project Staff in Pakistan and HQ - Project Implementing Partners, Sub￾awardees, and Service Providers What support and barriers have affected the achievement of outputs and contribution toward outcomes? - How have government policies affected project progress, achievements, and impact - How have market forces and infrastructure affected project progress, achievements, and impact - Desk Review of Semi￾Annual Performance Reports - Key Informant Interviews - Project Staff in Pakistan and HQ - Implementing Partners, Sub-awardees, and Service Providers What is the learning from project - What has worked well and what needs to be improved with respect - Desk Review of Semi￾Annual Performance Reports - Project Staff in Pakistan and HQ - Implementing Partners, Sub-awardees, and Service Providers 55 experience and how can it be improved? to project design, approach, and methodologies? - How can adjustments be made to improve the project’s contribution? - What financial and human resources are required to successfully bring about these changes? - Key Informant Interviews - Focus Group Discussions - In-Depth Interviews - All Project Beneficiaries, including farmers, traders, agro-dealers, and processors, etc. How have project interventions affected the attitudes and actions of producers and other value chain actors? - Have the project beneficiaries adopted any new practices as a result of participation in the project? - If no, what are the barriers preventing them from doing so? - Beneficiary Survey - Focus Group Discussions - In-Depth Interviews - All Project Beneficiaries, including farmers, traders, agro-dealers, and processors, etc. Are project interventions having the expected effect on the identified value chain constraints? - How has the project led to improvements in the access to inputs and services, increase in quality and quantity of produce, producer capacities, post-harvest operations, and access to markets, etc.? - Beneficiary Survey - Focus Group Discussions - In-Depth Interviews - All Project Beneficiaries, including farmers, traders, agro-dealers, and processors, etc. What factors are facilitating project effectiveness? What factors are hindering it? - What are the policy, regulatory, economic, and attitudinal aspects that drive the project success or hinder progress? - What are the project design aspects and methodologies that have facilitated the achievement of results or posed challenges to implementation? - Desk Review of Semi￾Annual Performance Reports - Key Informant Interviews - Beneficiary Survey - Focus Group Discussions - In-Depth Interviews - Project Staff in Pakistan and HQ - Implementing Partners, Sub-awardees, and Service Providers - All Project Beneficiaries, including farmers, traders, agro-dealers, and processors, etc. Efficiency How the project resources (inputs) Are current project activities being implemented in a - Was the project initiated on time as planned in design? - Desk Review of Semi￾Annual Performance Reports; DevResults Monitoring Database - Project Staff in Pakistan and HQ - Implementing Partners, Sub-awardees, and Service Providers 56 led to the desired results (outcomes)? timely and cost￾effective manner? - Are the activities being implemented in accordance with the Annual Work plans? Key Informant Interviews - Beneficiary Survey - Focus Group Discussions - In-Depth Interviews - All Project Beneficiaries, including farmers, traders, agro-dealers, and processors, etc. Have resources been used cost-effectively? - What are the project’s financial and human resources? - What is the distribution of resources among various project activities? - Have these been used in an effective manner? - Desk Review of DevResults Monitoring Database; Project’s Financial Progress Reports; TORs of Project Staff - Key Informant Interviews - Project Staff in Pakistan and HQ - Implementing Partners, Sub-awardees, and Service Providers Do the quantitative and qualitative results justify the resources expended? - What are the project’s contributions to the planned objectives and outcomes? - Do the project’s effectiveness and impact/contribution justify the allocation of these resources? - Desk Review of DevResults Monitoring Database; - Beneficiary Survey - Focus Group Discussions - In-Depth Interviews All Project Beneficiaries, including farmers, traders, agro-dealers, and processors, etc. Impact The radiation of effects beyond outcome targets, i.e., beyond targeted direct beneficiaries and/or beyond pre￾projected results. Is there evidence that the project is contributing to increased agricultural productivity and expanded trade? - Is there a change in yields, post￾harvest practices, increased market linkages, and incomes, etc.? - Beneficiary Survey - Focus Group Discussions - In-depth Interviews All Project Beneficiaries, including farmers, traders, agro-dealers, and processors, etc. How well is the project meeting its targets? - How many direct and indirect beneficiaries are benefitting? - Are these numbers considerably higher or lower than were planned? - Desk Review of DevResults Monitoring Database; If targets are not being met, why not? - What are the socio-economic, agro ecological, and policy conditions that hinder the project’s achievement of targets? - Desk Review – Cooperative Agreement between Winrock and USDA - Project Staff in Pakistan and HQ - Implementing Partners, Sub-awardees, and Service Providers 57 - What are the design approaches and project management practices that are leading to the shortfall in meeting targets? - Key Informant Interviews If targets are being exceeded, why is it so and were they set too low? - What are the socio-economic, agro ecological, and policy conditions that facilitate the project’s achievement of targets? - Were targets set too low with respect to the current implementation context? - Desk Review – Cooperative Agreement between Winrock and USDA - Key Informant Interviews - Project Staff in Pakistan and HQ - Implementing Partners, Sub-awardees, and Service Providers Sustainability The likelihood that the benefits of the project will endure over time after the project is completed. What measures have been taken to ensure project impacts will endure after the project has ended? To what extent is the intervention contributing to building an enabling environment and making systemic changes that will foster continuous and sustained productivity? - What have been the project’s major achievements thus far? - What is the possibility that these activities will be replicated/up￾scaled by beneficiaries or other public and private sector stakeholders? - What are the potential socio, economic, environmental, attitudinal, policy, etc. threats to the sustainability of continued benefits from these results? - Desk Review of Semi￾Annual Performance Reports - Key Informant Interviews - Beneficiary Survey - Focus Group Discussions - In-Depth Interviews - Project Staff in Pakistan and HQ - Implementing Partners, Sub-awardees, and Service Providers - All Project Beneficiaries, including farmers, traders, agro-dealers, and processors, etc. Learning Knowledge, skills and lessons gained from project implementation. Measures how the PAD allowed for project learning loops and continuous How is feedback from farmers and other key value chain actors reported and recorded? - What are the processes of collecting information from the field? - How is this information recorded? - What are the lines of communication between the project management and beneficiaries? - Desk Review: DevResults Monitoring Database; Monitoring and Progress Reporting; Structure of project Monitoring team - Key Informant Interviews - Focus Group Discussions - In-depth Interviews - Project Staff in Pakistan and HQ - Implementing Partners, Sub-awardees, and Service Providers - All Project Beneficiaries, including farmers, traders, agro-dealers, and processors, etc. 58 improvement and innovation. Are there examples of specific activities that were not in the original project but were undertaken to address a problem/challenge? - What have been the challenges and opportunities encountered by the project that were not foreseen during design? - How has the project responded to these? - Desk Review: Semi￾Annual Progress Reports - Key Informant Interviews - Project Staff in Pakistan and HQ - Implementing Partners, Sub-awardees, and Service Providers What tools are used or could be used to increase and record learning and ensure mistakes are not repeated? - How is the project utilizing monitoring data? - What major implementation decisions have been taken as a result of this information? - Desk Review: Semi￾Annual Progress Reports - Key Informant Interviews - Project Staff in Pakistan and HQ - Implementing Partners, Sub-awardees, and Service Providers 59 ANNEX 03: DATA COLLECTION TOOLS KII SHEETS Project Management Unit (PMU) Project Design 1. What was the logic behind determining the PMP targets? E.g. detailed sector studies, etc. 2. What aspects of the project design were: a. Realistic b. Ambitious c. Under-ambitious 3. What challenges has the above posed for project implementation? Adaptive Management 1. Has the project faced any exogenous or endogenous challenges since its start? E.g. change in government policy, significant climatic variations affecting targeted crops, etc. 2. If yes, how has the project dealt with some of the big challenges to ensure the achievement of its objectives? Staffing 1. Did the project face any difficulties in hiring or retaining of staff? e.g. availability of competent staff, lengthy hiring procedures, etc. Please provide details with examples. 2. How did these challenges with staffing affect project implementation? 3. Were the technical staff, e.g. M&E provided any trainings to improve their capacity in USDA project implementation procedures? If yes, please provide details of trainings, e.g. What was the training provided and When? 4. Was this training found to be useful towards improving project implementation? If yes, how? If no, why not? Monitoring and Evaluation 1. What are the main monitoring tools and processes used by the project? 2. Are the project’s logical results framework and monitoring plan effective in supporting the project? Please explain. 3. What is the process of getting monitoring information? And who are the key stakeholders/staff involved in gathering monitoring information at various levels? 4. What are the major issues faced with regards to collection and validation of Monitoring? E.g. multiple and widespread activities, lengthy processes, etc. 60 5. Has the project undertaken or plans to undertake any measures to overcome these issues? If yes, please provide details 6. Were any of the project targets modified based on results of the baseline survey? If yes, which ones? Why? 7. How is the Monitoring information utilized? E.g. progress reporting, planning and budgeting, course correction, etc. 8. What, if any problems has the PMU faced when preparing progress reports for submission to USDA? 9. What are your recommendations towards improvement of project M&E? 10. Does the project have an impact assessment planned? Planning 1. What is the process of AWP preparation and approvals? E.g. timing, who is responsible, etc? 2. Has the PMU faced any problems with AWP preparation and/or approval? 3. How has this affected the project delivery? E.g. delays, unable to meet targets, etc. Please provide details with examples Finance 1. How has devaluation in PKR affected project operations and targets? 2. What is the general mechanism of budget transfers from PMU to the beneficiaries and partners? 3. Have there been any major problems with financial approvals and disbursements? E.g. delayed approvals, issues with bank accounts, etc. Please elaborate. 4. Also, how have these issues affected project progress and timeliness? 5. And, what measures were taken to mitigate these issues? 6. Does the project make use of appropriate financial recording and tracking softwares? If no, what are the reasons? 7. Has the PMU faced any problems with regards to financial reporting to USDA? If yes, how have these been resolved? 8. Also, has the PMU faced any problems with regards to financial reporting from its partners or grantees, etc. if yes, how have these problems been resolved? 9. Have regular financial audits of the project been undertaken? 10. What have been some of the major audit observations over the years and to what extent were they addressed? 11. What are your recommendations for improving project financial management for the remainder of the PAD project? Monetization 1. Is all the planned monetization completed? 2. What is the total realized value of the soya bean oil monetized? 61 3. Is the total actual value of monetization lower or higher than initially planned? 4. What problems and opportunities did the project face with monetization? 5. How did this affect the project? Timeliness 1. Have all project activities been delivered according to the schedule set out in the project document and work plan? 2. If no, what were some of the major delays and how did these impact project implementation and results? Project Stakeholders and Partners 1. Who are the key project stakeholders and partners and what is the role of each? 2. Which stakeholders/partners have been instrumental in contributing to the project accomplishments? 3. What are the major stakeholder coordination mechanisms? E.g. Quarterly progress reviews? 4. What problems has the project faced with regards to partners? E.g. limited capacity, outreach, delayed reporting, etc. 5. How did these issues affect project implementation? 6. What measures were taken to overcome some of these issues? 7. What are your recommendations for improving project partnership and coordination for the remainder of the PAD project? Sustainability 1. Based on the project’s experience, which of the targeted products have shown the best potential for development? Why? 2. What additional support to this product VC can further enhance the project’s footprint in the long term? 3. Does the project have a documented exit strategy? PROJECT ACTIVITIES I - Farm Credit/Loans 1. Please provide a brief overview of how the potential design of the Loan Guarantee Fund. For instance, how would it operate, how much fund would be allotted, etc. 2. What demand-side activities has the project undertaken to promote formal borrowing among beneficiaries/farmers? 3. What challenges has the project faced with regards to farmer conversions to formal lending sources? 4. What measures have been taken or are planned to overcome these demand-side challenges? 62 5. What supply side activities has the project undertaken to promote agricultural lending? 6. What challenges has the project faced with regards to convincing lenders? 7. What measures have been taken or are planned to overcome these supply-side challenges? 8. What are the project’s recommendations with regards to the future modality of this activity? II - Grants - Equipment and Infrastructure 1. What are the major equipment and infrastructure targeted by the project? 2. Which of these have been in high demand and which ones are in low demand by beneficiaries? What are the reasons for this? 3. What was the reason for selection of these types of equipment and infrastructure? 4. What is the selection criteria of potential grantees? 5. What is the ratio of matching grants provided by the grantees? 6. Have grantees been provided training in O&M? 7. What challenges has the project faced in meeting its targets for this activity? (Grants) 8. What mitigation measures have been taken or are planned to overcome these issues? 9. What are the project’s recommendations with regards to the future modality of the grants activity? III - Grants - Inputs 1. What are the major inputs targeted by the project? 2. Which of these have been in high demand and which ones are in low demand by beneficiaries? What are the reasons for this? 3. What was the reason for selection of these types of inputs? 4. What is the selection criteria of potential grantees? 5. What is the ratio of matching grants provided by the grantees? 6. Have grantees been provided training in O&M? 7. What challenges has the project faced in meeting its targets for this activity? (Grants) 8. What mitigation measures have been taken or are planned to overcome these issues? 9. What are the project’s recommendations with regards to the future modality of the grants activity? IV – Market Access 1. What are some of the challenges and opportunities for connecting the producers and market agents of the four target crops to potential markets? 2. What are the major lessons learned by the project in this regard? V - Public Information Campaign: Disperse Improved Market Information 1. What are the contents of the public information campaign? 2. What are the technical and program features of the Telenor IVR? 63 3. How is the feedback on the IVR assessed in order to make improvements? 4. How is the impact of IVR on subscribers assessed? 5. Will this facility continue to operate after project closure? If yes, who will be the owner and how will its financial sustainability be ensured? VI, VII, IX, X - Training 1. Who developed the training modules? What were the criteria for development? 2. Who is involved in the delivery of trainings? 3. Do the farmer trainings also include aspects of governance, e.g. financial bookkeeping? 4. Who provides training to farmers? 5. What is the duration of each training and outline? 6. Are the outcomes of training monitored just for Tier I or also for Tier II? 7. What are the methods of monitoring training outcome? 8. What are the challenges in training delivery? 9. Are the training curricula/modules reviewed upon participant feedback? How? Provide examples. VIII - Producer Groups, Cooperatives, and Trade Associations 1. Has the project provided any support to any existing trade bodies? E.g. Chilies Grower Associations, PFVA, etc. 2. What have been the particular issues with establishment of formally registered groups? 3. How does this affect the sustainability of project interventions? 4. What measures have been taken to overcome this problem? E.g. dialogue with the registration authority, consideration of alternatives, etc. 5. What potential measures can be taken to ensure that the organizations are 1) registered, and/or 2) sustainable? 6. What is the composition and sector/product/agent/gender spread of the informally formed groups? Training (Tier I) Beneficiary Questionnaire 1. Name: ________________________ 2. Gender: i. Male ii. Female 64 3. Phone Number: 4. District: i. Khairpur ii. Matiari iii. Tando Allahyar iv. Thatta v. Umerkot vi. Khanewal vii. Multan viii. Muzaffargarh vi. Lodhran 5. Major Source of Income (Multiple Response): A. Agri-Farming B. Agriculture Processing C. Agriculture Marketing D. Livestock X. Other (Specify): _____________ 6. Which crops are you primarily involved in the production/processing/marketing of: Types of Products Names of Products (Please Circle) Other 1: Other 2: Other 3: A. Fruits i. Date ii. Banana B. Vegetables i. Red Chili ii. Tomato C. Grains i. Wheat ii. Maize D. Cash Crops i. Cotton ii. Sugarcane E. Fodder Crops 7. In which Crop Did You Receive Project Support (Multiple Response Possible) A) Red Chili B) Tomato C) Banana D) Date X) Other _______ 8. What support did you receive from the project? (Multiple Response Possible) A) Training B) Inputs C) Market Linkages D) Access to Credit E) Infrastructure and Facilities X) Other1_________ Other 2 ___________ Other 3___________ 9. How much area do you grow this crop on: Product Red Chili Tomato Banana Date Area (Acres) 65 Average Production per Season (Maunds) Average Sales per Season (Maunds) Average Selling Price per Maund (PKR) 10. What is the highest education level attained by you? i) Master Degree ii) Bachelor Degree iii) Intermediate iv) Matric v) Middle School vi) Primary School vii) No Formal Education 11. Please rate your language proficiency by answering Yes or No: Speak (Yes/No) Read (Yes/No) Write (Yes/No) A. Sindhi B. Punjabi C. Saraiki D. Urdu E. English Background 12. When did you receive training from the project and what was/were the topic(s) of the training? (Multiple Response Possible) Training Topic Month/Year A. Production B. Harvest C. Post-Harvest D. SPS E. Business Management F. Marketing G Financial Management H. Borrowing and Loans X. Other ______________________ 13. What was the average duration of the training? ______ Hours 66 14. Before the start of training, did the project or trainer ask you any questions about your existing knowledge? i) Yes ii) No Training Delivery 15. In which language was the training delivered? (Multiple Response possible) A. English B. Urdu C. Sindhi D. Punjabi X. Other (Specify):__________ 16. What was the proportion of hands-on and classroom delivery? A. Hands-On Training: ______% B. Classroom Delivery: ______% 17. Are you satisfied with the quality of learning provided in the training? i) Highly Satisfied ii) Somewhat Satisfied iii) Not Satisfied 18. Did you face any logistical problems to participate in the training? (Multiple Response) A. No Problems B. Difficult to Take Time Out for Training C. Distance to the Training Venue D. Difficult to Arrange Transport E. Other (Specify) _______________ 19. Did you face any challenges when attending the training? (Multiple Response) A. No Challenges B. Topics were Difficult to Understand B. Topics were Not Relevant to My Work D. Trainer(s) Was/Were Not Cooperative E. Training Duration Was Too Long F. Training Duration was Too Short G. Training was not at appropriate time (delivered after production/harvest time) X. Other (Specify) _______________ 20. Were you asked to take pre & post-training evaluation tests? i) Yes ii) No 21. If yes, were the tests easy to take? i) Yes ii) Somewhat iii) No 67 22. Also, do you believe that the tests were properly designed to assess the increase in your knowledge? i) Yes ii) Somewhat iii) No Post-Training 23. Have you applied any improved production technique or technology on your ‘target crop’ after attending the training? (Multiple response possible) A. No B. Improved Planting Material C. Land Preparation Techniques D. Improved Nutrient Management E. Pest Management F. Irrigation Techniques X. Other (Please Specify): ___________ 23.1. If yes, how many cropping seasons have you applied the improved techniques and technologies for? i) 1 Season ii) 2 Seasons iii) 3 Seasons iv) >3 Seasons 23.2. And, how much area did you apply these techniques or technologies on? _____ Acres 23.3. Also, how has your crop production benefited from your participation in the project-provided training? (Multiple Response Possible) A. Increased Production Quantity B. Increase Production Quality C. Less Disease or Pest Attacks D. Lower Cost of Production E. Less Product Wastage F. No Benefit X. Other (Specify):____________ 24. Have you adopted any improved harvest or post-harvest handling technique or technology on your ‘target crop’ after attending the training? (Multiple Response) A. No B. Better Picking and Cutting Techniques C. Improved Handling of Produce After Harvest D. Better Packaging Material D. Improved Storage Methods X. Other (Please Specify): ___________ 24.1. If yes, how much yield did you apply these techniques or technologies on? _____ Maunds 68 24.2. Also, how have your crop harvest and post-harvest handling and marketing benefited from your participation in the project-provided training? (Multiple Response) A. Higher Price B. Higher Quantity Marketed C. Access to New Markets/Buyers D. Less Product Wastage E. No Benefit X. Other (Specify): ____________ 25. Did you see an increase in your income from applying the knowledge and skills learned during the training? i) Yes ii) No 26. What problems have you faced with application of learning from the training in your agriculture production, processing, or marketing activities? A. No Problems Faced B. Recommended Inputs are Not Available C. Recommended Inputs are Expensive D. Training Methods are Difficult To Follow X. Other (Please Specify): ___________ 69 27. Have you tried to approach the project staff after receiving the-training to seek support to resolve any of the above issues? i) Yes ii) No 28. Have you received any post-training support from the Project in resolving some of the above issues? i) Yes ii) No 29. If yes, how satisfied are you with the support provided by the project? i) Highly Satisfied ii) Somewhat Satisfied iii) Not Satisfied 30. Have you been able to apply learning from the training to crops/products other than those focused on in the training? i) Yes ii) No 31. If yes, how satisfied are you with the results from the other crops? i) Highly Satisfied ii) Somewhat Satisfied iii) Not Satisfied Training Others 32. Have you shared the learning from training with other farmers? i) Yes ii) No 33. If yes, how: (Multiple Response Possible) No. of Men Farmers No. of Women Farmers A. I have Provided Formal Trainings to Other Farmers B. I have Informally Shared the Knowledge with Other Farmers 34. What problems did you face in training others? (Multiple Response) A. No Problems B. Difficult to Organize Farmers for Training C. Lack of Time D. Difficult to Convince Farmers to Adopt New Methods X. Others (Specify): __________ 70 Market Linkages 35. Have you been able to establish market linkages post-training? i) Yes ii) No 36. If no, do you need support with establishing market linkages? i) Yes ii) No 37. If yes, what kind of support do you need? A. Identifying Buyers B. Product Packaging C. Transport X. Other(Please elaborate) ____________ 38. Has the project offered you any support with regards to marketing? i) Yes ii) No Media Awareness 39. Do you have knowledge of the project’s public information campaign, i.e. Telenor Robocall service, SMS portal? i) Yes ii) No iii) Somewhat 40. Have you utilized the above services to obtain information? i) Yes ii) No 41. If yes, were you satisfied with the information provided through the service? i) Completely satisfied ii) Somewhat satisfied iii) Not satisfied 42. If the service was not utilized, what was/were the reason(s)? (Multiple Responses Possible) A. No Reason B. No Access to Mobile Phone C. Not Interested in Service D. Service is Not Helpful X. Other ______________ Micro-Finance Awareness 43. Have your received any knowledge regarding micro-finance through the project? i) Yes ii) No 44. After receiving the training do you wish to borrow credit from a formal source? i) Yes ii) No 71 45. If yes, has the project helped you with borrowing from a formal source? i) Yes ii) No 46. If no, do you need any assistance from the project for getting credit from a formal source? i) Yes ii) No Training from Other Sources 47. Have you received agriculture production, harvest, post-harvest, or marketing training from any source other than the PAD project? i) Yes ii) No 48. If yes, please provide the following details: Name of Organization Year Training Received 49. Was this training more useful than the training provided by PAD project? i) Not Useful ii) Equally Useful iii) Somewhat More Useful iv) Much More Useful 72 Training (Tier II) Beneficiary Questionnaire 1. Name: ________________________ 2. Gender: i. Male ii. Female 3. Phone Number: 4. District: ii. Khairpur ii. Matiari iii. Tando Allahyar iv. Thatta vi. Umerkot vi. Khanewal vii. Multan viii. Muzaffargarh vii. Lodhran 5. Major Source of Income (Multiple Response): B. Agri-Farming B. Agriculture Processing C. Agriculture Marketing D. Livestock X. Other (Specify): _____________ 6. Which crops are you primarily involved in the production/processing/marketing of: Types of Products Names of Products (Please Circle) Other 1: Other 2: Other 3: A. Fruits i. Date ii. Banana B. Vegetables i. Red Chili ii. Tomato C. Grains i. Wheat ii. Maize D. Cash Crops i. Cotton ii. Sugarcane E. Fodder Crops 7. In which Crop Did You Training from the Project (Multiple Response Possible) B) Red Chili B) Tomato C) Banana D) Date X) Other _______ 8. How much area do you grow this crop on: 73 Product Red Chili Tomato Banana Date Area (Acres) Average Production per Season (Maunds) Average Sales per Season (Maunds) Average Selling Price per Maund (PKR) 9. What is the highest education level attained by you? ii) Master Degree ii) Bachelor Degree iii) Intermediate iv) Matric vi) Middle School vi) Primary School vii) No Formal Education 10. Please rate your language proficiency by answering Yes or No: Speak (Yes/No) Read (Yes/No) Write (Yes/No) F. Sindhi G. Punjabi H. Saraiki I. Urdu J. English Background 11. When did you receive training from the Master Trainer and what was/were the topic(s) of the training? (Multiple Response Possible) Sr. No. Topic Month/Year A. Production B. Harvest C. Post-Harvest Handling D. SPS E. Other _____________ 12. What was the average duration of the training? ______ Hours 74 Training Delivery 13. In which language was the training delivered? (Multiple Response possible) A. English B. Urdu C. Sindhi D. Punjabi X. Other (Specify): ____________ 14. Are you satisfied with the quality of learning provided in the training? i) Highly Satisfied ii) Somewhat Satisfied iii) Not Satisfied 15. Did you face any logistical problems to participate in the training? (Multiple Response) A. No Problems B. Difficult to Take Time Out for Training C. Distance to the Training Venue D. Difficult to Arrange Transport E. Duration of Training was long X. Other (Specify) _______________ 16. Did you face any challenges when attending the training? (Multiple Response) A. No Challenges B. Topics were Difficult to Understand C. Topics were Not Relevant to My Work D. Trainer(s) Was/Were Not Cooperative E. Training Duration Was Too Long F. Training Duration was Too Short G. Training was not time (delivered after production/harvest time) X. Other (Specify) _______________ Post-Training 17. Have you applied any improved production technique or technology on your ‘target crop’ after attending the training? (Multiple Response Possible) A. No B. Improved Planting Material C. Land Preparation Techniques D. Improved Nutrient Management E. Pest Management F. Irrigation Techniques X. Other: _______________ 75 18.1. If yes, how many cropping seasons have you applied the improved techniques and technologies for? i) 1 Season ii) 2 Seasons iii) 3 Seasons iv) >3 Seasons 18.2. And, how much area did you apply these techniques or technologies on? _____ Acres 18.3. Also, has your crop production benefited from your participation in the training? (Multiple Response Possible) A. Increased Production Quantity B. Increase Production Quality C. Less Disease or Pest Attacks D. Lower Cost of Production E. Less Product Wastage F. No Benefit X. Other (Specify):____________ 19. Have you adopted any improved harvest or post-harvest handling technique or technology on your ‘target crop’ after attending the training? A. No B. Better Picking and Cutting Techniques C. Improved Handling of Produce After Harvest D. Better Packaging Material E. Improved Storage Methods X. Other (Please Specify): ___________ 19.1. If yes, how much yield did you apply these techniques or technologies on? _____ Tons 19.2. Also, how have your crop harvest and post-harvest handling and marketing benefited from your participation in the project-provided training? (Multiple Response) A. Higher Price B. Higher Quantity Marketed C. Access to New Markets/Buyers D. Less Product Wastage E. No Benefit X. Other (Specify): ____________ 20. Did you see an increase in your income from applying the knowledge and skills learned during the training? i) Yes ii) No 21. What problems have you faced with application of learning from the training in your agriculture production, processing, or marketing activities? 76 A. No Problems Faced B. Recommended Inputs are Not Available C. Recommended Inputs are Expensive D. Training Methods are Difficult To Follow X. Other (Please Specify): ___________ 22. Have you tried to approach the project staff for support to resolve any of the above issues? i) Yes ii) No 23. Have you received any post-training support from the Project in resolving some of the above issues? i) Yes ii) No 24. If yes, how satisfied are you with the support provided by the project? i) Highly Satisfied ii) Somewhat Satisfied iii) Not Satisfied 25. Have you been able to apply learning from the training to crops/products other than those focused on in the training? i) Yes ii) No 26. If yes, how satisfied are you with the results from the other crops? i) Highly Satisfied ii) Somewhat Satisfied iii) Not Satisfied Training Others 27. Have you shared the learning from training with other farmers? i) Yes ii) No 28. If yes, how: No. of Men Farmers No. of Women Farmers I have Informally Shared the Knowledge with Other Farmers 77 29. What problems did you face in training others? (Multiple Response) A. No Problems B. Difficult to Organize Farmers for Training C. Lack of Time D. Difficult to Convince Farmers to Adopt New Methods X. Others (Specify): __________ Market Linkages 30. Have you been able to establish market linkages post-training? i) Yes ii) No 31. If no, do you need support with establishing market linkages? i) Yes ii) No 32. If yes, what kind of support do you need? A. Identifying Buyers B. Product Packaging C. Transport D. Other ___________ 33. Has the project offered you any support with regards to marketing? i) Yes ii) No Media Awareness 34. Do you have knowledge of the project’s public information campaign, i.e. Telenor Robocall service, SMS portal? i) Yes ii) No iii) Somewhat 35. Have you utilized the above services to obtain information? i) Yes ii) No 36. If yes, were you satisfied with the information provided through the service? i) Completely satisfied ii) Somewhat satisfied iii) Not satisfied 37. If the service was not utilized, what was/were the reason(s)? (Multiple Responses Possible) A. No Reason B. No Access to Mobile Phone C. Not Interested in Service D. Service is Not Helpful X. Other ______________ 78 Micro-Finance Awareness 38. Have your received any knowledge regarding micro-finance through the project? i) Yes ii) No 39. After receiving the training do you wish to borrow credit from a formal source? i) Yes ii) No 40. If yes, has the project helped you with borrowing from a formal source? i) Yes ii) No 41. If no, do you need any assistance from the project for getting credit from a formal source? i) Yes ii) No Training from Other Sources 42. Have you received agriculture production, harvest, post-harvest, or marketing training from any source other than the PAD project Master Trainer? i) Yes ii) No 43. If yes, please provide the following details: Name of Organization Year Training Received 44. Was this training more useful than the training provided by PAD project Master Trainer? i) Much More Useful ii) Somewhat More Useful iii) Equally Useful iv) Not Useful 79 Training Female Labor Beneficiary Questionnaire 1. Name: ________________________ 2. Gender: i. Male ii. Female 3. Phone Number: 4. District: i. Khairpur ii. Matiari iii. Tando Allahyar iv. Thatta vii. Umerkot vi. Khanewal vii. Multan viii. Muzaffargarh viii. Lodhran 5. Major Source of Income (Multiple Response): C. Agri-Farming B. Agriculture Processing C. Agriculture Marketing D. Livestock X. Other (Specify): _____________ 6. Which crops are you primarily involved in the production of: Types of Products Names of Products (Please Circle) Other 1: Other 2: Other 3: F. Fruits i. Date ii. Banana G. Vegetables i. Red Chili ii. Tomato H. Grains i. Wheat ii. Maize I. Cash Crops i. Cotton ii. Sugarcane J. Fodder Crops 7. In which Crop Did You Training from the Project (Multiple Response Possible) C) Red Chili B) Tomato C) Banana D) Date X) Other _______ 8. What is the highest education level attained by you? i) Master Degree ii) Bachelor Degree iii) Intermediate iv) Matric 80 v) Middle School vi) Primary School vii) No Formal Education 9. Please rate your language proficiency by answering Yes or No: Speak (Yes/No) Read (Yes/No) Write (Yes/No) A. Sindhi B. Punjabi C. Saraiki D. Urdu E. English Background 10. When did you receive training from the Master Trainer and what was/were the topic(s) of the training? (Multiple Response Possible) Sr. No. Topic Month/Year A. Production B. Harvest C. Post-Harvest Handling D. SPS X. Other _______ 11. What was the average duration of the training? ______ Hours Training Delivery 12. In which language was the training delivered? (Multiple Response possible) A. English B. Urdu C. Sindhi D. Punjabi X. Other (Specify): ____________ 13. Are you satisfied with the quality of learning provided in the training? i) Highly Satisfied ii) Somewhat Satisfied iii) Not Satisfied 14. Did you face any logistical problems to participate in the training? (Multiple Response) 81 A. No Problems B. Difficult to Take Time Out for Training C. Distance to the Training Venue D. Difficult to Arrange Transport E. Duration of Training was long X. Other (Specify) _______________ 15. Did you face any challenges when attending the training? (Multiple Response) A. No Challenges B. Topics were Difficult to Understand C. Topics were Not Relevant to My Work D. Trainer(s) Was/Were Not Cooperative E. Training Duration Was Too Long F. Training Duration was Too Short G. Training was not at appropriate time (delivered after production/harvest time) X. Other (Specify) _______________ Post-Training 16. Have you applied any improved production technique or technology on your ‘target crop’ after attending the training? (Multiple response possible) A. No B. Improved Planting Material C. Land Preparation Techniques D. Improved Nutrient Management E. Pest Management F. Irrigation Techniques X. Other (Please Specify): ___________ 16.1. If yes, how many cropping seasons have you applied the improved techniques and technologies for? ii) 1 Season ii) 2 Seasons iii) 3 Seasons iv) >3 Seasons 16.2. And, how much area did you apply these techniques or technologies on? _____ Acres 16.3. Also, has your crop production benefited from your participation in the training? (Multiple Response Possible) A. Increased Production Quantity B. Increase Production Quality C. Less Disease or Pest Attacks D. Lower Cost of Production E. Less Product Wastage F. No Benefit X. Other (Specify):____________ 17. Did you see an increase in your income from applying the knowledge and skills learned during the training? i) Yes ii) No 18. What problems have you faced with application of learning from the training in your agriculture production? A. No Problems Faced B. Recommended Inputs are Not Available 82 C. Recommended Inputs are Expensive D. Training Methods are Difficult To Follow X. Other (Please Specify): ___________ 17. Have you tried to approach the project staff for support to resolve any of the above issues? i) Yes ii) No 18. Have you received any post-training support from the Project in resolving some of the above issues? i) Yes ii) No 19. If yes, how satisfied are you with the support provided by the project? i) Highly Satisfied ii) Somewhat Satisfied iii) Not Satisfied 20. Have you been able to apply learning from the training to crops/products other than those focused on in the training? i) Yes ii) No 21. If yes, how satisfied are you with the results from the other crops? i) Highly Satisfied ii) Somewhat Satisfied iii) Not Satisfied Knowledge Sharing 22. Have you shared the learning from training with other farmers? i) Yes ii) No 23. If yes, how: No. of Men Farmers No. of Women Farmers I have Informally Shared the Knowledge with Other Farmers Media Awareness 24. Do you have knowledge of the project’s public information campaign, i.e. Telenor Robocall service, SMS portal? i) Yes ii) No iii) Somewhat 25. Have you utilized the above services to obtain information? i) Yes ii) No 83 26. If yes, were you satisfied with the information provided through the service? i) Completely satisfied ii) Somewhat satisfied iii) Not satisfied 27. If the service was not utilized, what was/were the reason(s)? (Multiple Responses Possible) A. No Reason B. No Access to Mobile Phone C. Not Interested in Service D. Service is Not Helpful X. Other ______________ Training from Other Sources 28. Have you received agriculture production, harvest, post-harvest, or marketing training from any source other than the PAD project Master Trainer? i) Yes ii) No 29. If yes, please provide the following details: Name of Organization Year Training Received 30. Was this training more useful than the training provided by PAD project Master Trainer? ii) Much More Useful ii) Somewhat More Useful iii) Equally Useful iv) Not Useful 84 Credit Beneficiary Questionnaire 1. Name: ________________________ 2. Gender: i. Male ii. Female 3. Phone Number: 4. District: iii. Khairpur ii. Matiari iii. Tando Allahyar iv. Thatta viii. Umerkot vi. Khanewal vii. Multan viii. Muzaffargarh ix. Lodhran 5. Major Source of Income (Multiple Response): D. Agri-Farming B. Agriculture Processing C. Agriculture Marketing D. Livestock X. Other (Specify): _____________ 6. Which crops are you primarily involved in the production/processing/marketing of: Types of Products Names of Products (Please Circle) Other 1: Other 2: Other 3: A. Fruits i. Date ii. Banana B. Vegetables i. Red Chili ii. Tomato C. Grains i. Wheat ii. Maize D. Cash Crops i. Cotton ii. Sugarcane E. Fodder Crops 7. In which Crop Did You Receive Project Support (Multiple Response Possible) A) Red Chili B) Tomato C) Banana D) Date X) Other _______ 8. What support did you receive from the project? (Multiple Response Possible) A) Training B) Inputs C) Market Linkages D) Access to Credit E) Infrastructure and Facilities 85 X) Other1_________ Other 2 ___________ Other 3___________ 9. How much area do you grow this crop on: Product Red Chili Tomato Banana Date Area (Acres) Average Production per Season (Maunds) Average Sales per Season (Maunds) Average Selling Price per Maund (PKR) 10. What is the highest education level attained by you? i) Master Degree ii) Bachelor Degree iii) Intermediate iv) Matric v) Middle School vi) Primary School vii) No Formal Education 11. Please rate your language proficiency by answering Yes or No: Speak (Yes/No) Read (Yes/No) Write (Yes/No) A. Sindhi B. Punjabi C. Saraiki D. Urdu E. English Background 12. Has the project linked you to a formal source of credit? i. No ii. Khushali Microfinance Bank iii. Khushali Bank limited iv. Telenor Microfinance bank v. National Bank vi. First Microfinance bank vii. U Microfinance Bank 13. If yes, which crop did you borrow the money for? (Multiple Response Possible) 86 A. Red Chili B) Tomato C) Banana D) Date X) Other _______ 14. What was the purpose of borrowing? (Multiple Response Possible) A. Land Preparation B. Inputs C. Harvest D. Storage E. Marketing X. Other ___________ 15. When did you obtain this credit? _________ Month/_________Year 16. What is the amount of loan obtained by you through this source? i. PKR 25, 000 ii) PKR 30,000 -PKR 40,000 iii) PKR 50,000 - PKR 60,000 iv. PKR 65,000 - PKR 75,000 v) PKR 90,000 - PKR 100,000 vi) PKR 130,000 or Above 17. What was the interest rate on the loan? _________ % Interest 18. What were the repayment terms? i) Return in _________ Months ii) Return at the time of Harvest iii) Other__________ 19. What was the modality of the loan obtained? i) Individual lending ii) Group lending 20. If it was group lending, how many other farmers did you mobilize to join your group to seek benefit from the loan? i) 1-3 ii) 4-6 iii) 7-9 iv) 10 or more 21. Did you have to provide any collateral or guarantee when obtaining the loan? i) No Collateral ii) Land iii) Livestock iv) Crop iv) Personal or Group Guarantee vi) Other ____________ 22. How did the project help you with accessing the loan? (Multiple Response Possible) A. Training or Awareness About Benefits of Formal Borrowing B. Help with Filling Out the Loan Application C. Linking with Representatives of the Lending Bank X. Other: _______________________ Experience with Formal Credit 87 23. Did you face any challenges with obtaining the loan from the formal source through the project? i) No ii) Difficult to understand the process iii) High interest rate iv) Strict repayment conditions v) Other ___________ 24. Did you face any problems with loan utilization? i) No ii) Loan size is not sufficient iii) Difficult repayment terms iv) Other ___________ 25. Will you continue to borrow through formal lending sources in the future? i) Yes ii) No 26. Was this the first time you borrowed from a formal source? i) Yes ii) No 26.1. If No, how often have you borrowed from a formal source before? i) Only Once ii) A Few Times iii) Regularly 26.2. And, what is the name of the lending source most commonly used by you in the past? ______________________ Experience with Informal Credit 27. How often did you borrow from informal sources in the past to operate your farming enterprise? i) Never ii) A Few Times iii) Regularly 27.1. If yes, what was the most common source of borrowing? i) Commission Agent ii) Agro-Inputs Dealer iii) Market Intermediary iv) Family/Friends v) Other _____________ 27.2. What was the average amount of loan obtained by you through this informal source? i) >=PKR 25, 000 ii) PKR 30,000 -PKR 40,000 iii) PKR 50,000 - PKR 60,000 iv) PKR 65,000 - PKR 75,000 v) PKR 75,000 - PKR 100,000 x) PKR 100,000 or Above 27.3. What was the interest rate on the loan? _________ % Interest 88 27.4. What were the repayment terms? i) Return in _________ Months ii) Return at the time of Harvest iii) Other__________ 27.5. Did you have to provide any collateral or guarantee when obtaining the loan? i) No Collateral ii) Land iii) Livestock iv) Crop ii) Personal or Group Guarantee vi) Other ____________ 27.6. Also, based on your experience, what are the comparative benefits of borrowing from informal vs. formal sources? E.g. difference in interest rates, repayment terms, and availability of associated facilities such as extension advice, control over produce, etc. 28. What are your suggestions for the project to improve its credit linkages program with regards to awareness raising and linkage development? 89 Equipment Grants In-Depth Interview Questionnaire 1. Name: ____________________ 2. Gender: ___________________ 3. Phone Number: ___________________ 4. Major Source of Income (Multiple Responses Possible): A. Agri-Farming B. Agriculture Processing C. Agriculture Marketing D. Livestock X. Other (Specify): _____________ 5. Which crops are you primarily involved in the production/processing/marketing of: Types of Products Names of Products (Please Circle) Other 1: Other 2: Other 3: Fruits i. Date ii. Banana Vegetables i. Chili ii. Tomato Grains i. Wheat ii. Maize Cash Crops i. Cotton ii. Sugarcane Fodder Crops 6. In which Crop Did You Receive Project Support (Multiple Responses Possible) A) Chili B) Tomato C) Banana D) Date X) Other _______ 7. What support did you receive from the project? (Multiple Response Possible) A) Training B) Inputs C) Market Linkages D) Access to Credit E) Infrastructure and Facilities X) Other1_________ Other 2 ___________ Other 3___________ 8. How much area do you grow the following crop(s) on: Product Red Chili Tomato Banana Date Area (Acres) Average Production per Season (Maunds) Average Selling Price per Maund (PKR) 90 Average Price per Maund (PKR) 9. What is the highest education level attained by you? i) Master Degree ii) Bachelor Degree iii) Intermediate iv) Matric v) Middle School vi) Primary School vii) No Formal Education 10. Please rate your language proficiency by answering Yes or No: Speak (Yes/No) Read (Yes/No) Write (Yes/No) Sindhi Urdu English BACKGROUND 11. What is the name of the equipment provided by the project? i. Banana Ripening Unit ii. Banana Tissue Cultured Plant iii. Chili Cold Store and Processing Unit iv. Date Drying Tunnel Refurbishment v. Date Cold Store vi. Drip Irrigation vii. Grain Pro Storage Equipment viii. Trays /Peat Moss 12. When did you receive this equipment: _____________Month/ ____________Year 13. Did you pay any cost-share towards obtaining this equipment/infrastructure? i) Yes ii) No 14. If yes, what was your share? ______% of Project Cost = PKR ___________ 15. In addition to the cost-share, did you also provide any additional monetary or in-kind support? If yes, please provide details. E.g. labour, time, research, etc. 16. How did you hear about the availability of this support from the project? A. Newspaper B. Word of Mouth C. Directly Contacted by the Project X. Other: _______________ 17. What is the processing capacity of this equipment per season: ______ MT/Season 18. How many crop seasons have you used this equipment for? i) Not Used Yet ii) One Season iii) Two Seasons iv) Three or More Seaons 91 19. How much produce did you handle/store/process per season using this equipment? ____________ MT/Season 20. Do you also share this equipment with others? (Multiple Responses Possible) A) No B) Rent Out Space to Other Farmers in the Area C) Let Others Use it for Free X) Other: _____________ 20.1. If yes, how many other farmers use it? _________ Farmers 20.2. And what is the total approximate value of produce these farmers store/process/handle through this equipment? ___________ MT/Season Accruing Benefits 21. How has the use of this equipment help your farm enterprise? e.g. Lower post-harvest losses, Higher Quality Produce, Higher Price Obtained per KG, etc. Please elaborate by providing the before and after situation. 22. Also, have you been able to capitalize on these improvements? E.g. selling to higher paying markets, etc. please elaborate. 23. And were you able to undertake the above actions on your own or with support from the project? Operations and Maintenance 24. Did you face any challenges when accessing the project’s support for getting the equipment? E.g. delayed response, inadequate information, etc. 25. Did you face any problems during the installation process? E.g. inefficient vendor, lack of proper planning, etc. 92 26. Have you faced any problems with regards to operations or maintenance of the equipment/infrastructure? E.g. lack of electricity, unavailability of mechanical parts, lack of technical knowledge to operate, etc. If yes, please provide details. 27. Also, what measures have you or the project taken to resolve these issues? 28. Has any other farmer/processor in your area shown the willingness or initiative to also adopt this technology? If yes, please provide details, e.g. how many farmers, what are their capacities, etc. 29. What are your recommendations to the project for improving its in-kind grants for equipment and infrastructure in the future? E.g. with regards to farmer reach out, selection of technologies, process of disbursing grants, etc. 93 Market Access In-Depth Interview Questionnaire 1. Name: ________________________________ 2. Gender: ________________________________ 3. Phone Number: __________________________ 4. Major Source of Income (Multiple Responses Possible): E. Agri-Farming B. Agriculture Processing C. Agriculture Marketing D. Livestock X. Other (Specify): _____________ 5. Which crops are you primarily involved in the production/processing/marketing of: Types of Products Names of Products (Please Circle) Other 1: Other 2: Other 3: Fruits Date Banana Vegetables Red Chili Tomato Grains Wheat Maize Cash Crops Cotton Sugarcane Fodder Crops Other Crops 6. In which Crop Did You Receive Project Support (Multiple Responses Possible) A) Red Chili B) Tomato C) Banana D) Date X) Other _______ 7. What support did you receive from the project? (Multiple Responses Possible) A) Training B) Inputs C) Market Linkages D) Access to Credit E) Infrastructure and Facilities X) Other1_________ Other 2 __________ Other 3___________ 94 8. How much area do you grow the following crop(s) on: Product Red Chili Tomato Banana Date Area (Acres) Average Production per Season (Maunds) 9. What is the highest education level attained by you? i) Master Degree ii) Bachelor Degree iii) Intermediate iv) Matric v) Middle School vi) Primary School vii) No Formal Education 10. Please rate your language proficiency by answering Yes or No: Speak (Yes/No) Read (Yes/No) Write (Yes/No) Sindhi Urdu English Background 11. How did you hear about the availability of marketing support from the project? A. Newspaper B. Word of Mouth C. Directly Contacted by the Project X. Other: _______________ 12. Why did you decide to utilize this support? E.g. lack of local market, access to low paying markets, etc. Please elaborate. 13. What support did the project provide you with regards to market access? E.g. awareness sessions, market visits, meetings with potential buyers, etc. And when was this support provided? 14. What was the outcome of this support? E.g. ability to engage with new clients, improved sales, etc. Please elaborate. 95 15. How many crop seasons have you supplied to these new clients for? i) Not Used Yet ii) One Season iii) Two Seasons iv) Three or More Seasons 16. How much produce did you sell to these buyers/markets? ____________ MT/Season 17. How did the utilization of this support help improve your income? E.g. comparison of before and after prices and sales volumes, etc. 18. Did you pay any cost-share towards developing these market linkages? ii) Yes ii) No 19. If yes, what was your share? ______% of Project Cost = PKR ___________ 20. In addition to the cost-share, did you also provide any additional monetary or in-kind support? If yes, please provide details. E.g. labour, time, research, etc. 21. Did you face any challenges in meeting the demand of the buyers that the project linked you with? E.g. quantities, quality, packaging, etc. Please elaborate. 22. Do you anticipate having a strong linkage with this client(s) in the future, after the project end? If yes, why. If no, why not? 23. In your opinion, what additional measures can you undertake to improve market linkages? E.g. participation in expos, etc. 24. Also, what additional measures do you need to undertake to improve the marketability of your produce? E.g. improved planting material, pre-processing facilities, storage, packaging, etc. 96 25. What is the estimated cost of bringing about these improvements in your farming/processing/trading enterprise? 26. Has any other farmer/processor/trader in your area shown the willingness or initiative in market linkages? If yes, please provide details, e.g. how many farmers, processors or traders and what are their capacities, etc. 27. What are your recommendations to the project for improving its market linkages interventions in the future? E.g. meetings, expositions, exposure visits, etc. 28. Have you received marketing support from any other project or department in the past? If yes, from who and when? And what were the outcomes of this support? 97 Agro-Dealers In-Depth Interview Questionnaire 1. Name of Business: ___________________ 2. Name of Respondent: ____________________ 3. Gender: ______________________ 4. Phone Number: ________________________ INTRODUCTION 5. When was your business established? _______ Year 6. What are the major products traded by your business? (Multiple Response Possible) A. Seed B. Fertilizer C. Pesticides/Insecticides, etc. D. Farming Equipment X. Other (Specify): ___________ 7. On how much area are the target crops produced by the various category of clients served by you: Area Under Crops Chili (Acres) Tomato (Acres) Banana (Acres) Date (Acres) Small Farmers Medium Farmers Large Farmers PROJECT SUPPORT 8. What support did you receive from the project? (Multiple Response Possible) A) Training B) Inputs C) Linkages D) Infrastructure and Facilities X) Other TRAINING 9. What is the highest education level attained by you? i) Master Degree ii) Bachelor Degree iii) Intermediate iv) Matric v) Middle School vi) Primary School vii) No Formal Education 98 10. Please rate your language proficiency by answering Yes or No: Speak (Yes/No) Read (Yes/No) Write (Yes/No) Sindhi Urdu English 11. When did you receive training from the project and what was/were the topic(s) covered? (Multiple Response Possible) Sr. No. Topic Month/Year A. Seed Selection B. Proper Handling of Stock/Storage C. Inventory Management D. Customer Relations E. Business Management F. Marketing G. Financial Management X. Other ______________ 12. What was the average duration of the training? ______ Hours 13. Of the above, which topics did you find most beneficial? A. Storage B. Safe disposal of expired inputs C. Inventory management D. Customer relationship E. Business Management F. Marketing G. Financial Management X. Other ________ 14. Before the start of training, did the trainer test you on your existing knowledge? i) Yes ii) No 15. Do you believe that the test was properly designed to assess your existing knowledge at the time? i) Yes ii) Somewhat iii) No Training Delivery 99 16. In which language was the training delivered? (Multiple Response Possible) A. English B. Urdu c. Sindhi X. Other (Specify): ____________ 17. What was the proportion of hands-on and classroom delivery? Hands-On Training: ______% Classroom Delivery: ______% 18. Are you satisfied with the quality of learning provided in the training? i) Highly Satisfied ii) Somewhat Satisfied iii) Not Satisfied 19. Did you face any logistical problems to participate in the training? (Multiple Response Possible) A. No Problems B. Difficult to Take Time Out for Training D. Distance to the Training Venue C. Difficult to Arrange Transport X. Other (Specify) _______________ 20. Did you face any challenges when attending the training? (Multiple Responses possible) A. No Challenges B. Topics were Difficult to Understand C. Topics were Not Relevant to My Work D. Trainer(s) Was/Were Not Cooperative E. Training Duration Was Too Long F. Training Duration was Too Short G. Training was not time (delivered after production/harvest time) X. Other (Specify) _______________ 21. Did you receive any other support due to participation in the training program, etc.? i) Yes ii) No 21.1. If yes, what support did you receive? (Multiple Response Possible) a.Input 1:___________ b. Input2: ____________ c. Input 3: __________ Support 1: _____________ Support 2: ____________ Support 3: _________ 22. Were you asked to take a post-training evaluation test? i) Yes ii) No 23. If yes, was the test easy to take? i) Yes ii) Somewhat iii) No 100 24. Also, do you believe that the test was properly designed to assess the increase in your knowledge? i) Yes ii) Somewhat iii) No Post-Training 25. Have you started selling any improved production inputs or technologies after attending the training? (Multiple Response Possible) A. No B. Improved Planting Material C. Improved Pesticides B. Improved Nutrient Management E. Improved Production Equipment X. Other1.__________ Other 2.___________ Other 3.___________ 25.1. If yes, approximately what percentage of your clientele have purchased these improved production inputs or technologies from you? (Multiple Response Possible) A. _______% Small Farmers B. _______% Medium Farmers C. _____% Large Farmers 25.2. And, what has been the improvement in farm productivity of these clients? (Multiple Response Possible) A. Improved Germination B. Higher Yields C. Better Quality Produce D. Lower Cost of Production E. Less Disease of Pest Attacks X. Other1.__________ Other 2.___________ Other 3.___________ 26. Have you started selling any improved harvest or post-harvest inputs or technologies after attending the training? (Multiple Response Possible) A. No B. Better Picking and Cutting Equipment C. Improved Handling Equipment D. Better Packaging Material X. Other1.__________ Other 2.___________ Other 3.___________ 101 26.1. If yes, approximately what percentage of your clientele has purchased these improved post-harvest inputs or technologies from you? (Multiple response possible) A. _______% Small Farmers B. _______% Medium Farmers C. _____% Large Farmers 26.2. And, according to your knowledge, how have these clients benefited from using these technologies? (Multiple Response Possible) A. Higher Price B. Higher Quantity Marketed C. Access to New Markets/Buyers D. Less Product Wastage E. No Benefit X. Other 1.__________ Other 2.___________ Other 3.___________ 27. Have you already seen or expect to see any particular benefits to your business from attending the training? (Multiple Response Possible) A. Increased Variety of Products B. Increase in Number of Clients C. Increase in Revenue D. Improved Business Reputation E. Improved Financial Management F. Proper Management of Inputs and Stocks G. Improved Inventory Management H. Improved Customer Relations I. Developed Linkages with Other Suppliers X. Other1.__________ Y. Other 2.___________ Z. Other 3.___________ 28. What problems have you faced with improving your business products or practices when applying results from the training? (Multiple Response Possible) A. No Problems Faced B. Recommended business management practices adoption will take time X. Other1.__________ Other 2.___________ Other 3.___________ 29. Have you tried to approach the project staff for support to resolve any of the above issues? i) Yes ii) No 30. Have you received any post-training support from the Project in resolving some of the above issues? i) Yes ii) No 30.1. If yes, how did the project assist you? (Multiple Response Possible) A. Linking to farmers B. Integrating with in-kind grant activities C. Follow Up Training X. Other1.__________ Other 2.___________ Other 3.___________ 102 30.2. And how satisfied are you with the support provided by the project? i) Highly Satisfied ii) Somewhat Satisfied iii) Not Satisfied 31. Knowledge Sharing Have you shared the learning from training with your team? i) Yes ii) No 32. If yes, how: No. of Men No. of Women I have Informally Shared the Knowledge with my team 33. What problems did you face in sharing the knowledge with others? (Multiple Response) A. No Problems B. Difficult to Convince People to Adopt New Methods C. Difficult to Organize People for Training D. Lack of Time X. Others (Specify): __________ Recommendations 34. What have been the general benefits derived by your clientele and business as a result of your participation in the training program? 35. Have your clientele or you faced any challenges with regards to application of the learned knowledge and techniques? If yes, please elaborate: 36. What are your recommendations for improving the contents of the training program in the future? 103 37. What are your recommendations for improving other aspects of the training program, e.g. outreach, logistics, etc.? 104 Input Grants Focus Group Discussion Introduction 1. Major Source of Income? 2. Which crops are you primarily involved in the production/processing/marketing of: 3. In which Crop Did You Receive Project Support 4. What support did you receive from the project? (Multiple Response Possible) 5. How much area do you grow the following crop(s) on: Product Red Chili Tomato Banana Date Area (Acres) Average Production per Season (Maunds) 6. What is the highest education level attained by you? 7. Please rate your language proficiency by answering Yes or No: Speak (Yes/No) Read (Yes/No) Write (Yes/No) Sindhi Urdu English Background 8. What is the name of the input provided by the project? (Multiple Response Possible) 9. When did you receive this input? 10. Did you pay any cost-share towards obtaining this input? 11. If yes, what was your share? 12. How did you hear about the availability of this support from the project? 105 13. What is the quantity of the input provided to you by the project? 14. How many crop seasons have you used this input for? Accruing Benefits 15. How have these inputs helped your farming operation? e.g. Higher Yields, Higher quality produce, Lower post-harvest losses, Higher Price Obtained per KG, etc. Please elaborate by providing the before and after situation. 16. Also, have you been able to capitalize on these improvements? E.g. selling to higher paying markets, etc. please elaborate. 17. And were you able to undertake the above actions on your own or with support from the project? Operations and Maintenance 18. Did you face any challenges when accessing the project’s support for getting the inputs? E.g. delayed response, inadequate information, etc. 19. Have you faced any problems with regards to using the inputs? E.g. lack of technical knowledge to operate, faulty inputs, etc. If yes, please provide details. 20. Also, what measures have you or the project taken to resolve these issues? 21. Are you planning to continue using these inputs in the future even without the project’s help? If yes, do you know where to obtain these inputs from and the process to obtain them? 22. What potential challenges do you foresee in obtaining these inputs without the project’s support? E.g. lack of affordability, lack of access to market, etc. 106 23. Has any other farmer in your area shown the willingness or initiative to also adopt these new types of inputs? If yes, please provide details, e.g. how many farmers, what are their capacities, etc. 24. Were any farmers in your area already using these inputs? If yes, approximately what percentage of farmers? And what prevented you from you using these inputs before project support? 25. What are your recommendations to the project for improving its in-kind grants for inputs in the future? E.g. with regards to farmer reach out, selection of technologies, process of disbursing grants, etc. 107 Producer Groups Focus Group Discussion Demographics 1. When was your group formed? __________Year 2. Who supported the establishment of this group? i) PAD Project ii) Self Help iii) Other (Specify): _________ 3. How many members does your group have? ________Men, _______Women 4. What is the average landholding of the group members? __________ Acres 5. What is the average area on which the group members grow the target product? _______Acres Background 6. How did your group learn about the project? 7. What was the purpose of forming the group? Please elaborate. 8. What was the process of group formation? Please elaborate. 9. How have farmers benefited by becoming members of your group? Please elaborate 10. What is the criterion for becoming a group member? E.g. production of a particular crop, paying membership dues, etc. Please provide details, e.g. how much is the membership fee, etc. Project Support 11. What support have members of your group received from the project? e.g. training, inputs, infrastructure, linkages, etc. Please provide details. E.g. what inputs, linkages, trainings were provided, when, etc. 12. Which support/intervention by the project is mostly highly valued by members of your group? Why? 108 13. How has utilization of project support helped the members of your group improve their farm production? E.g. more production, less cost, etc. Please elaborate by providing the before and after situation. 14. How has utilization of project support helped the members of your group improve their harvest and post-harvest operations? E.g. lesser post-harvest losses, higher price, etc. Please elaborate by providing the before and after situation. 15. Also, has your group faced any challenges when trying to utilize project support? e.g. recommended inputs not available locally, practices are high cost, etc. Please provide details. 16. Has your group faced any challenges when accessing support or facilities provided by the project? E.g. difficult to access packages, limited support as compared to demand, lack of timely support, etc. Governance and Management 17. What are the criteria for becoming a group member? E.g. production of a particular crop, paying membership dues, etc. Please provide details, e.g. how much is the membership fee, etc. Sustainability 18. Does your group expect to continue operations even after project support ends? If yes, what activities are expected to continue in the future? 19. Does your group currently receives or foresees receiving any support from any other public or private sector organizations? If yes, please provide details. 20. What potential challenges does your group foresee for operations beyond the life of PAD project? 109 Training (Tier I) Focus Group Discussion (FGD) Background 1. How did you hear about the project? 2. What was the process and criteria of being selected as a training beneficiary? E.g. being member of a group, engagement in production of a crop, etc. 3. When did you receive training from the project? _________ Year 4. What were the topics of the training and what was the duration of the training? 5. Did the project or trainer ask you questions before the start of training about your existing knowledge? If yes, please provide details of what questions were asked. 6. Did you agree with the results of this pre-test? If no, why not? Training 7. Did you face any logistical challenges to attend the training? E.g. distance, timing, etc. 8. Did you have any problems when attending the training? E.g. difficult to follow, irrelevant topics, high focus on classroom vs. hands-on training, etc. 9. Did you receive any inputs due to participation in the training program, e.g. seed, fertilizer, etc.? if yes, please provide details? 10. Also, did you have any issues with the inputs provided? E.g. limited quantity, low quality, etc. 110 11. Were you asked to take a post-training evaluation test? If yes, do you believe that the test results correctly reflected your ability after the training? Please elaborate. 12. Did you have any problems taking the test? If yes, please elaborate. Post-Training 13. How has application of training helped with improving your agricultural production, processing, or marketing in the focused product? e.g. higher quality product, higher quantities, less wastages, access to new markets, etc. Please elaborate. 14. How has attending the training helped with improving your agriculture income? Please provide the before and after incomes. 15. What challenges did you face when applying the knowledge and techniques from the training to your agriculture-related activities? E.g. lack of availability of recommended inputs, high costs, etc. 16. Were you able to seek help from the project to overcome some of these problems? If yes, please provide details on how the project helped you? 17. Do you intend to continue using the knowledge & techniques transferred by the training? If yes, why? If no, why not? Training Others 18. Have you been able to train other farmers in the knowledge and techniques learned by you during the training? If yes, please provide details, e.g. was this a formal or informal exchange, the topics that you trained others in, and number of farmers trained, etc. 19. Also, what challenges did you face when training others? E.g. lack of time, difficult to convince others, etc. 111 20. Have other farmers in your area also started using these methods and techniques? If yes, how many farmers and what techniques? According to your information, what have been the results? Recommendations 21. What are your recommendations for improving the contents of the training program? 22. What are your recommendations for improving the logistics and other arrangements of the training program? Training from Other Sources 23. Have you received agriculture production, harvest, post-harvest, or marketing training from any source other than the PAD project? If yes, please provide details, including name of organization, year training received, and topics of training, etc. 24. Was this training more useful than the training provided by PAD project? Please provide details. 112 Training (Tier II) Focus Group Discussion (FGD) Background 1. What was the process and criteria of being selected as a training beneficiary? E.g. being member of a group, engagement in production of a crop, etc. 2. When did you receive training? _________ Year 3. Who delivered the training and what was the duration of the training? E.g. a member of your community, professional trainer, or group of trainers, etc. 4. What were the topics of the training? 5. Did the project or trainer ask you questions before the start of training about your existing knowledge? If yes, please provide details of what questions were asked. 6. If yes, did you agree with the results of this informal pre-test? If no, why not? Training 7. Did you face any logistical challenges to attend the training? E.g. distance, timing, etc. 8. Did you have any problems when attending the training? E.g. difficult to follow, irrelevant topics, high focus on classroom vs. hands-on training, etc. Post-Training 9. How has application of training helped with improving your agricultural production, processing, or marketing in the focused product? e.g. higher quality product, higher quantities, less wastages, access to new markets, etc. Please elaborate. 113 10. How has attending the training helped with improving your agriculture income? Please provide the before and after incomes. 11. What challenges did you face when applying the knowledge and techniques from the training to your agriculture-related activities? E.g. lack of availability of recommended inputs, high costs, etc. 12. Were you able to seek help from the trainer to overcome some of these problems? If yes, please provide details on how the project helped you? Knowledge Sharing 13. Have you been able to share the information with other farmers regarding the knowledge and techniques learned by you during the training? If yes, please provide details, e.g. was this a formal or informal exchange, the topics that you trained others in, and number of farmers trained, etc.? 14. If yes, have other farmers in your area also started using these methods and techniques? If yes, how many farmers and what techniques? According to your information, what have been the results? 15. Also, what challenges did you face when training/sharing knowledge with others? E.g. lack of time, difficult to convince others, etc. Recommendations 16. What are your recommendations for improving the contents of the training program? 17. What are your recommendations for improving the logistics and other arrangements of the training program? Training from Other Sources 18. Have you received agriculture production, harvest, post-harvest, or marketing training from any source other than the PAD project? If yes, please provide details, including name of organization, year training received, and topics of training, etc. 114 19. Was this training more useful than the training provided through the PAD project? Please provide details. 115 KEY INFORMANT INTERVIEW (KII) SHEET PAKISTAN AGRICULTURE DEVELOPMENT (PAD) PROJECT SUB-GRANTEES ❏ Name of the Respondent ❏ Designation ❏ Contact Details ❏ Date of KII ❏ Starting Time of KII ❏ Finishing Time of KII 116 KII SHEETS SUB-GRANTEES 1. Since when has Bureau Veritas been working with the project? And was Bureau Veritas involved in the design as well? 2. What qualifies Bureau Veritas to work with the project? What is your unique contribution? 3. What is Bureau Veritas’ role? 4. How many people have you reached so far? 5. What is the actual or potential impact? 6. What is the process of training delivery, e.g. selection of farmers, etc.? 7. Do you provide literature aid/handouts to the trained farmers to use after the training? 8. How do you coordinate with Winrock? 9. How often do you report and what have been the challenges with reporting? 10. How much staff do you have? 11. How do you monitor the services? 12. What have been the challenges in coordination, planning, and delivery? 13. How can these be resolved? 14. Which are the most and least sustainable activities of the project? And why? 15. What are their lessons learned and recommendations for future projects? 117 KEY INFORMANT INTERVIEW (KII) SHEET PAKISTAN AGRICULTURE DEVELOPMENT (PAD) PROJECT SUB-GRANTEES 1. Name of the Respondent 2. Designation 3. Contact Details 4. Date of KII 5. Starting Time of KII 6. Finishing Time of KII 118 KII SHEETS SUB-GRANTEES 1. Since when have Haji Sons been working with the project? And was Haji Sons involved in the design as well? 2. What qualifies Haji Sons to work with the project? What is your unique contribution? 3. What is Haji Sons’ role? 4. How many people have you reached so far? 5. What is the actual or potential impact? 6. What is the process of training delivery, e.g. selection of farmers, etc.? 7. Do you provide literature aid/handouts to the trained farmers to use after the training? 8. How are the dealers for agro-dealers training selected? 9. In the case of farmers provided with tomato and red chili seeds, does the company do any follow-ups? What is the process of follow-ups? 10. How are farmer complaints or problems addressed? Please give examples. 6. How do you coordinate with Winrock? 7. How often do you report and what have been the challenges with reporting? 8. How much staff do you have? 9. How do you monitor the services? 10. What have been the challenges in coordination, planning, and delivery? 11. How can these be resolved? 12. Which are the most and least sustainable activities of the project? And why? 13. What are their lessons learned and recommendations for future projects? 119 KEY INFORMANT INTERVIEW (KII) SHEET PAKISTAN AGRICULTURE DEVELOPMENT (PAD) PROJECT 7. Name of the Respondent 8. Designation 9. Contact Details 10. Date of KII 11. Starting Time of KII 12. Finishing Time of KII 120 KII SHEET WINROCK INTERNATIONAL (WI) HEADQUARTER PROJECT DESIGN & DELIVERY 4. What was the process of designing the project and who was involved in the process, e.g. WI Pakistan Office, Sub-Awardees, etc.? 5. What was the logic behind determining the PMP targets? E.g. detailed sector studies, etc. 6. What role does the HQ play in the implementation of the project and what role is played by Pakistan office? 7. What major administrative and programming challenges have been faced by WI in project delivery? MONETIZATION 6. Is all the planned monetization completed? 7. What is the total realized value of the soya bean oil monetized? 8. Is the total actual value of monetization lower or higher than was initially planned? 9. What problems and opportunities did the project face with monetization? 10. How did this affect the project? FINANCE 12. How has devaluation in PKR affected project operations and targets? 13. Has WI faced any problems with regards to financial reporting to USDA? If yes, how have these been resolved? 14. Also, has the PMU faced any problems with regards to financial reporting from its partners or grantees, etc. if yes, how have these problems been resolved? 15. What are your recommendations for improving project financial management for the remainder of the PAD project? EFFECTIVNESS and SUSTAINABILITY 4. In your opinion, what are the most notable achievements of the project? Why? 5. Based on the project’s experience, which of the targeted products have shown the best potential for sustainability? Why? RECOMMENDATIONS 1. What are the key lessons learned from the following: a. Project Design 121 b. Project Implementation 2. What are your recommendations for the project’s activities going forward? 3. What are your recommendations for improving the project’s operations going forward? 122 KEY INFORMANT INTERVIEW (KII) SHEET PAKISTAN AGRICULTURE DEVELOPMENT (PAD) PROJECT 13. Name of the Respondent 14. Designation 15. Contact Details 16. Date of KII 17. Starting Time of KII 18. Finishing Time of KII 123 KII SHEETS SHAN FOODS 1. How long have you been working with the PAD project? 2. What activities have you been involved with in the project? 3. What critical measures would the project or farmers need to take in order to be able to sell the produce to your company? 4. How much investment would this require? 5. How long do you think will it take the PAD farmers to reach that stage? 6. Has your company worked with another development organization like Winrock in the past? If yes, what has your experience been? 124 KEY INFORMANT INTERVIEW (KII) SHEET PAKISTAN AGRICULTURE DEVELOPMENT (PAD) PROJECT SUB-GRANTEES 19. Name of the Respondent 20. Designation 21. Contact Details 22. Date of KII 23. Starting Time of KII 24. Finishing Time of KII 125 KII SHEETS SUB-GRANTEES SPROUTS BIOTECH LAB 1. Since when has your business been operating? 2. What are the major products marketed by you? 3. How many clients do you have in Pakistan and where are they generally located? 4. Since when have you been interacting with the project? 5. Do you have any other competitors in the country? If yes, how do you compare your business with them in terms of products marketed, size, and number of clients? 6. What are your particular responsibilities with respect to PAD? 7. How many tissue culture banana plants have you sold through the matching grants program of PAD? 8. And how many additional plants have you sold to PAD farmers? 9. Has demand for banana plants by farmers outside PAD beneficiaries increased since you started participation in the project? 10. What have been some of your challenges in participation in the program? E.g. meeting high demand, coordination, reporting, etc.? 11. How were these challenges overcome? 12. Does your company have a training or follow up component in the project as well? If yes, how does it work? 13. What challenges do you foresee in meeting the rest of your targets under PAD? 14. What support would you require to overcome these challenges? 15. What are your key lessons from the experience of implementing PAD project and what are your recommendations for similar future projects? 126 KEY INFORMANT INTERVIEW (KII) SHEET PAKISTAN AGRICULTURE DEVELOPMENT (PAD) PROJECT 25. Name of the Respondent 26. Designation 27. Contact Details 28. Date of KII 29. Starting Time of KII 30. Finishing Time of KII KII SHEETS TELENOR 1. Since when has Telenor been engaged with PAD in disseminating the public awareness messages? 2. What is the content of the messages and who are the main subscribers? 3. How do people come to know about this service? 4. Are there any conditions for subscribing? E.g. fees, geographic presence, etc. 5. How many subscribers do you have thus far? 6. Does Telenor have a way to assess the effectiveness of this service? If yes, what have been the findings? 7. Will Telenor continue to provide this service even after the contract with PAD ends? If yes, what will be the source of funds? 8. Based on your experience of implementing this activity, what are your lessons learned and recommendations for improvement? 9. Has Telenor worked on other similar initiatives in the past, either funded by Telenor or a third party? If yes, how would you compare the results of this service with the others? 128 ANNEX 4: SAMPLING METHODOLOGY 1.1. SAMPLING METHODOLOGY To undertake the evaluation, both quantitative as well as qualitative information was collected through a combination of primary and secondary sources. For this purpose, the Consultant’s team traveled to selected PAD sites in sampled districts of Sindh and Punjab. Using a participatory approach, the evaluation was carried out by collecting information at the (i) organizational level through conducting Key Informant Interviews as well as (ii) the community level through conducting Focus Group Discussions, Performance Survey, and In-Depth Interviews. A summary of the field activity against respective PAD activities and sample sizes is provided below in Table 1. TABLE 1: SUMMARY OF FIELD ACTIVITY Data Collection Tool Respondent Type PAD Activity Covered No. of Respondents/Groups Survey Men and Women Beneficiaries of Various Activities Against Each Commodity 1,3,6,7,10 796 FGD Men and Women Beneficiaries of Various Activities 3,6,7,8,10 12-18 IDI Men and Women Beneficiaries of Various Activities 2,4,9 16 KII Institutional Stakeholders 5 6 1.2. SAMPLING BREAKDOWN Details of sampling for KIIs, IDIs, FGDs and Performance Survey are presented in the following sections. 1.2.1. PERFORMANCE SURVEY As part of the evaluation, a quantitative performance survey of stakeholders using an appropriate sample size from the direct beneficiaries was conducted. The survey was aimed to reach Tier-1 and Tier-2 farmers involved in PAD activities reaching the highest number of beneficiaries, i.e. Activity 3, Activity 6, Activity 7, and Activity 10. Since the project beneficiaries of these activities are classified according to their respective crops/commodities, therefore, keeping population as the number of beneficiaries covered against each commodity, confidence level at 95%, and confidence interval at 7%, the Consultant drew a sample of 746 beneficiaries to be targeted during the course of the survey. Using multi-stage stratified random sampling, the sample was divided into subgroups or 129 strata based on distribution of population among type of beneficiary (as shown in the following table) and random samples were taken from each of the strata created. 130 TABLE 2: CROP WISE SAMPLE DISTRIBUTION ACROSS BENEFICIARY TYPES Crop/Activity Beneficiary Type Unique Farmer Beneficiaries of Trainings (Activity: 6,7,10) + Activity 3 Population Overall Samples (C.I. 7%) Beneficiary Type Wise Sample Distribution Banana Master Trainer 733 3,158 185 43 Producers 2,425 142 Red Chili Female Labor 257 7,031 191 8 Master Trainer 1,052 29 Producers 5,619 153 Inputs Grantees 103 2 Tomato Master Trainer 815 2,687 183 57 Producers 1,807 123 Inputs Grantees 65 4 Date Master Trainer 857 4,043 187 141 Producers 3,186 146 Total 16,919 746 In addition, the project reached a sizable amount of beneficiaries through Activity 1. Therefore, keeping population according to the number of beneficiaries of the activity, confidence level at 95%, and confidence interval at 10%, the Consultant drew a sample of 50 beneficiaries for Activity 1. The sample was further stratified crop-wise based on the distribution of total population in the two crops, as shown below. TABLE 3: SAMPLE DISTRIBUTION FOR PAD ACTIVITY 1 Crop/Activity Population Total Sample (C.I. 10%) Red Chillies 28 14 Tomatoes 77 36 Total 105 50 Thus, the survey sample size adds up to a total of 796 beneficiaries. During the second stage of sample distribution34, the sample was further classified into groups based on the district sample belongs to. For instance, if Master Trainers for Banana in Matiari and Tando Allahyar comprised of 38% and 62% respectively of the total number of Master Trainers for Banana in Sindh; a sample of 38% of the beneficiaries to be interviewed would be selected from Matiari, and 62% from Tando Allahyar out of the total sample size of 43 Master Trainer beneficiaries for Banana in Sindh. Table 6 below provides a detailed distribution of sample size across target districts within Sindh province. TABLE 4: 2ND STAGE SAMPLE DISTRIBUTION (DISTRICT LEVEL – SINDH) Crop District Number of Beneficiaries for Activity 3, 6, 7 and 10 Number of Beneficiaries for Activity 1 Total Number of Beneficiaries for Activity 1, 6, 7 and 10 Red Chili Umerkot 155 14 169 Banana Tando Allah Yar 117 0 117 Matiari 68 0 68 Tomato Thatta 95 36 131 Date Khairpur 187 0 187 Total Beneficiaries 622 50 672 Sample distribution for Punjab province is detailed in table 7 below: 34 Purposive Random Sampling was undertaken at this stage, i.e. for each commodity, districts with the highest number of beneficiary population were selected to be included in the survey for the sake of efficiency. 132 TABLE 5: 2ND STAGE SAMPLE DISTRIBUTION (DISTRICT LEVEL – PUNJAB) Crop District Number of Beneficiaries for Activity 6, 7 and 10 Red Chili Khanewal 15 Lodhran 36 Tomato Muzzafargarh 51 Multan 22 Total Beneficiaries 124 ANNEX 05: LIST OF KEY INFORMANTS MET/INTERVIEWED Interviews with Stakeholders / Key Informants Counterpart/Organization Representative(s) Title Date Time Winrock International, Pakistan / PAD PMU Mr. Farrukh Mirza; Mr. M. Iqbal; Mr. M. Moazzam; Mr. Abdullah Khan Chief of Party (PAD); Deputy Chief of Party (PAD); Director Finance & Administration; M&E Manager 02-Apr-20 11:00 am-1:00 pm Winrock International Headquarters Mr. Jeffery Apigian Senior Programme Officer 09-Apr-20 08:00-9:00 pm Haji Sons Mr. Tahir Saleemi Chief Executive Officer 09-Apr-20 3:00-04:00 pm Bureau Veritas Mr. Tariq Qamar Deputy Chief Executive Officer 08-Apr-20 03:00-04:00 pm Sprouts Biotech Laboratories Mr. Nausherwan Haider Managing Partner 08-Apr-20 04:30-05:15 pm Telenor Pakistan Mr. Asim Zahoor Assistant Manager mAgri 13-Apr-20 05:00-05:30 pm ANNEX 06: DOCUMENTS REVIEWED 1. PAD Organization Chart 2. Agreement for the PAD project 3. PAD Results Framework 4. PAD Performance Monitoring Plan 5. PAD Baseline Study Report 6. In-Kind Grants tracker 7. PAD performance reports (bi-annual reports) 8. PAD DevResults Database System (including beneficiary database and training records) 9. Post-Harvest Market Analysis for Horticulture in Punjab and Sindh 10. Annual Work Plans ANNEX 07: PAD ORGANOGRAM ANNEX 08: ACTIVITY WISE FINANCIAL EXPENDITURE ACTIVITY-WISE FINANCIAL EXPENDITURE (AS OF MARCH 2020) ACTIVITY ALLOCATED BUDGET (USD) ACTUAL BUDGET SPENT PERCENT BUDGET SPENT Activity I: Micro-credit 419,764 74,424 18% Activity II: Equipment Grants 4,199,649 466,594 11% Activity III: Input Grants 761,951 84,790 11% Activity IV: Market Access 567,574 94,240 17% Activity V: Information Campaign 600,504 145,497 24% Activity VI: Improved agricultural production techniques 541,792 187,508 35% Activity VII: Post-harvest handling and processing 688,481 170,706 25% Activity VIII: Producer Groups 429,942 43,542 10% Activity IX: Agro-Dealers 456,609 68,408 15% Activity X: Training SPS 482,753 37,068 8% 137 ANNEX 09: EFFECTIVENESS AND IMPACT PAD ACTIVITY Target no. of Beneficiaries Actual Target Achieved (As of Mar 2020) Percent Target Achieved (As of Mar 2020) Data Sources Activity I: Micro-credit 6,800 loans 126 loans 2% PADs Micro Finance Database Activity II: Equipment Grants NA 93 approved grantees NA PADs In-king Grant database Other individuals benefiting from operational infrastructures provided through grants: 107, reported Bi-Annual reports Total: 200 Activity III: Input Grants NA 2 approved grantees NA PADs In-Kind Grant database Other individuals benefiting from inputs provided through these grants on subsidy: 2,118 PADs In-Kind Grant database Total: 2,120 Activity IV: Market Access 20 agreements 9 agreements 45% PADs Linkages database Activity V: Information Campaign 395,000 individuals 900,233 individuals 228% PADs Comms database Activity VI: Improved agricultural production techniques 35,820 21,141 59% PADs attendance database Banana Farmers 7,200 4,199 58% Dates Farmers 8,100 5,388 67% Red Chili Farmers 11,610 7,572 65% Tomatoes Farmers 8,910 3,982 45% Activity VII: Post￾harvest handling and processing 35,820 20,409 57% PADs attendance database Banana Farmers 7,200 4,164 58% Dates Farmers 8,100 5,031 62% Red Chili Farmers 11,610 7,284 63% Tomatoes Farmers 8,910 3,930 44% Activity VIII: Producer Groups 170 groups 132 groups 78% PADs group tracker Activity IX: Agro￾Dealers 50 dealers 64 dealers 128% PADs attendance database Activity X: Training SPS 4,110 individuals 2,143 individuals 52% PADs attendance database Banana Farmers 800 442 55% Dates Farmers 900 500 56% Red Chili Farmers 1,290 687 53% Tomatoes Farmers 990 514 52% Other 130 0 0% ANNEX 10: PERFORMANCE INDICATOR ANNEX Result Increased use of improved agricultural techniques & technologies Indicator# - SI￾01 Number of hectares of land under improved techniques or technologies as a result of USDA assistance Type - Outcome Target - 6,448 Hectares The MTE team found that as a result of the PAD project interventions, 85% of the Tier-I and Tier-II surveyed respondents reported having applied production techniques learned during the various trainings on an average area of 1 Hectare and 0.5 Hectares, respectively. The improved techniques adopted as a result of these trainings include adoption of improved planting material, land preparation, nutrient management, pest management, and irrigation techniques. In addition, the following technologies introduced/supported by the project were adopted by farmers: 1. Drip irrigation, which can result in time saving (from 4 hours to 30 minutes for a 5 Acre/2 Hectare plot), 60% reduction in water usage, and 50% reduction in labor and lower incidents of disease. 2. Plug trays and peat moss for nursery raising of chili and tomato. This technology can help save PKR 16,000 – PKR 24,000 worth of seed per acre. 3. Production of tissue culture plants of banana on 25 hectares. As reported in the latest Semi-Annual Performance Report of the project, as of March 2020, improved techniques and technologies have been applied on 32,883 Hectares of land. Indicator# - SI￾02 Number of individuals who have applied new techniques or technologies as a result of USDA assistance Type - Outcome Target – 11,400 individuals As per the PAD Semi-Annual Results till March 2020, 17,445 individuals (153% of the LOP target) have applied new techniques or technologies on their crops as a result of USDA assistance through the PAD project interventions. Among those interviewed during the MTE, 85% reported having applied production techniques (Activity VI) learned during the various trainings. These include the adoption of improved planting material (65%), land preparation (65%), nutrient management (37%), pest management (22%), and irrigation techniques (23%). Similarly, harvest and post-harvest training (Activity VII) was applied by 70% trainees and included better picking and harvesting techniques (50%), post-harvest handling of produce (50%), better packaging material (6%), and improved storage (7%). For Tier II, , the MTE found that these benefits have been derived by the large majority (86%) using the disseminated improved production techniques, including improved planting material (29%), land preparation (29%), nutrient management (12%), irrigation (9%), and pest management (7%). Furthermore, 70% reported adopting improved harvest and post-harvest handling techniques, including better picking methods (36%), post-harvest handling (30%), and others such as packaging material and storage, etc. (3%). For Activity X: Sanitary and Phytosanitary Standards (SPS) trainees, among those interviewed by the MTE, 93% have applied the training on crop production and 68% on post-harvest and handling processes, including better harvesting techniques (52%), improved handling of produce after harvest (47%), adoption of better packaging material (7%), and improved storage methods (4%). As a result of applying these practices, 65% have been able to market higher volumes, 42% experienced lower product wastage, and 16% reported getting access to new markets. 139 Result Increased availability of improved inputs Indicator# - CI￾02 Number of agricultural input suppliers/firms that sell or supply improved inputs as a result of USDA assistance Type - Outcome Target – 50 suppliers The project has engaged a private sector seed company to support chili and tomato crops by distributing input grants (Activity III) and providing training to agro-dealers (Activity IX). Thus far, the company has trained 64 agro-dealers (116% of target) through three one-day seminars. Of these, the project reported that 14 input suppliers participated in promotional campaigns initiated by PAD and provided inputs and technologies to farmers in the target area. The MTE findings suggest that these interventions have had limited impact on the sales of improved inputs by the trained firms due to issues with trainee selection, training methodology, and the fact that the trained businesses generally avoid conducting business with the sub-awardee company due to its often-difficult payment terms and problematic sales practices. Indicator# - CI￾18b Number of grants disbursed to producer / producer groups for inputs purchases Type – Output Target – 52535 As of March 2020, two grants have been disbursed to the following grantees: M/S Haji Sons and M/S Sprouts Biotech Laboratories (SBL), who have in turn delivered inputs to 2,118 farmers (2,055 tomato and chili farmers provided seedling trays and peat moss; and 63 farmers provided banana tissue cultured plants). MTE interviews with tomato farmers in Sindh revealed that although they were already aware of such nursery raising techniques, PAD facilitated these farmers to adopt this technology through free availability of trays and peat moss. While chili farmers said that they were new to raising seedlings in plug trays. However, in both cases, while the general response to trays and peat moss was positive, most farmers were critical of the seed quality (e.g. lack of germination, low germination rates, soft fruit, etc.) and lack of follow up support provided by the input supplier (PAD sub-awardee). Whereas interviews with beneficiaries of banana plants revealed that this activity has been highly effective, having resulted in almost 40% lesser time to mature (10 months vs. 16 months), double yields of 40,000 kg (40 MT) per acre as compared to 20,000 kg (20 MT), lower post-harvest losses, and almost 60% price increase (PKR 1.4 million vs. PKR 0.6 million per acre). These results have convinced existing beneficiaries to purchase additional plants and others to invest in the new variety. However, the lack of availability of tissue culture plants in the country due to limited production is a valid concern with regards to scale-up. Indicator# - CI￾19b Value of grants disbursed to producer / producer groups for inputs purchases Type – Output Target – 4,550,00036 Thus far, two grants worth USD 7,030 have been disbursed to producers/producer groups for inputs purchases. Indicator# - CI￾20b Number of individuals benefiting from and/or using inputs grants Type - Outcome Target – 12,300 individuals37 As of March 2020, 2,118 have received and/or are benefitting from inputs grants. Input grants included seedling trays and peat moss (2,055 tomato and chili farmers); and banana tissue cultured plants (63 farmers), using a supplier-based subsidy model. MTE interviews with tomato farmers in Sindh revealed that although they were already aware of such nursery raising techniques, PAD facilitated these farmers to adopt this technology through free availability of trays and peat moss. While chili farmers said that they were new to raising seedlings in plug trays. However, in both cases, while the general response to trays and peat moss was positive, most farmers were critical of the seed 35 The targets set of Activities 2 and 3 are the same in the agreement signed with USDA. However, this was probably a typo, since a budget has been allocated for grants under each activity (Activity 2: USD 4.2 Million; Activity 3: USD 761,951) and PAD is using that to decide how many grants to materialize. 36 Ibid 37 Ibid 140 quality (e.g. lack of germination, low germination rates, soft fruit, etc.) and lack of follow up support provided by the input supplier (PAD sub-awardee). Whereas interviews with beneficiaries of banana plants revealed that this activity has been highly effective, having resulted in almost 40% lesser time to mature (10 months vs. 16 months), double yields of 40,000 kg (40 MT) per acre as compared to 20,000 kg (20 MT), lower post-harvest losses, and almost 60% price increase (PKR 1.4 million vs. PKR 0.6 million per acre). These results have convinced existing beneficiaries to purchase additional plants and others to invest in the new variety. However, the lack of availability of tissue culture plants in the country due to limited production is a valid concern with regards to scale-up. Result Increased use of financial services Indicator# - SI￾04 Number of individuals receiving financial services as a result of USDA assistance Type – Output Target - 6,800 individuals Against an LOP target of 6,800 individuals obtaining agricultural credit as financial services, the project has been able to link 126 farmers to formal credit thereby achieving 2% of its LOP target. The major reasons for this low performance are the absence of the Loan Guarantee Facility (LGF) that was envisioned in the initial project design, the high cost of formal borrowing in the country, existing farmer-contractor relationships which are difficult to disrupt, and cultural/religious beliefs of farmers that reflect negatively on interest/usury associated with formal borrowing. A survey by the MTE team of randomly selected beneficiaries of credit showed that this was the first time for 92% of them to borrow from a formal source, and the majority (91%) had primarily obtained the loans for land preparation (91%). Moreover, the project continues to raise awareness among farmers regarding the benefits of formal borrowing by conducting sessions on formal lending in the pre and post-harvest trainings under PAD Activities VI and VII, and additional awareness raising sessions by the project’s Microfinance Officer and invited bank representatives. This however has had limited impact on the actual availability or utilization of loan, as demonstrated by the achievement of 2% against the LOP target. Indicator# - SI￾05 Number of loans disbursed as a result of USDA assistance Type – Output Target – 6,800 individuals A total number of 126 loans (2% of the LOP target) have been disbursed through the project till March 2020. A survey by the MTE team of randomly selected beneficiaries of credit showed that this was the first time for 92% of them to borrow from a formal source, and the majority (91%) had primarily obtained the loans for land preparation (91%). Indicator# - SI￾06 Value of loans provided as a result of USDA assistance Type – Output Target – USD. 3,400,000 The project has thus far disbursed 126 loans amounting to a total value of USD 39,842 (1.2% of the LOP target). A survey by the MTE team of randomly selected beneficiaries of credit showed that this was the first time for 92% of them to borrow from a formal source, and the majority (91%) had primarily obtained the loans for land preparation (91%). Result Increased knowledge by farmers of improved agricultural techniques & technologies Indicator# - SI￾16 Number of individuals who have received short￾term agricultural sector productivity or food security training as a result of USDA assistance Type – Output Target – 39,192 individuals 141 The project has thus far trained 22,262 individuals on short-term agricultural sector productivity or food security, thereby achieving 57% of the LOP target. These trained individuals include farmers and farm workers, agro-dealers and suppliers. The MTE team found that 65% of surveyed Tier I farmers have witnessed an increase in income as a result of: higher yields (60%), improved quality produce (60%), reduced disease or pest attacks (30%), lower cost of production (17%), and reduced wastage (15%). Similarly, harvest and post-harvest training was applied by 70% of trainees and resulted in benefits such as: higher price (42% farmers), larger quantity marketed (21% farmers), and access to new markets (11% farmers). However, of the four targeted crops, trained date farmers (Tier I) reported that they are not able to apply the techniques due to difficulty in organizing human, financial, and logistical resources due to the very short harvest season. While assessing the effectiveness of Tier II trainings, it was determined that 53% reported an increase in income and 26% were able to establish new market linkages because of applying techniques learned during the training. In view of these immediate results, the MTE team found the Tier II training to be effective. However, 43% of the interviewed Tier II men and women trainees have found it difficult to apply the training due to various reasons, including: lack of availability or high cost of recommended inputs, and difficulty to follow the training due to limited education. Indicator# - CI￾24 Number of trainers trained (ToT) Type – Output Target – 552 trainers As of March 2020, 3,860 Master Trainers (Tier-I) have been trained in Improved Agriculture Production Techniques and Post-Harvest Handling and Processing. The Master Trainers trained through the project are also responsible for further training their fellow farmers (Tier-II) in their communities. Thus far, 1,143 Master Trainers (30%) have further provided training to 17,703 farmers in their respective communities. The MTE team found that 65% of surveyed Tier I farmers have witnessed an increase in income as a result of: higher yields (60%), improved quality produce (60%), reduced disease or pest attacks (30%), lower cost of production (17%), and reduced wastage (15%). Similarly, harvest and post-harvest training was applied by 70% of trainees and resulted in benefits such as: higher price (42% farmers), larger quantity marketed (21% farmers), and access to new markets (11% farmers). However, of the four targeted crops, trained date farmers (Tier I) reported that they are not able to apply the techniques due to difficulty in organizing human, financial, and logistical resources due to the very short harvest season. While assessing the effectiveness of Tier II trainings, it was determined that 53% reported an increase in income and 26% were able to establish new market linkages because of applying techniques learned during the training. In view of these immediate results, the MTE team found the Tier II training to be effective. However, 43% of the interviewed Tier II men and women trainees have found it difficult to apply the training due to various reasons, including: lack of availability or high cost of recommended inputs, and difficulty to follow the training due to limited education. Indicator# - CI￾27 Number of training sessions offered through USDA assistance Type – Output Target – 137 sessions A total of 388 sessions for the above-mentioned trainings have been conducted till March 2020. These included sessions/batches for Master Trainers (ToT), Agro Dealers, and Female Laborers. The MTE team found that 65% of surveyed Tier I farmers have witnessed an increase in income as a result of: higher yields (60%), improved quality produce (60%), reduced disease or pest attacks (30%), lower cost of production (17%), and reduced wastage (15%). Similarly, harvest and post-harvest training was applied by 70% of trainees and resulted in benefits such as: higher price (42% farmers), larger quantity marketed (21% farmers), and access to new markets (11% farmers). However, of the four targeted crops, trained date farmers (Tier I) reported that they are not able to apply the techniques due to difficulty in organizing human, financial, and logistical resources due to the very short harvest season. Result Improved farm management 142 Indicator# - SI￾03 Number of individuals who have applied improved farm management practices (i.e., governance, admin, or financial management) as a result of USDA assistance Type - Outcome Target – 13,781 individuals Thus far, 17,445 individuals have applied improved farm management practices (i.e., governance, admin, or financial management) as a result of USDA assistance as per the latest Semi-Annual Performance Report October 1, 2019 – March 31, 2020. The MTE team found that 65% of surveyed Tier I farmers have witnessed an increase in income as a result of: higher yields (60%), improved quality produce (60%), reduced disease or pest attacks (30%), lower cost of production (17%), and reduced wastage (15%). Similarly, harvest and post-harvest training was applied by 70% of trainees and resulted in benefits such as: higher price (42% farmers), larger quantity marketed (21% farmers), and access to new markets (11% farmers). However, of the four targeted crops, trained date farmers (Tier I) reported that they are not able to apply the techniques due to difficulty in organizing human, financial, and logistical resources due to the very short harvest season. While assessing the effectiveness of Tier II trainings, it was determined that 53% reported an increase in income and 26% were able to establish new market linkages because of applying techniques learned during the training. In view of these immediate results, the MTE team found the Tier II training to be effective. However, 43% of the interviewed Tier II men and women trainees have found it difficult to apply the training due to various reasons, including: lack of availability or high cost of recommended inputs, and difficulty to follow the training due to limited education. Result Increase access to improved market information Indicator# - CI￾02 Number of agricultural producers in target region who have access to current agriculture market information through outlets supported by USDA activities Type - Outcome Target – 395,000 individuals Since its launch in September 2018, the information services supported by USDA/PAD activities have reached 900,233 agricultural producers not only in the project districts but also across Punjab, resulting in achievement of 228% of the project target. Thus far, activities have included a public awareness campaign through Telenor-Pakistan, a leading cellular company; text messages focusing on GAP disseminated to PAD trained farmers through an SMS portal; and a WhatsApp group of farmers who are interested in exchanging information and experiences. The MTE determined that 67% of surveyed Tier-I beneficiaries and 49% of Tier-2 beneficiaries are aware of the services. Of these, 71% from Tier I and 47% from Tier II have utilized the services, with all users reporting some degree of satisfaction. However, 29% in Tier I and 53% in Tier II are not utilizing the services, either due to the lack of access to a mobile phone (54%) or are subscribers of phone companies other than Telenor (33%). Based on customer usage tracking, Telenor also believes that this has been a useful service. In fact, the company found the message content provided by the PAD project to be of better quality than the existing messages under the company’s ongoing initiative of mAgri, thereby leading to adoption of similar approaches when developing messages for its other initiatives on agriculture-related information. Indicator# - CI￾03 Number of sources / outlets disbursing agriculture market information as a result of USDA assistance Type – Output Target – 111 outlets Thus far, two sources / outlets are part of the activity including a public awareness campaign through Telenor￾Pakistan, a cellular company; text messages disseminated by the project to PAD trained farmers through an SMS portal; and a WhatsApp group of farmers who are interested in exchanging information and experiences. Indicator# - CI￾23 Number of campaigns held, by topic, on market information Type – Output Target – 12 143 16 public information campaigns have been held thus far by the two sources / outlets. These focused on information on i) Pre-harvest and ii) Post-harvest GAP practices for the four target crops: banana, date, chili, and tomato. Result Improved Capacity of Key Groups in Agriculture Production Sector Indicator# - SI￾07 Number of private enterprises, producer organizations, water users associations, women’s groups, trade and business associations, and CBOs that applied improved techniques and technologies as a result of USDA assistance Type – Outcome Target – 170 CBOS Under Activity VIII, the project was to form 170 registered producer groups and cooperatives. However, the MTE team found that restrictive regulations by the GOP as part of the National Action Plan 2015 on registration of local organizations forced the project to form informal organizations instead. By March 30, 2020, the project has organized 133 groups. The MTE determined that these groups, although transitory in nature, have proven to be a useful platform for the project to reach out to beneficiaries. Result Improved quality of Post-Production agricultural products Indicator# - CI￾08 Volume of products utilizing new infrastructure Type – Outcome Target – 350,000 Metric Tons Against a target of 350,000 MT, the volume of products utilizing new infrastructure till March 2020 is 795.4 MT through provision of two cold stores38 of 300 MT capacity each and refurbishment of ten date drying tunnels as indicated in the Semi-Annual Perfomance Report October 1, 2019 – March 31, 2020. MTE interviews with two date cold store grantees revealed that this is a unique contribution of the project, since in the absence of project support the grantees would have continued using sub-standard rented space. Moreover, the storage facilities are expected to result in better quality product, thereby potentially generating 50% higher prices. In fact, it was reported that other traders in the area are also interested in establishing similar date storage facilities. In addition, as a result of refurbishment of ten date drying tunnels, 52,120 kgs of dates were sold by tunnel grantees, valuing at 3,938,100 PKR (USD 25,244). Among these, two grantees also allow about seven other neighboring farmers to utilize the tunnels, once free from their own use. 38 Of these, two cold stores are completed and handed over to the grantees, while three are completed and awaiting hand over to grantee. The stores are valued at USD 869,292 144 Result Increased adoption of established standards by industry Indicator# - CI￾09 Volume of chilies certified as meeting industry standards as a result of USDA assistance Type – Outcome Target – 4000 The MTE found that due to the absence of established industry standards in the country PAD has not been tracking this indicator. Indicator# - CI￾26 Number of individuals who have received training on sanitary & phytosanitary standards Type – Output Target – 4,110 individuals By March 2020, 2,143 individuals have received training on Sanitary and Phytosanitary standards. The MTE found that as a result of applying these practices, 65% have been able to market higher volumes, 42% experienced lower product wastage, and 16% reported getting access to new markets. Consequently, 73% of those who applied the training have reported an increase in income. Result Increased efficiency of Post-Production processes Indicator# - CI￾10 Percent of agricultural producers reporting decreases in post-harvest losses Type – Outcome Target – Missing The target for this indicator is missing in the Cooperative Agreement. Therefore, PAD does not report on this indicator. Result Improved Post-Harvest Infrastructure Indicator# - SI￾11 Total increase in installed storage capacity (dry or cold storage) as a result of USDA assistance Type – Output Target – 4,180 Cubic Meter The project till March 2020 has provided two cold stores39 of 300 MT capacity each. The total storage capacity of the two cold stores in cubic meters comes out to be 1,679 cu m. MTE interviews with selected grantees revealed that this is a unique contribution of the project, since in the absence of project support the grantees would have continued using sub-standard rented space. In addition to the savings on rent, the date’s storage facilities are expected to result in better quality product, thereby potentially generating 50% higher prices. In fact, it was reported that other traders in the area are also interested in establishing similar date storage facilities as the available storage space is insufficient, costly, and ineffective. Result Increased access to market to sell agricultural products Indicator# - CI￾14 Number of individuals who attend buyer/seller events Type – Output Target – 50 individuals Thus far, facilitated by the PAD project, 139 farmers of the four targeted value chains from Sindh and Punjab have participated in buyer/seller events. With the purpose of creating market linkages, participation in such events creates awareness of new technologies and market opportunities and helps in building relationships with input/equipment suppliers and prospective buyers of produce. MTE interviews farmers reported that the linkages established thus far have been effective and could not have been established without the project’s support. In particular, date farmers linked with markets reported being able to grow sales volumes, improve packaging practices, and increase revenue by up to 20%. Similarly, a banana farmer who was earlier exploited by middlemen through low prices and delayed payments is now linked to major markets in Punjab for advance payments and (USD 15.3/MT) higher rate. Moreover, in addition to the six medium/large farmers linked to markets, several smaller farmers in the area are also expected to benefit through sub-contracting. One such date farmer, having demand for 200 MT, has 39 Of these, two cold stores are completed and handed over to the grantees, while three are completed and awaiting hand over to grantee. The stores are valued at USD 869,292 145 already started exploring supply agreements with neighboring farmers. Similarly, a banana farmer linked to markets for sale of 300-400 MT is purchasing produce from another 40 farmers in the area. Result Improved linkages between buyers & sellers Indicator# - CI￾21 Number of buyer/seller events Type – Output Target – 29 events As of March 2020, 129 project beneficiaries have participated in a total of 6 buyer/seller events, including two expos (DAWN Agri. and Livestock Expo 2019 and Punjab Agri. and Food Expo 2019), one agriculture fair, and three meetings between corporate buyers and project beneficiary farmers (Shan Foods and Imtiaz Super Market). The aim of these interventions was to strengthen the bargaining power of smallholder farmers with powerful private-sector actors by selling aggregated produce volumes that bring better, more consistent prices by cutting out the middlemen. MTE interviews farmers reported that the linkages established thus far have been effective and could not have been established without the project’s support. In particular, date farmers linked with markets reported being able to grow sales volumes, improve packaging practices, and increase revenue by up to 20%. Similarly, a banana farmer who was earlier exploited by middlemen through low prices and delayed payments is now linked to major markets in Punjab for advance payments and (USD 15.3/MT) higher rate. Moreover, in addition to the six medium/large farmers linked to markets, several smaller farmers in the area are also expected to benefit through sub-contracting. One such date farmer, having demand for 200 MT, has already started exploring supply agreements with neighboring farmers. Similarly, a banana farmer linked to markets for sale of 300-400 MT is purchasing produce from another 40 farmers in the area. Indicator# - CI￾15 Number of agreements signed between buyers and sellers Type – Output Target – 20 agreements Facilitated by the PAD project, thus far, nine sales agreements (one banana and 8 date), have been signed between six producers (five dates and one banana) and bulk buyers. These have resulted in the sale of 725 MTs produce worth USD 205,116, including 567 MT bananas (valued at USD 62,016) and 158 MT dates (valued at USD 143,100). MTE interviews farmers reported that the linkages established thus far have been effective and could not have been established without the project’s support. In particular, date farmers linked with markets reported being able to grow sales volumes, improve packaging practices, and increase revenue by up to 20%. Similarly, a banana farmer who was earlier exploited by middlemen through low prices and delayed payments is now linked to major markets in Punjab for advance payments and (USD 15.3/MT) higher rate. Moreover, in addition to the six medium/large farmers linked to markets, several smaller farmers in the area are also expected to benefit through sub-contracting. One such date farmer, having demand for 200 MT, has already started exploring supply agreements with neighboring farmers. Similarly, a banana farmer linked to markets for sale of 300-400 MT is purchasing produce from another 40 farmers in the area. Result Improved market & trade infrastructure 146 Indicator# - CI￾18a Number of grants disbursed to producer / producer groups for improved equipment and infrastructure purchases Type – Output Target – 52540 The project aimed to provide in-kind grants for equipment and infrastructure worth USD 4.2 Million. Thus far, the project has disbursed payments to 36 grantees worth USD 535,580, thereby meeting 13% of its LOP goal. Delivered infrastructure grants include date cold stores, date drying tunnels (refurbished), drip irrigation, chili drying and storage solutions (GrainPro products), and a chili cold store and processing unit. Since most of the delivered infrastructure have been set up only recently, the MTE determined their effectiveness will become clear after at least one or two crop cycles. However, of the grants delivered thus far, date storage and drip irrigation have the highest potential for effectiveness based on productivity improvement and enterprise profitability. Also, PAD project approved a grant in March 2019 for the Pakistan Agricultural Research Council (PARC) for strengthening tissue culture lab at Thatta for farmers to access to disease free banana plants. However, the MTE found that despite reported follow-ups with concerned officials by PAD staff, the grant has not been countersigned by the PARC due to red-tape. Indicator# - CI￾19a Value of grants disbursed to producer / producer groups for improved equipment and infrastructure purchases Type – Output Target – 4,550,00041 The project aimed to provide in-kind grants for equipment and infrastructure worth USD 4.2 Million. Thus far, the project has disbursed grants worth USD 535,580, thereby meeting 13% of its LOP goal. Indicator# - CI￾20a Number of individuals benefitting from and/or using improved equipment, storage infrastructure grants Type – Output Target – 12,300 individuals42 The project has disbursed equipment and infrastructure grants to 36 direct beneficiaries (grant awardees). The MTE found that out of these 13 infrastructures project have been completed and are being utilized. While most grantees are using the equipment for their individual use only, two of those who were assisted with refurbishment of date drying tunnels reported allowing about seven other neighboring farmers to utilize the tunnels, once free from their own use; while one beneficiary of date cold store rented out his space to another 100 farmers. Chili aggregator setup through project assistance also shared the benefit of the equipment provided to him with three other farmers. Hence, in total around 123 individuals have benefitted from the infrastructures and equipment provided through PADs assistance. Result Increased leverage of Private-Sector resources Indicator# - SI￾08 Number of public private partnership formed as a result of USDA assistance Type – Output Target – 13 partnerships No public private partnerships were formed under the project. PAD through its in-kind cost share grants leverages private sector investment in infrastructure and inputs. Indicator# - SI￾09 Value of new public and private sector investment leveraged as a result of USDA assistance Type – Output Target – 1,500,000 No public private partnerships were formed under the project. PAD through its in-kind cost share grants leverages private sector investment in infrastructure and inputs. 40 The targets set of Activities 2 and 3 are the same in the agreement signed with USDA. However, this was probably a typo, since a budget has been allocated for grants under each activity (Activity 2: USD 4.2 Million; Activity 3: USD 761,951) and PAD is using that to decide how many grants to materialize. 41 Ibid. 42 Ibid. 147 Result Increased agricultural productivity Indicator# - SI￾18 Total number of individuals benefiting indirectly from USDA-funded intervention Type – Output Target – 531,480 individuals It is estimated that a total of 1,003,683 individuals (including 900,233 users of the IVR – Activity V) have benefitted indirectly from USDA-funded intervention till March 2020 against an LOP target of 531,480 thus achieving 189% of the project target. Indicator# - SI￾17 Number of individuals benefiting directly from USDA-funded intervention Type – Output Target – 58,580 Against an LOP target of 58,580, the project has thus far benefitted 24,209 individuals (41% of LOP target). Result Improved Trade of Agricultural Products Indicator# - SI￾15 Number of jobs attributed to USDA assistance Type – Output Target – 470 Thus far, 3 jobs can be attributed to USDA assistance against a target of 470. Adil Traders hired 3 new permanent staff for the cold store provided through assistance of PAD. Indicator# - SI￾13 Value of sales by project beneficiaries Type – Outcome Target – 24,452,380 As reported in the latest Semi-Annual Performance Report, the value of project sales by project beneficiaries stands at USD 123 million. Of this, USD 75.7 million sales were made in the last two quarters, i.e. October 2019 to March 2020. PAD team captures these estimates carrying out follow-ups with its beneficiaries every six months. As the follow-up is sample-based, the estimates are extrapolated on the total beneficiary pool and reported. Commodity wise sales till March 2020 are as follows: Banana: USD 34.9 million Dates: USD 23.7 million Red Chili: USD 28.6 million Tomato: USD 35.7 million As elaborated above in the sections on training, equipment, and input grants, the MTE found that beneficiaries have been able to reduce post-harvest losses and improve incomes as a result of project interventions. Indicator# - SI￾14 Volume of commodities (MT) sold by project beneficiaries Type – Outcome Target – 76,850 Metric Tons By March 2020, project beneficiaries have reportedly sold 537,288 MT of produce. Estimates are captured similarly as mentioned for SI-13 by PAD. Commodity wise sales are as follows: Banana: 302,361 MT Dates: 49,215 MT Red Chili: 18, 848 MT Tomato: 166,864 MT As elaborated above in the sections on training, equipment, and input grants, the MTE found that beneficiaries have been able to reduce post-harvest losses and improve incomes as a result of project interventions. www.cynosure-intl.com info@cynosure-intl.com +92 51 831 9116 Pakistan Office Cynosure Consultants (Pvt.) Ltd. House 497, Street 19, Shahzad Town Islamabad, Pakistan. 44000 Canada Office Cynosure International, Inc. 21 Ave. Laurier East, Apt.2 Montreal, QC, Canada, H2T 1E4