PARTNERSHIPS FOR INNOVATION AND KNOWLEDGE IN AGRICULTURE FINAL EVALUATION MAIN REPORT MARCH 8, 2011 This publication was produced for review by the United States Agency for International Development. It was prepared by Douglas Krieger (Management Systems International), Donald Taylor, Ravi Singh, Jitendra Kumar Sinha (Social Impact), and Umesh Rao Adapa (Sambodhi). PARTNERSHIPS FOR INNOVATION AND KNOWLEDGE IN AGRICULTURE FINAL EVALUATION Contracted under RAN-I-00-09-00019-00, Task Order No. AID-386-TO-10-00003 Evaluation Services Indefinite Quantity Contract DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation i TABLE OF CONTENTS Executive Summary ..................................................................................................................... vii 1 Introduction ....................................................................................................................... 1 2 The Development Problem and USAID’s Response ...................................................... 3 2.1 USAID’s Response: PIKA........................................................................ 3 3 Purpose of the Evaluation ................................................................................................ 5 4 Evaluation Methodology .................................................................................................. 5 4.1 Data Limitations ....................................................................................... 5 5 Findings and Conclusions ................................................................................................ 6 5.1 Impact ....................................................................................................... 6 5.1.1 Findings ........................................................................................ 6 5.1.2 Conclusions ........................................................................................ 8 5.2 Sustainability ............................................................................................ 9 5.2.1 Findings ........................................................................................ 9 5.2.2 Conclusions ...................................................................................... 10 5.3 Relevance ................................................................................................ 11 5.3.1 Findings ...................................................................................... 11 5.3.2 Conclusions ...................................................................................... 12 5.4 Effectiveness ........................................................................................... 13 5.4.1 Findings ...................................................................................... 13 5.4.2 Conclusions ...................................................................................... 15 5.5 Gender..................................................................................................... 16 5.5.1 Findings ...................................................................................... 16 5.5.2 Conclusions ...................................................................................... 17 6 Recommendations ........................................................................................................... 18 6.1 Recommendations for Ongoing PIKA Activities ................................... 18 6.2 Recommendations for Future Programming ........................................... 18 7 Lessons Learned .............................................................................................................. 20 Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation ii TABLE OF TABLES AND FIGURES Table 1. PIKA Project Summary .................................................................................................... iv Table 2. PIKA Projects .......................................................................................................... vii Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation iii Acknowledgments The evaluation team owes a great debt to the many individuals and organizations who gave generously of their time to provide input for the evaluation. First in this substantial list are the many individual farmers who uncomplainingly, and with unfailing hospitality, took time out of their busy schedules to host the evaluation teams in their villages and homes. Management and field personnel from implementing and local partners (University of Wisconsin-Madison, Mahindra & Mahindra, Tasty Bite [Preferred Brands International], Rajiv Gandhi Charitable Trust, Michigan State University, National Horticulture Board, Quality Council of India, Confederation of Indian Industry, Tamil Nadu Agricultural University, FoodCert India, YES Bank, Jain Irrigation, International Food Policy Research Institute, Hariyali Kisan Bazar, ITC Ltd. Agribusiness Division, ACDI/VOCA, Rice and Wheat Consortium, and Action for Food Production) who orchestrated field work and patiently explained their projects and accomplishments also deserve recognition. The frank and open discussions with these individuals contributed invaluably to the evaluation. The evaluation team also greatly appreciates the support of USAID/India personnel whose experience, knowledge, and insights helped guide the evaluation team. Although many people deserve mention in this regard, the team would especially like to acknowledge the contributions (in alphabetical order) of A.S. Dasgupta, Program Manager, General Development Office; Elizabeth Callender, Program Officer; Gary Robbins, Food Security Office; John Thurow, General Development Officer (Agriculture), Food Security Office; M. Erin Soto, Mission Director; and Neelakshi Mann, Development Program Specialist (M&E). Finally the evaluation team must acknowledge the support of the many people at Social Impact, Management Systems International, and Sambodhi who backstopped the team’s activities at every stage. You are too many to name individually but you know who you are. Douglas Krieger Don Taylor Jitendra Kumar Sinha Ravi Singh Umesh Rao Adapa Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation iv PROJECT SUMMARY PIKA consists of four sub-projects implemented under three cooperative agreements and one contract. All component projects are implemented as public private partnerships. Table 1 summarizes PIKA project components. TABLE 1. PIKA PROJECT SUMMARY Project name Partnerships for Innovation and Knowledge in Agriculture (PIKA) Objective, program area, and program elements Objective: Economic Growth Program area/element: Agriculture ($3,795,000)/Agricultural Sector Capacity Program area/element: Economic Opportunity ($700,000)/Strengthen Microenterprise Productivity Life of project Two years from award for each sub-project Project funding $4,600,000 Project name Increasing Productivity and Market Links for High Value Agricultural Products Life of project October 1, 2008–September 30, 2010 (no cost extension to March 30, 2011) Implementing partner(s) University of Wisconsin- Madison Key partners Mahindra & Mahindra (M&M), Rajiv Gandhi Charitable Trust (RGCT), Tasty Bite (Preferred Brands International), the Agricultural Consultancy Management Foundation (ACMF) Contracting mechanism Cooperative Agreement No. 386-A-00-08-00097-00 Project funding USAID: $950,000; Cost-share: $70,600; Leveraging: $3,006,000 Project name Indian Horticultural Development Alliance (IHDA) Life of project October 1, 2008–September 30, 2010 (no-cost extension to September 30, 2011) Implementing partner(s) Michigan State University Key partners National Horticulture Mission (NHM), National Horticulture Board (NHB), Quality Council of India (QCI), Confederation of Indian Industry (CII), GlobalGAP, Tamil Nadu Agricultural University (TNAU), FoodCert India, YES Bank, Jain Irrigation Contracting mechanism Cooperative Agreement No. 386-A-00-08-00096-00 Project funding USAID: $1,395,000; Cost-share: $218,459; Leveraging: $9,194,052 Project name Rural Services Hubs: Business Catalysts for Rural Competitiveness with Inclusiveness Life of project October 1, 2008–September 30, 2010 (no cost extension to June 30, 2011) Implementing partner(s) International Food Policy Research Institute (IFPRI) Key partners Hariyali Kisan Bazar (HKB), ITC Ltd., Agribusiness Division (ITC), Viswas Business Synergies Ltd., Jawaharlal Nehru Krishi Vishwa Vidhyalaya, Indira Gandhi Institute of Development Research (IGIDR), Govind Ballabh Pant Social Science Institute, Centre of Economic and Social Studies. Contracting mechanism Grant No. 386-G-00-08-00099-00 Project funding USAID: $700,000; Cost-share: $10,318; Leveraging: $4,090,270 Project name Improving efficiencies in commodity value chains through advanced cropping technologies, Uttar Pradesh Life of project October 1, 2008–September 30, 2010 (no cost extension to July 30, 2011) Implementing partner(s) World Vision Key partners ACDI/VOCA, Rice and Wheat Consortium (RWC), Action for Food Production (AFPRO). Contracting mechanism Cooperative Agreement No. 386-A-00-08-00098-00; Project funding USAID: $1,555,000; Cost-share: $325,703; Leveraging: $4,703,139 Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation v Glossary Farm classification The Agricultural Census of India1 defines five categories of farms, depending on the size of operational holdings.2 Definitions are in metric units (hectares), although English units (acres) appear to be more commonly used. The following table summarizes the definitions used in this report. For the purposes of this report, small farmers include both marginal and small farmers. Classification Size in hectares Approximate size in acres Marginal farm 0.001–1.00 hectares 0.1–2.4 acres Small farm 1.01–2.00 hectares 2.5–5.0 acres Small-medium farm 2.01–4.00 hectares 5.1–10.0 acres Medium farm 4.01–10 hectares 10.1–25.0 acres Large farm More than 10.00 hectares More than 25 acres 1 “Analysis of Agriculture Census Results – All India All Social Groups,” Agricultural Census of India, 2000-01, . 2 An operational holding refers to “a techno-economic unit wholly or partly for agricultural production (defined below) and operated (directed/managed) by one person alone or with the assistance of others, without regard to title, size or location. The holding might consist of one or more parcels of land, provided these are located within the country and form part of the same technical unit. In the context of agricultural operations, a technical unit is a unit with more or less independent technical resources covering items like land, agricultural equipment and machinery, draught animals, etc. Holdings used exclusively for livestock and poultry raising and for production of livestock and poultry products (primary) and/or pisciculture are considered as operational holdings whereas holdings put exclusively to uses other than agricultural production are not considered as operational holdings. Holdings operated by cooperative farms are also not considered as operational holdings.” Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation vi ACRONYMS ADP Area Development Plan AFPRO Action for Food Production AKI Agricultural Knowledge Initiative CII Confederation of Indian Industry CRPs Community Resource Persons FAO Food and Agriculture Organization of the United Nations GAP Good Agricultural Practices GDP Gross Domestic Product GoI Government of India HKB Hariyali Kisan Bazar IFPRI International Food Policy Research Institute IGIDR Indira Gandhi Institute of Development Research IHDA Indian Horticultural Development Alliance IRRI International Rice Research Institute ITC Indian Tobacco Company LOP Life of Project M&M Mahindra & Mahindra MANAGE National Institute of Agricultural Extension Management MSU Michigan State University NHB National Horticulture Board PIKA Partnerships for Innovation and Knowledge in Agriculture PPP Public-Private Partnerships QCI Quality Council of India RBH Rural Business Hub RGCT Rajiv Gandhi Charitable Trust Rs. Indian Rupees RWC Rice and Wheat Consortium SHG Self Help Groups SOW Scope of Work TNAU Tamil Nadu Agricultural University UP Uttar Pradesh US United States USG United States Government USAID United States Agency for International Development UW University of Wisconsin-Madison WSHG Women’s Self Help Groups Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation vii EXECUTIVE SUMMARY USAID/India’s Partnership for Innovation and Knowledge in Agriculture (PIKA) program aims to increase small-farmer incomes by leveraging public-private partnerships (PPP)3 TABLE 2. PIKA PROJECTS to develop and disseminate technologies and practices that will increase productivity and foster more effective market linkages for Indian agricultural producers and processors. The program includes four separate projects, summarized in the following table. US Implementing partner Primary activities University of Wisconsin￾Madison (UW) • Helped Mahindra & Mahindra (M&M) establish 135 soil testing laboratories through which it offers productivity-enhancing technical assistance to small farmers. • Helped the Rajiv Gandhi Charitable Trust (RGCT) develop and implement training materials to teach women how to improve dairy and vegetable productivity. • Helped Tasty Bite identify improved production practices for dissemination to Tasty Bite contract farmers. Michigan State University (MSU) Identified and addressed constraints across the entire horticulture value chain. Built the capacities of horticulture producer groups to meet market demands and linked them to markets. International Food Policy Research Institute (IFPRI) Conducted detailed market research to help guide ITC Ltd. (ITC), Hariyali Kisan Bazar (HKB), and Viswas Business Synergies Ltd. to improving the quality of services they offer small farmers. World Vision (WV) World Vision coordinated with partners (ACDI/VOCA, RWC, AFPRO) to introduce farmers to productivity-enhancing and water-conserving technologies and practices and commercial market linkages. The projects were initially funded for two years beginning in October, 2008 although all received no cost extensions. In January of 2011, just months before the program’s end, USAID commissioned a final evaluation of PIKA. The evaluation used individual and group interviews with partners, stakeholders, and beneficiaries; document review, and analysis of secondary data to assess (1) the impact, relevance, and effectiveness of the PIKA program and the four projects implemented under PIKA, (2) the sustainability of results and (3) the extent to which the projects have addressed gender issues. Mission personnel articulated two primary purposes for conducting the evaluation: to assess the performance of the four PIKA projects and the overall PIKA program. and to generate lessons learned to guide USAID’s ongoing agricultural programming, both in India and elsewhere. The evaluation team visited each of the PIKA projects and collected data through personal interviews with 112 key informants and structured group interviews with over 500 farmer beneficiaries. The evaluation team encountered several obstacles that limited the depth and rigor of the evaluation. Perhaps most importantly, the travel time required to visit seven individual activities distributed across the country severely limited the amount of time the evaluation team could spend with each activity. With one exception (IFPRI), the projects had not designed or implemented adequate impact assessment strategies. In the case of IFPRI, the project itself did not design or implement an impact assessment strategy. Instead, the extensive research the Indian Tobacco Company (ITC) and HKB conducted while forming their 3 The projects were not PPPs in the usual sense of the term because the public sector (i.e., government) played little or no role in most projects. In the context of PIKA, “public” refers to USAID. Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation viii business strategy established a baseline that may be suitable for impact assessment. 4 This executive summary summarizes key conclusions of each of the five evaluation questions—impact, sustainability, relevance, effectiveness, and gender. It then presents synopses of the main recommendations and lessons learned. Michigan State University (MSU) also estimated farmers’ income increments associated with improved market linkages and M&M estimated productivity impacts for a subset of farmers. Estimates of the impact of the projects on farmer productivity and incomes for the other projects are largely derived from thin, anecdotal evidence. Conclusions Regarding Impact The four PIKA projects disseminated information about productivity-enhancing production and marketing practices to a conservative estimate of over 240,000 farmers. Farmers who have adopted improved practices, in all likelihood, are disseminating that knowledge to a second layer of farmers, thus substantially swelling the number of beneficiaries. Field research by two partners (ITC and HKB) concluded that farmers who adopted their recommended package of practices increased yields by fifteen to twenty percent. Mahindra & Mahindra reported that ninety-six percent of 3,800 farmers following its recommended practices had increased productivity by an average of fourteen percent—although it is not clear from the data whether productivity increased due to project interventions or to exogenous factors such as prices. Michigan State University estimated that its work linking fourteen producer clusters to markets had increased the incomes of 3,625 farmers by an aggregate $3 million during the 2009/10 agricultural season. No other reliable estimates of income impacts were possible with the available data. Even though, in most cases, it was not feasible to quantify yield and income impacts rigorously, the consistent, anecdotal evidence provided by the vast majority of farmers the evaluation team interviewed strongly supports the conclusion that the practices did, in most cases, increase yields, productivity, and incomes. Conclusions Regarding Sustainability In general, it is premature to determine the sustainability of any of the PIKA projects. The long-term sustainability of the private sector-led partnerships (i.e., M&M, Tasty Bite, IHDA, ITC, HKB) will 4 Proprietors of the ITC farmer service centers keep track of farmers’ progress and report regularly to ITC headquarters. In addition to the exhaustive survey by HKB of farmers in the catchment area before an HKB rural business hub (RBH) is opened, agronomists stationed at each of the RBHs gather and report information from each farm village on a regular basis. Evaluation Questions: Based on results to date, how likely is it that PIKA results will be sustained after USAID funding ends? To what extent did PIKA activities catalyze private sector capacity building and adoption of innovative technologies and business practices, influence policy, and contribute to scaling up of project-supported activities? Evaluation Questions: To what extent did PIKA promote adoption of best practices among targeted beneficiaries and others? To what extent have PIKA activities increased productivity and incomes of targeted beneficiaries? Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation ix depend on the projects’ achievement of financial viability—the innovative policies and business practices must ultimately generate sufficient revenue to cover costs. Results based on knowledge and skills embedded in individuals and communities (e.g., knowledge of improved production practices, animal husbandry, importance of soil health) is generally sustainable as long as other elements of the value chain (market access, input availability, etc.) exist to provide the incentives for investing in productivity. Most of the private sector activities (with the exception of Tasty Bite) show substantial potential for scaling up. In each case, partners have targeted large and relatively untapped markets, and the evolving markets for agricultural inputs and products will drive expansion of economically viable projects. Tasty Bite is a comparatively small operation with limited capacity or motivation to disseminate knowledge beyond the relatively small number of farmers that could directly supply its processing facility. All of the PIKA projects, with the exception of World Vision, contributed either to existing programs or to ongoing initiatives of partners by effectively filling technology, knowledge, or expertise gaps (see Annex A for a complete description of the contributions of the PIKA projects to their partners’ initiatives). Conclusions Regarding Relevance Government extension services and university researchers in India are failing to provide relevant, tailored advice to small farmers. Each of the PIKA projects, in its own way, contributed to filling this gap by facilitating the transfer of appropriate, productivity-enhancing technologies and practices to small farmers. One of the most significant challenges to further improving the economic status of small farmers in India is finding the means to fill this significant gap in research and extension services. The fact that PIKA-promoted technologies and practices almost uniformly increased yields and productivity implies that the interventions were relevant to achieving project objectives. There is insufficient information, however, to estimate the magnitude of changes in productivity or income or to determine whether the promoted practices are the “best” practices in the current environment or the most applicable for the targeted beneficiaries. 5 PIKA partners have generally adapted their activities to the changing environment or to the realities of their work environments. ITC and HKB were particularly good at adapting their activities to maintain relevance because they were designed specifically for that purpose. The companies are using the results of the market research provided by IFPRI to target their products and services to small farmers. Conclusions Regarding Effectiveness While some of the projects have failed to meet all of their specific output targets (see Annex D), all of them—with the exception of significant elements of the World Vision component and the Tasty Bite 5 For example—and there is insufficient empirical evidence to do more than flag the question for additional research—if M&M is promoting mechanized solutions that substantially increase production costs, these may not be the most applicable to small farmers. Evaluation Questions: Are program activities relevant to the needs of targeted beneficiaries and applicable in the current environment? Are the original hypotheses on which the program was designed still valid and relevant to the needs of the region? Have PIKA partners maintained relevance by adapting to emerging opportunities to achieve program objectives? Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation x project—appear to be generally achieving their expected outcomes (i.e., disseminating relevant productivity enhancing technologies and practices to small farmers). The projects were largely successful in their efforts to target small farmers. University of Wisconsin’s project with the Rajiv Gandhi Charitable Trust (RGCT) exclusively targets small farmers. The University of Wisconsin’s project with Mahindra & Mahindra (M&M) has initially engaged a disproportionate number of medium and large farmers, but its hub and spoke dissemination model shows potential for increased smallholder engagement as it matures. The ITC, HKB, and MSU projects all engage smallholder farmers in numbers roughly proportional to their prevalence of the population. The close correspondence between the business objectives of private sector partners and USAID's development objectives and the ability of NGOs to build on and expand their previous successful initiatives contributed to the overall effectiveness of these PPP approaches. Successful PPP development projects in India and elsewhere share two common characteristics: they are based on commercially viable activities with definite links to profitable markets, and they follow a value￾chain approach. Conclusions Regarding Gender There have been few changes in women’s positions as a result of PIKA. Farm women continue to provide much of the labor on the farm but have little or no decision-making power and earn lower wages than men for similar work. To a large extent, projects’ capacity-building activities (i.e., training) engaged mostly men. Male farmers are usually the decision makers; women implement the decisions.6 While all of the PIKA projects included at least some treatment of gender issues in their proposals, none of them, at least initially, effectively addressed the issues. Some (RGCT and IHDA) eventually recognized the value of addressing the concerns of both men and women in a coordinated fashion and began to integrate more gender-appropriate approaches. However, IHDA has had difficulty identifying appropriate partners to address gender concerns. Other projects (ITC, HKB, WV) have not yet effectively addressed gender issues. Recommendations Enhance the prospects for sustainability and impact of existing PIKA projects. PIKA partners can enhance the prospects for sustainability of PIKA results by identifying potential improvements in partnership activities and positioning partners to implement changes. For example: • ITC can extend procurement activities to commonly produced crops other than rice and wheat (e.g., gram) and offer dairy support services, and devise commercially viable measures to aggregate and transport small farmers’ crops to the ITC procurement hubs. • World Vision could enhance the prospects for sustainability of their on-going Area Development Programs (ADPs) by building the capacities of the self-help groups (SHGs) and utilizing lessons learned to improve their small-farmer development approaches. • UW might try to help M&M creates links to local resources to help establish additional soil￾testing laboratories and train technicians to operate the laboratories. 6 In the UW project with RGCT, women are experiencing social empowerment but they still lack economic empowerment. The men, in general, control the income generated by women’s economic activities. Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation xi For future agricultural programming in India, USAID should consider the following: Ineffective research and extension remain critical constraints to agricultural development in India. Interested parties, including donors, should support alternative (to government) arrangements for providing extension services to small farmers, perhaps including a means of developing and financing private extension services. In the case of agricultural research, focused programs to link users of research with the institutional researchers need to be effectively implemented. Plan for impact assessment. None of the PIKA projects planned or implemented a satisfactory impact assessment strategy. Consequently, USAID has little more than anecdotal evidence on which to assess the impact of its work. If USAID is serious about impact assessment, it needs to integrate the assessments into projects (the PIKA Action Memorandum did not mention either impact assessment or monitoring and evaluation), rather than letting implementing partners address it only as an afterthought. It should require bidders to propose an impact assessment strategy, carefully evaluate the adequacy of the proposed strategy, determine whether the cost is warranted, and ask bidders to rework the strategy if the cost is too high or the approach not rigorous enough. Improve gender performance. The PIKA solicitation was clear about gender objectives. However, none of the projects dealt with gender issues effectively. Specific recommendations that could have helped projects improve their approach to gender issues include: (1) clarification on the part of USAID on what it means by gender issues—as opposed to women’s empowerment, (2) incorporate gender training for staff into all projects, (3) provide capacity building opportunities relevant to the roles to both men and women in projects, and (4) implement training for women near project areas so they can return home in the evenings. Lessons Learned Characteristics of successful PPPs. At least in the environment in India, the PPP approach has been an effective and efficient7 mechanism to leverage USAID resources to extend benefits to a large number of farmers. Lessons from the seven PPP approaches implemented under PIKA8 • The PIKA experience with private sector partners suggests that aligning the business development interests of commercial firms with the objectives of donor-sponsored, small-farmer development programs is an effective means of marshaling the divergent areas of expertise required to effectively achieve development objectives. include: • The successful PIKA projects all contributed specific technical expertise to well formed, existing strategies that were closely aligned with development objectives (see Annex A for details of each project). When entering future PPP arrangements, look for opportunities where USAID can leverage a comparative advantage (e.g., access to technical expertise and knowledge) to augment/compliment initiatives of motivated and qualified partners backed by documented strategies for pursuing activities consistent with development objectives. • USAID can enhance prospects for sustainable results from private sector-led partnerships by carefully assessing the quality of the partner’s business plan and the prospects for achieving the long-term financial viability necessary to sustain the activities. NGO partners who work through 7 To make one comparison, a donor-driven FAO project in western Pakistan that follows a value chain community development approach has engaged 50,000 farmers over a six-year period at a cost substantially above that of the PIKA program. To make a fair comparison, the FAO project has to contend with security concerns that are not factors in India. 8 UW (M&M), UW (Tasty Bite), UW (RGCT), IFPRI (HKB), IFPRI (ITC), MSU, and World Vision. Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation xii community organizations need to focus first on building the capacities of the organizations as a sustainable vehicle for disseminating information, and then on the information itself. • One of the keys to the success of the alliance approach is the selection process. Potential partners should be carefully vetted to determine their capability and willingness to carry out their individual responsibilities and to merge their expertise with that of the other alliance partners. • While it may seem to promote exclusivity, starting projects with a single-firm approach may be necessary in order to demonstrate to industry that the suggested innovations are commercially viable and sustainable. The single-firm approach is also more cost effective and easier to launch. The commercial successes of the rural business hubs (RBHs) are prompting other firms to adopt similar models and at last one other RBH in the ITC and HKB project area is beginning to target the small-farmer market. • The potential to scale up successful activities depends on the vision, network, and resources a partner has to expand a financially viable model. Soil tests were effective in reducing fertilizer application rates, increasing yields, and reducing production costs. Almost all of the farmers the evaluation team interviewed who had obtained soil tests through M&M and HKB had been over-applying basic fertilizers, particularly urea and to some extent DAP, and under-applying trace elements. Farm-specific information provided by the soil tests, coupled with outreach to help farmers interpret test results, has allowed farmers to calibrate fertilizer use to soil needs and, in most cases, lower production costs and increased yields. Mobile connectivity. The institution of mobile technology for providing current market and weather information to small farmers, introduced by ACDI/VOCA under the World Vision project, proved ineffective in the first instance, due to use of the wrong system. By linking with a tested and successful commercial service in the second instance, ACDI/VOCA was able to successfully initiate the application of mobile connectivity for these purposes. The sustainability of this innovation is questionable, since the Rs. 90 monthly cost has been subsidized by the project for the first three months and there is some doubt that small farmers will be willing to pay for the services in future. Gender objectives are often misunderstood and inadequately addressed. People often confuse gender development with women’s development or women’s empowerment—all of which may be valid development objectives in different contexts. USAID solicitations need to be clear about the concept appropriate within a given context and ensure that bidders understand the perspective and submit proposals consistent with objectives. Women’s roles in agriculture. Interviews with farmers confirmed that women perform most of the tasks associated with producing crops. Men generally prepare the land and harvest and market the crops. Women do most of the planting/transplanting, weeding, and post-harvest handling. When farmers hire labor for these tasks, women often manage the labor. However, with the exception of RGCT, the projects are training men, not women, in improved agricultural practices. Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 1 1 INTRODUCTION With a population of over 1.1 billion, India is the world’s second most populous country.9 The country experienced chronic food shortages and high poverty rates through the late 1960s,10 The declining role of agriculture in India’s economy (sixteen percent of GDP in 2009—down from forty￾five percent in 1967 and thirty-six percent as recently as 1980 However, the introduction of new seed varieties for basic food grains in the 1970s dramatically increased agricultural productivity, led to self-sufficiency in food grains, and commenced a steady decline in poverty rates that continues to the present. Recent, rapid growth in the IT, specialty manufacturing, and service sectors, coupled with a generally declining growth trend in the agricultural sector, however, represents a transformation of India’s economy that has exacerbated many development problems—especially in rural areas—and may dictate revised objectives and approaches to agricultural development. 11) belies its importance to economic growth, poverty reduction, and rural welfare. Some seventy percent of India’s population lives in rural areas12 and an estimated fifty-two percent of the workforce is employed in agriculture13 which means that as many as 600 million people depend on agriculture for their livelihoods. Poverty rates in rural India approach thirty percent, implying a population of as many as 245 million rural poor. Furthermore, rural poverty may be concentrated among households engaged in agriculture. India’s eleventh Five Year Plan (2007-2012)14 classified forty-seven percent of agricultural labor households among the rural poor. The states of Bihar, Madhya Pradesh, Orissa, and Uttar Pradesh—particularly Madhya Pradesh and Uttar Pradesh, with their dependence on agriculture—have poverty rates well above the national average. 15 The World Development Report of 2008 placed renewed emphasis on the potential of agriculture to reduce poverty and improve rural livelihoods.16 The Government of India (GoI) has established aggressive targets for growth in agricultural productivity to combat rural poverty and ensure food security17 9 “Fact Book – India,” CIA, 2010, . . The two Indias—states with diversified economies and less reliance on agriculture versus relatively poor states that still rely largely on agriculture—may require different approaches to agricultural development. The World Development Report suggests that “in transforming economies, characterized by rapidly rising rural-urban income disparities and continuing extreme rural poverty, addressing income disparities requires a comprehensive approach that pursues multiple pathways out of poverty—shifting to high value agriculture, decentralizing nonfarm economic activity to rural areas, and providing assistance to help move people out of agriculture.” In agriculture-based economies (which describes some of the poorer Indian states), the report states that “agriculture and its associated industries are essential to growth and to reducing mass poverty and food insecurity. Using agriculture as the basis for economic growth in the agriculture-based countries requires a productivity revolution in smallholder farming.” 10 “World Development Report,” World Bank, 2008, . 11 “World Development Indicators,” World Bank, . 12 “Monitoring Agri-trade Policy (MAP): India’s Role in World Agriculture,” Director General for Agriculture and Rural Development, European Commission, No. 03-07, December, 2007, . 13 “Fact Book – India,” CIA, 2010, . 14 “Eleventh Five Year Plan (2007-2012),” Planning Commission, Government of India, . 15 Duraisamy, P. and Ajay Mahal. “Health, Poverty, and Economic Growth in India,” . 16 “World Development Report, 2008,” World Bank, . 17 “Eleventh Five Year Plan (2007-2012),” Planning Commission, Government of India, . Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 2 Pursuing either strategy will require new knowledge, technologies, and practices to increase productivity and access new markets. The World Bank and the Food and Agriculture Organization (FAO) suggest that agricultural research and development is crucial to enhancing agricultural productivity because there is limited potential for increasing the area under cultivation.18 The World Bank states that “Indian agriculture will have to shift from resource- and input-based growth to knowledge and science based growth triggered by innovations and the application of science to agriculture.” According to the FAO, future increases in crop production must accrue essentially through increased production per unit land area. 19 Most small farmers in India practice mixed crop–livestock farming in order to diversify income sources and manage production risk. While practices vary by state, nationwide, between sixty-seven and seventy￾six percent of India’s marginal and small farmers own livestock. 20 Livestock provide many products that contribute to livelihoods, but milk represents the most economically important output. A study conducted in Gujarat in 1994 found that seventy-five percent of rural households kept milk animals and that milk accounted for an average of twenty percent of household income.21 While these estimates are dated, the GoI’s Land and Livestock Holdings (LHS) survey of 200322 found that ownership of milking livestock among marginal and small farmers has not changed much since 1994. 23 India faces myriad challenges in pursuing agricultural development through increased innovation. These include a policy environment that favors subsidies, which crowds out private investment in productivity￾enhancing technology and discourages farmers from diversifying into alternative, high-value products; restrictions on domestic trade that limit access to markets and inflate transactions costs; inadequate transportation and post-harvest infrastructure; insufficient funding for the agricultural research necessary to foster innovation; and underfunded extension that fails to effectively disseminate productivity￾enhancing practices to the farmers. The report concluded that marginal and small farmers owned an estimated seventy-one percent of the in-milk bovine stock in India in 2003. 24,25,26,27 The remainder of this report first establishes the background for the evaluation. It summarizes the rationale for the PIKA interventions, presents evaluation objectives, and describes evaluation procedures. PIKA aims to address some of these issues by supporting the transfer of knowledge about productivity-enhancing technologies and practices to Indian farmers and businesses. 18 The World Bank, India. “Agricultural R&D is Critical to Enhance Productivity,” . 19 Food and Agriculture Organization of the United Nations. “Agricultural Growth and Total Factor Productivity,” . 20 Food and Agriculture Organization of the United Nations. “Smallholder Farmers in India: Food Security and Agricultural Policy,” March 2003, . 21 ILRI. “Dairying in integrated farming systems,” . 22 National Sample Survey Organisation. “Livestock Ownership Across Operational Land Holding Classes in India, 2002-03,” Ministry of Statistics & Programme Implementation, Government of India,, January 2006, NSS Report No. 493(59/18.1/1), < http://planningcommission.nic.in/sectors/agri_html/Livestock%20ownership%2059round%202003.pdf>. 23 The data do not provide a direct estimate of the percentage of small and marginal farmers who own milk cattle. 24 The World Bank, India. “Agricultural R&D is Critical to Enhance Productivity,” . 25 The World Bank, India. “Priorities for Agriculture and Rural Development,” . 26 The World Bank, India. “Re-energizing the Agricultural Sector To Sustain Growth and Reduce Poverty,” Report No. 27889- IN, July 30, 2004. 27 Food and Agriculture Organization of the United Nations. “Smallholder Farmers in India: Food Security and Agricultural Policy,” March, 2003, . Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 3 Following the background material, the report then presents the findings of the evaluation (the facts that the evaluation team found), the conclusions supported by the findings, recommendations for current and ongoing agricultural programming based on the conclusions, and lessons learned. 2 THE DEVELOPMENT PROBLEM AND USAID’S RESPONSE The PIKA Action Memorandum describes a development problem in essentially the same terms outlined in the Introduction—that is, the necessity of dramatically accelerating growth of the agricultural sector in order to reduce the poverty, malnutrition, and unemployment that impede India’s economic growth. The remainder of this section briefly describes the PIKA program and the four projects implemented under PIKA as USAID’s response to the development problem. The necessarily brief descriptions of the four projects serve as background for the evaluation. The section concludes with a brief summary of USAID’s future agricultural programming that frames the recommendations and lessons of the evaluation. 2.1 USAID’s Response: PIKA The Partnership for Innovation and Knowledge in Agriculture (PIKA) program supports the Agricultural Knowledge Initiative (AKI), a shared framework under which the U.S. Government (USG) and the Government of India (GoI) cooperate to develop the Indian agricultural sector. USAID activities within the AKI focus on “increasing farm-based incomes by increasing productivity and enhancing value through more effective market linkages.” The PIKA Action Memorandum describes a program to increase small-farmer incomes by leveraging public-private partnerships (PPP) to develop and disseminate technologies and practices that will increase productivity and foster more effective market linkages for Indian agricultural producers and processors. To implement PIKA, USAID/India invited interested parties to submit concept papers for PPP projects that would foster innovative approaches to developing microenterprises, building institutional capacity, and disseminating knowledge and information to support agricultural development. Key areas of focus included (1) increasing productivity and developing more efficient market linkages,( 2) risk mitigation and provision of financial services, (3) water resource management, (4) workforce development, (5) biotechnology, and (6) microenterprise development. Proposals also needed to address two cross-cutting issues: (1) designing strategies for documenting and widely disseminating the innovative approaches, knowledge, and best practices developed during the project and feeding the experience and data into the policy dialogue and (2) building the capacity of Indian institutions to sustain PIKA results beyond USAID support. The Action Memorandum for PIKA also stipulated that PIKA proposals should, to the extent possible, build on previous USAID activities in agriculture, focus on poorer states (i.e., Rajasthan, Orissa, Bihar, Uttar Pradesh, Madhya Pradesh, Jharkhand, etc.), and build linkages and synergies with other USAID program areas. It also instructed bidders to design projects that increase understanding of gender issues in agricultural reform and empower and benefit women farmers and agricultural laborers. While each project will develop its own outcome indicators and benchmarks, the Action Memorandum suggests the following: • Increased farmer incomes (including for women farmers and agricultural laborers); • Improved smallholder access to markets; Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 4 • Enhanced agricultural value chains that enable market development and increased trade; • Increased use of systems and approaches that lead to better management of water resources; • Increased institutional capacity for support to the agricultural sector through training, extension services, applied research, and information dissemination; and • Effective technologies and practices adopted into training and technical assistance curricula of public and private institutions. The mission expects the PPP model to leverage USAID investments, attain scale more quickly, extend impacts broadly throughout India’s agricultural sector, and produce results that are more sustainable than through unilateral activities. The primary PIKA-supported interventions by each project include: Indian Horticultural Development Alliance (IHDA) – Implemented by Michigan State University (MSU), the IHDA project focuses on building the capacities of horticulture producer groups to meet market demands and then linking them to markets. The project works across the entire horticulture value chain, from standards-setting to production to marketing. MSU’s primary role is coordinating the activities of the myriad partners and connecting producer groups to markets. Increasing Productivity and Market Links for High Value Agricultural Products – The project is implemented by the University of Wisconsin-Madison (UW) and aims to enhance farmers’ incomes by facilitating the transfer of productivity-enhancing technologies and practices to small farmers. The primary project activities include: • Providing technical assistance and expertise to help Mahindra & Mahindra establish soil testing laboratories in 140 dealerships across the country and train soil testing technicians and outreach specialists to staff the laboratories. • Helping the Rajiv Gandhi Charitable Trust develop and implement training materials to disseminate knowledge of improved dairy and vegetable production practices to members of women's self-help groups. • Helping Tasty Bite, a producer of Indian prepared foods for domestic and export markets, to identify improved production practices for dissemination to Tasty Bite contract farmers. Improving Efficiencies in Commodity Value Chains Through Advanced Cropping Technologies, Uttar Pradesh – The project is implemented by World Vision with the goal of increasing household incomes for male and female small farmers and farm laborers. The project coordinates with partners (ACDI/VOCA, RWC, AFPRO) to introduce farmers to productivity-enhancing and water-conserving technologies and practices and commercial market linkages. Rural Business Hubs: Business Catalysts for Rural Competitiveness with Inclusiveness – The International Food Policy Research Institute (IFPRI) implements the project in partnership with Michigan State University (MSU). The project provides detailed market research to three retail partners, ITC Ltd., Hariyali Kisan Bazar (HBK), and Viswas Business Synergies Ltd. to guide them in reducing the cost and improving the quality of inputs and services to small farmers provided through their Rural Business Hubs. The project provides no direct funding, only research support and strategic business advice. Annex A provides more detailed synopses of each project. Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 5 3 PURPOSE OF THE EVALUATION The PIKA evaluation Scope of Work (SOW) describes a standard evaluation task that addresses the impact, relevance, effectiveness, and efficiency of the PIKA program, and the four projects implemented under PIKA, and sustainability. In addition, the evaluation will determine the extent to which the projects have addressed gender issues. Finally, the evaluation will highlight lessons learned and their implications for improving ongoing performance and the design of future programs. During meetings at the outset of the evaluation, mission personnel articulated two perspectives for the evaluation. The backward looking perspective assesses the performance of the PIKA program and the four component projects in attaining objectives. The forward looking perspective generates recommendations and lessons learned which USAID can use to guide ongoing agricultural programming, both in India and in conjunction with Feed the Future initiatives in Africa. The discussions also clarified the mission’s expectations with respect to each of the evaluation themes (i.e., impact, relevance, effectiveness, efficiency, and sustainability) and refined the specific questions that the evaluation would address. The introductory paragraph in each section of the Findings and Conclusions chapter summarizes the parameters of the evaluation themes agreed to between mission staff and the evaluation team. 28 4 EVALUATION METHODOLOGY Annex B contains the specific evaluation questions. The five-person evaluation team spent five weeks in India (between January 9 and February 12, 2011) and about three weeks in the field. The team split into two sub-teams to conduct separate evaluations of the four projects implemented under the PIKA program. The teams employed a variety of methods to collect data, including individual interviews with 113 key informants, structured group discussions with more than 500 farmers, document review, site visits, and secondary data (when applicable and available). The teams used individual interviews to collect data from key informants, implementing partners, alliance partners, selected beneficiaries, stakeholders, and others as appropriate and conducted structured group discussions, primarily with farmer/beneficiary groups. Annex C provides details of the field work, including a summary of data collection activities and a complete list of the individuals and groups interviewed. The team reviewed project planning and implementation documents provided by USAID and the implementing partners as well as other studies and background material relevant to the evaluation. 4.1 Data Limitations The data available to the evaluation teams limited the scope and results of the evaluation to some extent. Significant limitations included: • An almost complete lack of an impact assessment strategy or data to support impact assessment (i.e., baseline and follow-up data on beneficiaries and, perhaps, control groups) meant that the evaluation teams could produce few rigorous estimates of the projects’ impacts on productivity or incomes. Consequently, assessment of impacts is restricted to largely anecdotal information on likely impacts with little reliable data on the magnitude of the impact (See Recommendations). • The evaluation team also had to rely on implementing partners to arrange field visits with beneficiaries. Neither evaluation team felt that partners had cherry-picked beneficiaries. Nevertheless, the potential remains that evaluation findings are not entirely representative. 28 The evaluation SOW specifies an assessment of whether “results achieved under PIKA are being produced at an acceptable cost compared with alternative approaches. . . .” The evaluation team and mission personnel agreed that the evaluation could not produce meaningful evidence of efficiency and agreed to eliminate the question. Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 6 • Time constraints and the wide geographic distribution of project activities restricted the number of sites the evaluation teams could feasibly visit and the amount of time they could spend at each site. An extended evaluation schedule would have improved the reliability and depth of evaluation findings. Additional time, rather than a larger team, would have been the most effective answer to this problem. • The small number of projects (four) and their relative homogeneity of approach (e.g., all are PPPs) to some extent limits the depth of analysis that the evaluation team could conduct about the relative performance of alternative approaches (e.g., PPP versus other approaches). • PIKA projects broadly supported ongoing activities of their partners by providing technical expertise and knowledge. It is often difficult in such situations to reliably determine the counterfactual—i.e., what would have happened without the PIKA projects. 5 FINDINGS AND CONCLUSIONS This chapter presents findings and conclusions for each of the evaluation questions (see Annex B). Each section presents findings and conclusions for each evaluation question, organized in two sub-sections: a summary of key findings (Annex D contains detailed findings) and conclusions. Annex E contains complete field notes from all interviews with groups of individuals. Field notes from individual interviews are not presented so as to protect the confidentiality of interview subjects. 5.1 Impact It is premature to expect the project to have achieved its full, anticipated impact on productivity or incomes. Furthermore, with some exceptions, the PIKA projects did not design strategies or collect data to support rigorous impact assessment. Reported estimates of impact are, therefore, largely anecdotal in nature. 5.1.1 Findings With regard to the extent to which PIKA activities promoted adoption of improved practices among targeted beneficiaries and the extent to which these practices increased beneficiaries’ productivity and incomes, the evaluation found: • Based on data supplied by M&M, the evaluation team estimated that at least 16,800 (and perhaps as many as 38,000) farmers adopted improved production practices as a result of services provided by PIKA-supported M&M Samriddhi centers. Most farmers who obtained soil tests (16,800) likely increased their productivity—and incomes—but it is not clear whether the increase was due to implementing improved practices or to external factors (e.g., prices). • The RGCT reports that it has trained 90,000 women’s self-help groups (WSHG) members in improved dairy and vegetable production practices. It is not possible to determine how many have actually adopted improved practices as a result. Consistent anecdotal data from 147 women strongly suggest that improved dairy practices have increased milk production and incomes, but it is not possible to quantify the impact. Only three households reported selling large quantities of vegetables. When asked about the benefits of using improved vegetable production practices, Evaluation Questions – To what extent did PIKA promote adoption of best practices among targeted beneficiaries and others? To what extent have PIKA activities increased productivity and incomes of targeted beneficiaries? Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 7 most women spoke about having an increased variety and quality of vegetables for home consumption rather than increased income from the sale of vegetables. • Projects that implemented improved dairy production practices among small farmers similar to those promoted by RGCT reported production increases in excess of fifty percent. • The IHDA project reports that it has directly trained 3,625 members of fourteen producer clusters in improved production and post harvest practices. It reports having increased the incomes of these farmers by an aggregate of over $3 million during the 2009/10 agricultural season. APK Banana World will soon open a second packing facility which will engage in additional 2,100 to 2,400 farmers—most of whom will eventually receive training in improved production practices. • All of the nine M&M lead farmers that the evaluation team interviewed who had tested their soils reported increased yields. M&M reports that ninety-six percent of the 3,800 farmers engaged in the productivity enhancement program registered a productivity increase of at least ten percent with a minimum increase of four percent, a maximum of fifty-two percent, and an average of fifteen percent to eighteen percent. 29 • Based on data provided by ITC and HKB and the observations of the evaluation team, the team estimated that 22,161 farmers supported by ITC and 96,187 farmers supported by HKB adopted improved production practices and increased yields (in the range of fifteen to twenty percent according to the field research of ITC and HKB). Although anecdotal evidence suggests that all of the beneficiaries experienced increased incomes of a similar magnitude, it is not possible to verify the exact magnitude of the change. • ITC has found it difficult to incorporate small farmers into its wheat and soybean procurement system due to the individual small lots on offer and the inability of the small farmers to absorb the cost of transporting their products to the ITC procurement hubs. • Through the Rice and Wheat Consortium (RWC) subcomponent of the World Vision project, 8,250 farmers were persuaded to adopt improved practices and of these, 7,425 improved their productivity and incomes. Evidence includes: o Based on reports from farmers who participated in 169 on-farm zero tillage demonstrations, RWC concluded that farmers can gain Rs. 12,000 to Rs. 17,000/ha from savings in energy and water use, yield increases due to timely sowing, better grain quality and higher market prices by adopting zero tillage instead of common plowing. o In thirty-one on-farm trials of direct seeding of rice, rice grain yields averaged eight percent higher than for transplanted rice. o Based on twenty-one integrated crop management trials conducted by RWC, laser land leveling, planting of improved varieties, application of balanced fertilizers, integrated weed management, and need-based pest management increased yields by fifteen to twenty percent, as compared with traditional systems of rice cultivation. o Raised-bed planting demonstrations with wheat by RWC enabled planting of intercropped sugar cane with no additional field preparation or irrigation water, adding to 29 The yield and productivity results are not conclusive because they do not control for external factors (e.g., weather) that may have affected yields. Current (2010) data on yields are not available as a control. However, the consistency of the reported results over two regions, different crops, and data collection methods (interviews and detailed quantitative data collection) strongly suggest that they are at least partially attributable to the intervention. Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 8 farmer income. During 2010, six farmers adopted this technology. Production of green gram during the summer period, when rice and wheat fields normally lie fallow, resulted in additional gains of Rs. 10,000 to Rs 20,000/ha. while green manuring increased transplanted rice yields by one half to one tonnes/ha over rice grown without green manure. • With the exception of the activities carried out by key partners RWC and AFPRO, the World Vision project had no discernable impact on farmer productivity and incomes. • All of the lead farmers in the M&M hub and spoke dissemination model reported training and/or advising other farmers. Although the potential is large, it was not possible to estimate the scale of this secondary dissemination. • Representatives of each of the private sector partners (i.e., M&M, Tasty Bite, ITC, HKB) said they sought to improve their relationships with farmers in order to earn customer loyalty. And, in the case of ITC, to increase procurement volumes. • Limited data from M&M imply that increased production costs for cotton almost completely offset the value of increased yields, leaving productivity almost unchanged. However, these data are much too thin (small sample, one crop) to draw any broad conclusions about the effectiveness of the interventions. • Based on their own field research, ITC and HKB concluded that the yield gains for soybeans and wheat brought about by the adoption of the improved practices and inputs resulted in commensurate gains in net income. Annex D contains detailed findings by project and project component. 5.1.2 Conclusions PIKA projects disseminated information about productivity-enhancing production and marketing practices to a conservative estimate of over 240,000 farmers. The numbers might be much larger since it is difficult to determine the extent of dissemination through the hub and spoke model employed by some projects or the extent of spillover effects as farmers not directly involved with a project adopted practices they observe their neighbors using effectively. Project partners had different motivations for disseminating improved production and marketing practices. Private sector partners (M&M, Tasty Bite, ITC Ltd, HKB, and processors) generally sought to improve their relationship with farmers in order to earn their loyalty as customers; enhance the quantity, quality, and consistency of produce entering their facilities; or gain more control over their supply chains. NGO partners, on the other hand, were motivated largely by social responsibility. The source of motivation, however, had no apparent effect on the extent to which partners disseminated the knowledge which ultimately benefited farmers. The size of partners’ networks and their approach to disseminating information through these networks were the more important determinants of the number of farmers exposed to improved practices. All of the PIKA projects promoted practices or technologies that were effective in increasing agricultural yields—although some projects (WV) did not promote the practices particularly effectively. Only three (M&M, ITC and HKB) attempted to collect the data necessary to determine whether the practices actually improved productivity—i.e., whether the magnitude of the yield increases justified the (possibly) increased costs associated with implementing the practices. Quantitative data on productivity improvement, however, require additional analysis to be convincing (see detailed findings in Annex D). Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 9 Even though, in most cases, it was not possible to rigorously quantify yield and income impacts, the consistent anecdotal information provided by a vast majority of farmers in all projects strongly supports the conclusion that the practices did, in most cases, increase yields, productivity and incomes.30 5.2 Sustainability It is premature to make final judgments about the sustainability of results since the projects are not yet complete and business models are still in a testing phase. This section assesses the prospects for sustainability, in part by assessing the extent to which the projects have built partners’ capacities to sustain the activities. 5.2.1 Findings With regard to determinations of likely sustainability, the evaluation found: • Managers of all of the private sector programs (M&M, Tasty Bite, ITC Ltd, HKB and processors) indicated that they would continue the initiatives in the long run only if they were ultimately financially viable—i.e., generated returns sufficient to cover costs. • HKB credits a twenty-five to thirty percent annual growth in sales of agricultural products, and a fifty percent growth in nonagricultural products, between 2009 and 2010 to the services provided by their two-year old agri-services division, to which PIKA contributed. • With technical assistance from UW, M&M established 135 advanced soil-testing laboratories at its Samriddhi centers. Individual dealerships invest as much as Rs.300,000 to establish a lab and Rs.30,000 or more annually for operation. Samriddhi Center managers describe the labs as the foundation of their strategy to enhance their engagement with farmers. M&M ultimately expects to transform 400 dealerships into Samriddhi centers. • RGCT staff and community resource persons (CRPs) said that the innovative training materials produced by UW for the RGCT have substantially enhanced the organization’s capacity to disseminate information on productivity-enhancing dairy and vegetable production technologies to members of WSHGs and built the capacity of the CRPs to conduct effective trainings. • RGCT is now supporting 27,000 SHGs with more than 400,000 women as members (about one￾fifth of whom have been trained in improved production practices to date)—substantially more than the roughly 8,500 SHGs it anticipated as recently as 2008. Part of its work with the SHGs focuses on building the capacities of the SHGs themselves (organizational management training, basic bookkeeping skills, etc.). By contrast, the World Vision project, which also works with SHGs, does not engage in any institution- and capacity-building activities with them. • APK Banana World partners have made substantial investments in developing an integrated cold chain facility for bananas in Tamil Nadu. They reported that the support of both IHDA and partners in disseminating improved production practices and linking producers to processors was 30 Defining income broadly to include the value of any additional production whether or not it is actually sold. Evaluation Questions: Based on results to date, how likely is it that PIKA results will be sustained after USAID funding ends? To what extent did PIKA activities catalyze private sector capacity building and adoption of innovative technologies and business practices, influence policy, and contribute to scaling up of project-supported activities? Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 10 an important factor in their investment decisions. APK will soon open its second facility, which will increase its capacity more than tenfold, and it has a third facility of similar capacity on the drawing board. • Tasty Bite personnel said that they could engage about 135 farmers in contract farming in the coming two or three years if they continued to grow as expected. • Tasty Bite was using wastewater from its processing plant to irrigate adjacent fields. However, it noticed a decreased yield in fields on which it was applying the water. UW helped Tasty Bite diagnose the problem (residues in the water clogging the soil and preventing absorption of water and nutrients) and design a solution. Tasty Bite now recycles 35,000 liters of water per day through a state-of-the-art filtration system. • All of the PIKA partners that the evaluation teams interviewed (with the exception of World Vision) said that they would probably have pursued many of the initiatives that PIKA projects supported even without PIKA support. They reported that PIKA support helped them advance these initiatives more quickly and effectively and enabled them to adopt some new initiatives that they might not have considered otherwise. • Tamil Nadu Agricultural University personnel reported that their long association with MSU has increased their capacity to develop horticulture value chains and disseminate information to farmers. • IHDA is supporting development of the Horticulture Knowledge Network and has contributed to creation of two horticulture knowledge centers through which to broadly disseminate knowledge of horticulture value chains. Annex D contains detailed findings by project and project component. 5.2.2 Conclusions In general, it is premature to determine the sustainability of any of the PIKA projects. The long-term sustainability of the private sector-led partnerships (M&M, Tasty Bite, IHDA, ITC, HKB) will depend on the projects achieving financial viability—i.e., that the innovative business practices ultimately generate sufficient revenue to cover costs. Sustainability may also require that partners develop the capacity to continue the activities performed directly by PIKA’s implementing partners (e.g., facilitating market linkages, establishing soil testing laboratories, conducting market research). Results based on knowledge and skills embedded in individuals and communities (for example, knowledge of improved production practices, animal husbandry, importance of soil health) is generally sustainable as long as other elements of the value chain (market access, input availability, etc.) exist to provide the incentives for investing in productivity. Most of the private sector activities (with the exception of Tasty Bite) show substantial potential for scaling up. In each case, partners have targeted large and relatively untapped markets and the evolving markets for agricultural inputs and products will drive expansion of economically viable projects. Tasty Bite is a relatively small operation with limited capacity or motivation to disseminate knowledge beyond the relatively small number of farmers that could directly supply its processing facility. All of the PIKA projects, with the exception of World Vision, contributed to existing programs or ongoing initiatives of partners by effectively filling technology, knowledge, or expertise gaps (see Annex A for a complete description of the contributions of the PIKA projects to their partners’ initiatives.) Since the partners were already pursing these initiatives, it is likely that they might have eventually made most Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 11 of the investments even without PIKA. PIKA, however, was able to substantially accelerate the investment process and in several cases, prompt initiatives that might never have been made due to lack of essential information on the part of PIKA partners. Examples include HKB’s dairy program, agri￾services division, and other program aimed at small farmers; ITC established 170 agri-service centers throughout its commodity procurement zones to serve small farmers; IFPRI’s research convinced ITC and HKB that small farmers represent a huge and under-serviced market for farm inputs and services; UW provided the technical expertise necessary to help M&M establish effective and accurate soil testing laboratories and helped RGCT enhance its capacity to train farmers in improved dairy and vegetable production; and MSU’s expertise facilitated the creation of Good Agricultural Practices (GAP) standards and effectively linked farmers to more remunerative markets. 5.3 Relevance Questions of relevance have a backward- and forward-looking focus. They contribute both to guiding changes in project approach for the remainder of the implementation period and to formulating future programming. 5.3.1 Findings With regard to determinations of relevance, the evaluation found: • None of the many farmers interviewed during the course of the evaluation reported receiving relevant advice on seed selection, fertilizer and pesticide use, or cultivation practices custom tailored to their farm from government extension, universities, or non-PIKA affiliated input suppliers. • All of the PIKA partners interviewed during the course of the evaluation who spoke to the issue said that government and university extension services in India did not effectively reach small farmers. They expressed the opinion that universities and government extension were not interested in working directly with farmers and that universities were not interested in applied research at the farm level. In addition, the World Bank and FAO both concur that extension services in India are weak. 31 • With only three exceptions, all of the farmers who obtained soil tests from M&M Samriddhi centers reported reducing their use of fertilizer (and thus their fertilizer cost) and increasing yields as a result of advice based on the soil tests. The soil-testing component of the WV project was not effective, because without sufficient follow-up by WV, the volunteers who received the kits lacked the motivation and knowledge required to use the kits themselves or to introduce soil testing to other farmers in their villages. • A large majority of the 540 farmers interviewed during the course of the evaluation who had adopted improved technologies and practices reported increased productivity and incomes as a result. 31 See footnotes on page 2. Evaluation Questions: Are program activities relevant to the needs of targeted beneficiaries and applicable in the current environment? Are the original hypotheses on which the program was designed still valid and relevant to the needs of the region? Have PIKA partners maintained relevance by adapting to emerging opportunities to achieve program objectives? Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 12 • Even though UW's pre-project assessment identified a number of potential constraints that women faced in implementing improved dairy practices (e.g., some resources/materials not available locally, some practices did not pay off financially, some practices too labor intensive for some households, milk collection infrastructure not sufficiently developed in some villages), few of the seventy-four women the evaluation team interviewed mentioned these constraints. RGCT is currently pursuing a partnership with Mother Dairy to help develop milk collection infrastructure, thus enhancing market access for project-assisted dairy producers and addressing one identified constraint to improving incomes through milk production. • ITC and HKB use market research provided by IFPRI specifically to improve and maintain the relevance of their marketing and outreach efforts to small farmers. Examples of adaptations to improve relevance to small farmers include: o ITC, to accommodate small farmers’ concerns about price, has formed an alliance with BASF to market a private-label herbicide at a lower price than the similar products sold under the BASF brand. o HKB is marketing inputs in sizes and price ranges suited to small-farmer requirements. o HKB has launched an improved milk collection and marketing system that has resulted in significant increases in local milk prices to small dairy farmers. • HKB management reported that they are experimenting with a broad range of innovative services to farmers, including a mobile input delivery program; custom crop protection services; and tie￾ups with financial institutions, insurance providers, education providers, and even a service that provides transportation to the HKB retail hubs for farmers with no personal means of transport. • RWC introduced more than 8000 farmers to new technologies—including laser land leveling, zero tillage, direct seeding, raised beds and others —that resulted in application of reduced volumes of irrigation water, seeds and fertilizers, resulting in significant reductions in production costs. • AFPRO built twenty-two water-retention ponds and two small dams, resulting in raising water tables in the affected areas. • ACDI/VOCA is currently training farmers to take advantage of the new National Warehouse Act, which would enable the farmers to store their grain for later sale at a higher price, rather than selling it at harvest time. The intervention is too recent to evaluate its effect. Annex D contains detailed findings by project and project component. 5.3.2 Conclusions Government extension services and university researchers in India for the most part are failing to provide relevant, tailored, technical information to small farmers. A common consensus among those that the evaluation team interviewed was that university scientists are interested in research to develop general knowledge (e.g., best practices or technologies) but have little interest in adapting the research to practical farm use and that government extension services are not sufficiently funded or motivated to work effectively with small farmers. Each of the PIKA projects, in its own way, contributed to filling this gap by facilitating the transfer of relevant, productivity-enhancing technologies and practices to small farmers. One of the most significant challenges to further improving the economic status of small farmers in India is finding means to fill this significant gap in research and extension services. Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 13 The fact that PIKA-promoted technologies and practices almost uniformly increased yields and productivity (see findings for Impact) implies that the interventions were relevant to achieving project objectives. There is insufficient information, however, to estimate the magnitude of changes in productivity or income or to determine whether the promoted practices are the best practices in the current environment or the most applicable for the targeted beneficiaries32 PIKA partners have generally adapted their activities to the changing environment or to the realities of their work environments. ITC and HKB were particularly good at adapting their activities to maintain relevance because they were designed specifically for that purpose. The companies are using the results of the market research provided by IFPRI to target their products and services to small farmers. . 5.4 Effectiveness Questions of effectiveness focus on the extent to which individual projects have achieved their outcomes and contributed to broader outcomes of the PIKA program. They also assess the strengths and weaknesses of the PPP approach as an implementation model in India and elsewhere. 5.4.1 Findings With regard to project effectiveness, the evaluation found: • M&M is engaging, in the initial stages of implementation, a disproportionate number of medium and large farmers relative to small farmers. Of the sixty-one farmers from whom the evaluation team obtained landholding data, twenty percent were large farmers (cultivating more than twenty￾five acres), thirty-five percent were medium farmers (cultivating five to twenty-five acres), and forty-five percent were marginal or small farmers (cultivating less than five acres). The 2000– 2001 Agricultural Census of India reports that eighty-two percent of India’s farmers were marginal or small. 33 • The RGCT project focused exclusively on small farmers. • Some sixty to eighty percent of the farmers in the areas of activity covered by World Vision, ITC and HKB can be classified as small farmers (cultivating less than five acres)34 • UW helped M&M establish 135 soil-testing laboratories and train 121 soil technicians and 121 outreach specialists—slightly short of the target of 140 soil-testing laboratories (see Annex D for detailed findings on projects’ performance relative to targets). and the market￾based approach of these projects engaged small farmers in numbers proportional to their prevalence in the population. 32 For example—and there is insufficient empirical evidence to do more than flag the question for additional research—if M&M is promoting mechanized solutions that substantially increase production costs, these may not be the most applicable to small farmers. 33 Agricultural Census of India, 2000–2001, Analysis of Agriculture Census Results – All India All Social Groups . 34 Throughout the report, “small farmers” refers in aggregate to marginal and small farmers. Evaluation Questions: How effective has PIKA been in achieving its objectives? To what extent have PIKA partners achieved their individual objectives and contributed to broader PIKA objectives and sub-objectives? Is the PIKA model (i.e., public private partnerships, GDA) an effective mechanism for catalyzing innovative technologies and business practices to achieve program objectives? Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 14 • UW personnel do not believe that PIKA activities were entirely consistent with Tasty Bite’s expectations or needs. Tasty Bite personnel reported that the company did not have a documented corporate program or initiative to which PIKA contributed. • RGCT personnel reported that effective training materials, appropriately designed for the audience by UW, were crucial to implementing the RGCT’s livelihoods improvement strategy in the UP context. • The RGCT reports forming 27,000 SHGs and training 1,700 CRPs to disseminate information about improved dairy and vegetable production practices. This exceeds the initial targets of 8,500 SHGs and 900 trained CRPs contained in the UW project award document. According to CRP training records, the CRPs have subsequently trained 90,000 SHG members in improved dairy and vegetable production practices. • Although the lead farmers in the IHDA project are all large farmers, project personnel and processors linked to the project supported producer groups said that the project engages small farmers in numbers proportional to their share of all farms (about 70%). • World Vision management confirmed the assessment by their key partners that lack of experience in agricultural development by World Vision project staff, basic mismanagement, and staffing problems (i.e., frequent turnover, complete replacement of staff, extended vacancies in specific posts, inappropriate experience/training) greatly reduced project effectiveness. Selected examples of issues they mentioned include: o None of the original World Vision PIKA team, nor any of the World Vision ADP personnel in the project area, had any previous experience with agricultural development projects or were qualified agriculturalists. This resulted in failure to realize that project implementation needed to be coordinated with cropping cycles and failure to respond to market linkage opportunities in a timely fashion. o Recent university graduates with no agricultural background were hired as field coordinators and no qualified agriculturists were included on project staff. o High staff turnover: field coordinators remained on the job an average of four months, primarily due to the fact that they were forced to relocate to blocks with few amenities. One of the field coordinator positions has remained unfilled for several months. The entire WV team has been replaced within the past six months. The M&E position is still vacant. o WV failed to engage in basic institution and capacity building for SHGs and other farmer groups. WV project management and key partner ACDI/VOCA reported that they have established no links to date between the SHGs formed by WV and any financial institutions. o Local farmers appointed to serve as village technical service providers did not receive further training following their initial training session, nor was any technical back-up provided on a continuing basis to strengthen their skills and increase their motivation. o According to ACDI/VOCA, which was responsible for most of the training, World Vision routinely paid male and female farmers a fee for attending training and furnished snacks and soft drinks. Thus, the primary motivation for attending the training sessions became access to the fees and snacks. After the lunch snacks were served, most of the farmer trainees typically disappeared. Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 15 o ACDI/VOCA, in its advisory role, attempted to inject a number of innovative elements into the project. Some of these were successful, including the second attempt at introducing mobile connectivity for market and weather information and the training of farmers to take advantage of the new warehouse receipt system, but most were not, due to the lack of adequate response from the lead firm. ACDI/VOCA was limited to an advisory role and had no control over the lead firm’s adoption and implementation of their recommended initiatives. Examples include:  A trading firm introduced by ACDI/VOCA offered to contract for a major tonnage of wheat from project farmers. World Vision agreed but was unable to organize the aggregation of the specified quantities for two years in a row. The trading firm walked away, disgusted.  A second company introduced by ACDI/VOCA offered to assist 500 farmers to establish a producer company, and was willing to provide half the cost and contract to purchase the farmers’ entire crop. WV delayed its decision regarding the proposal for four months, and the opportunity was lost.  ACDI/VOCA arranged a meeting between WV and HKB for the purpose of proposing a contract arrangement with project farmers for seed production. HKB and ACDI/VOCA met, but WV, with no notice, failed to attend the meeting.  ACDI/VOCA proposed to train farmer volunteers to carry out soil testing, on a fee basis, including distribution of more than 400 soil-testing kits to the volunteers. ACDI/VOCA conducted the initial training, but the kits remained in the WV district offices until a new WV management team had them distributed to the volunteers. However, no provisions were made for continuing technical back or follow-up training for the volunteers, or monitoring to determine if the kits were being used. The evaluation team interviewed a dozen of the volunteers who had received kits and found that none of them were being utilized. These are just a few of the instances where WV failed to take advantage of opportunities presented by one of their key partners. o WV was unwilling to seek approval from the mission to abort activities included in the original project proposal that proved to be unworkable once the project got underway. A prime example was the decision to go ahead with a mango value chain assessment, even after it was discovered that there were very few mango trees in the project area. o On the positive side, the subcomponents for which RWC and AFPRO were responsible were successful in achieving their project goals. o Perhaps most important, current (new) project management under World Vision realizes the mistakes that were made in project implementation and is attempting to rectify as many of these as possible during the limited time left to the project. Annex D contains detailed findings by project and project component. 5.4.2 Conclusions While some of the projects have failed to meet all of their specific output targets (see Annex D findings on Effectiveness), all of them—with the exception of significant elements of the World Vision and Tasty Bite projects—appear to be generally achieving their expected outcomes, including disseminating relevant productivity enhancing technologies and practices to small farmers. Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 16 The projects were largely successful in their efforts to target small farmers. For instance: • The RGCT project focused entirely on small farmers. • While World Vision’s efforts to benefit small farmers with technical services have been largely ineffective, ITC and HKB have been successful in reaching 131,499 small farmers with improved inputs and technical extension services. • Reports from ITC and HKB, confirmed during the evaluation team’s visits, found that a majority of the villages in the project area were composed of sixty to eighty percent small farmers and ITC and HKB targeted this segment of the farming population with their outreach efforts. • The IHDA project, while it seems to engage most directly with larger farmers, ultimately appears to be disseminating productivity-enhancing technologies and practices to a representative distribution of farm sizes. • The M&M hub and spoke dissemination model focuses first on engaging customers of the dealership, generally tractor owners, and are thus larger farmers (medium and large). Compelling evidence suggests that these farmers are effectively disseminating information to a potentially large number of small farmers. However, data do not exist at this time to estimate the magnitude of this effect. In the case of these four projects, PPPs have proven an effective model for disseminating improved technologies and practices to small farmers. Successful PPP development projects in India and elsewhere have two common characteristics: they are based on commercially viable activities with definite links to profitable markets and they follow a value chain approach. 5.5 Gender Gender refers to the respective roles of men and women and not to an exclusive focus on empowering women. The question on gender mainstreaming asks whether PIKA projects have effectively considered women’s roles in agriculture during design and implementation. Annex F provides a detailed assessment of how PIKA and the individual projects addressed gender issues. 5.5.1 Findings With regard to project’s mainstreaming of gender issues, the evaluation found: • The Action Memorandum acknowledged the important contribution of women to agriculture and required that proposals support initiatives that invest in women and strengthen their access to knowledge. • None of the PIKA project proposals contained an adequate or detailed description of how the project would address gender issues. Evaluation Question: To what extent have partners mainstreamed gender issues into PIKA activities? Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 17 • The UW project with RGCT initially focused exclusively on women’s empowerment. While it is starting to engage men to gain permission to work more effectively with women, the respective roles of women and men in agriculture or the household remain largely unchanged. • The IHDA project monitoring report indicates that 2,656 women farmers have been trained. However, none of the twelve women (four farmers and eight laborers) that the evaluation team interviewed had received any training. Men said that they received the training and then trained their wives, who managed the farming operations. • IHDA personnel report attempting to identify partners to address gender issues (Daht, BAIF, and the National Institute of Agricultural Extension Management (MANAGE)) but failing, so far, to find a suitable partner. • IFPRI project activities with ITC and HKB do not appear to have any evident gender focus.35 • World Vision’s existing Area Development Program (ADP) works with WSHGs to address welfare and livelihood issues, but does not include programs addressed to the role of women in farming. Under the PIKA project, World Vision, with the assistance of key partner ACDI/VOCA, formed 153 men and women SHGs and provided some rudimentary training. Due to lack of proper capacity building for the SHGs, however, there were no concrete results. There are no specific measures included to increase involvement of women or to address the respective roles of men and women in agriculture. • World Vision management reported that their definition of “working” with beneficiaries includes counting everyone who attended trainings or meetings or encountered project personnel in any other context. The evaluation team interviewed 143 village women (farm women and laborers) who attended training or meetings or were members of an SHG formed by WV. In most cases, with the exception of a few women who opened accounts in local banks, the women reported that there had been no change in their activities, status, or wages as a result of “working” with WV. WV’s plans to bring in national bank,s including ICICI and Oriental Bank of Commerce, did not materialize, primarily due to the fact that too few of the project SHGs were sufficiently well organized for their members to be eligible for bank services. Annex D contains detailed findings by project and project component. 5.5.2 Conclusions Though there have been changes in women’s positions, these were normally not significant. Farm women continue to provide much of the labor on the farm but have little or no decision-making power and earn lower wages than men for similar work. To a large extent, projects’ capacity-building activities ( training, for example) engaged mostly men. Socio-cultural norms and traditional gender-biased practices contributed to inequality between men and women. Male farmers are usually the decision makers; women implement the decisions. 36 While all of the PIKA projects included at least some treatment of gender issues in their proposals, none of them, at least initially, effectively addressed the issues. Some (RGCT and IHDA) eventually recognized the value of addressing the concerns of both men and women in a coordinated fashion and 35 The evaluation team did not interview any female farmers in connection with the IFPRI project. 36 In the UW project with RGCT, women are experiencing social empowerment but they still lack economic empowerment. The men, in general, control the income generated by women’s economic activities. Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 18 began to integrate more gender-appropriate approaches. However, IHDA has had difficulty identifying appropriate partners to address gender concerns. Other projects (ITC, HKB, WV) have not yet effectively addressed gender issues. 6 RECOMMENDATIONS 6.1 Recommendations for Ongoing PIKA Activities Enhance the prospects for sustainability and impact of existing PIKA projects. It is too late to alter substantively the activities of the four PIKA implementing partners. However, all of the PIKA-supported initiatives will likely continue beyond the PIKA program, so there is scope for guiding the future activities of project partners to improve performance. Implementing partners should be in the process of identifying potential improvements in partnership activities and positioning partners to implement changes. For example: • ITC can enhance the relevance of its services to farmers by extending procurement activities to commonly-produced crops other than rice and wheat (e.g., gram) and offering dairy support services, provided that the initiatives are consistent with corporate policy and are commercially viable. • ITC could increase the relevance of its work to small farmers by devising commercially-viable measures to aggregate and transport small farmers’ crops to the ITC procurement hubs. • World Vision could enhance small-farm development project sustainability by planning and effectively implementing institutional- and capacity-building measures aimed at agricultural SHGs and other groups in their on-going ADPs, and by utilizing lessons learned to improve their small-farmer development approaches. • UW should ensure that M&M has access to local resources to help them establish additional soil￾testing laboratories and train technicians to operate the laboratories. 6.2 Recommendations for Future Programming Ineffective research and extension remain critical constraints to agricultural development in India. The evaluation (and WB and FAO37,38,39 37 The World Bank, India. “Agricultural R&D is Critical to Enhance Productivity,” . ) identified ineffective government and university research and extension services as key constraints to agricultural development—especially for small farmers. The GoI should address this deficiency if it is to fully support small-farmer engagement in agricultural development. Given that these same shortcomings also exist in most other Asian countries, however, a more practical approach might be for interested parties, including donors, to seek alternative arrangements for providing extension services to small farmers, perhaps including means of developing and financing private extension services. Small farmer outreach by ITC and HKB are two examples of effective, private-extension services. In the case of agricultural research, focused programs to link users of research with the institutional researchers need to be effectively implemented. The evaluation concluded that PIKA’s PPP approach provided the expertise and technologies to help private sector and 38 Food and Agriculture Organization of the United Nations, “Agricultural Growth and Total Factor Productivity,” . 39 The World Bank, India. “Re-energizing the Agricultural Sector To Sustain Growth and Reduce Poverty,” Report No. 27889- IN, July 30, 2004. Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 19 NGO partners effectively fill the research and extension gap, as illustrated by the activities of ITC. HKB, RWC, M&M, RGCT and IHDA. Plan for impact assessment. The evaluation team was not able to obtain meaningful estimates of the impact of most of the PIKA projects on productivity and incomes, because the projects had not established the conditions necessary for assessing impact (the IFPRI project was an exception, but only because ITC and HKB conducted extensive market research prior to PIKA as part of developing their business strategy). Establishing a baseline is a necessary first step in impact assessment. Establishing a credible baseline a year or two after the fact is nearly impossible when dealing with small farm production systems. Beyond establishing the baseline, impact assessment requires (1) determining (by comparing baseline measurements to follow up measurements) whether conditions have changed for project beneficiaries and (2) attributing any observed change to the intervention. In some cases, assigning attribution may be relatively easy. For example, reliable research may have established the productivity improvements associated with a particular package of practices. In this case, the research results provide the counterfactual. In more complex situations where myriad external factors may affect outcomes, control groups or data-intensive econometric techniques may be necessary to establish the counterfactual. The expense of rigorous impact assessment may not always be warranted. If missions desire impact assessment, then they must be willing to fund it as an integral part of the project and not as an afterthought. They should ask bidders to propose an impact assessment strategy, carefully evaluate the adequacy of the proposed strategy, determine whether the cost is warranted, and ask bidders to rework the strategy if the cost is too high or the approach not rigorous enough. Improve gender performance. The PIKA solicitation was clear about gender objectives. However, none of the projects dealt with gender issues effectively. Recommendations for these projects relative to gender may help future projects improve their gender performance. Specific recommendations that could be addressed under gender issues include: • All staff involved in project implementation should be trained on gender and project-specific gender integration opportunities. In addition, there should be a gender point person who is responsible for guiding and supporting the progress of gender-related activities. This person should have sufficient resources and authority to make programmatic changes to improve gender￾related outcomes. • Ensure that proposals include the specific gender areas and activities required by the solicitation. All work plans should have activities or strategies for addressing gender disparities. The project review and monitoring should include gender indicators and/or ways to track changes in gender relations. • Provide capacity-building opportunities relevant to their roles to both men and women. To accommodate the needs of women, training venues should be near project areas. • Wherever appropriate, training activities should involve husband-and-wife teams in order to increase male support for women’s activities. • To improve women’s participation in the work force, work places should include provision of a crèche for infants and small children. Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 20 7 LESSONS LEARNED Characteristics of successful PPPs – At least in the environment in India, the PPP approach has been an effective and efficient40 mechanism to leverage USAID funding (by piggybacking on partners’ existing programs) to extend benefits to a large number of farmers. Because all of the PIKA projects were PPPs, however, the evaluation produced no evidence with which to compare the PPP approach to non-PPP approaches. Some lessons emerge from comparisons of the separate projects, however. The four PIKA projects and subcomponents represented seven individual PPP approaches. 41 • The PIKA experience with partners ITC, HKB, M&M and IHDA-supported processors demonstrate that aligning the business development interests of commercial firms with the objectives of donor-sponsored, small-farmer development programs is an effective means of marshaling the divergent areas of expertise required to effectively achieve development objectives. Two were NGO models and five were private sector models. While specific approaches to small-farmer development must be tailored to fit conditions in each country, some general principles learned through the PIKA experience can be applied in most instances. These include: • The successful PIKA projects (M&M, RGCT, ITC, HKB, IHDA) all contributed specific technical expertise to well formed, existing strategies closely aligned with development objectives. Those that did not perform as well suffered from poor management and qualifications (World Vision) and weak partner commitment (Tasty Bite). When entering future PPP arrangements, look for opportunities where USAID can leverage a comparative advantage (e.g., access to technical expertise and knowledge) to augment and compliment initiatives of motivated and qualified partners backed by documented strategies for pursuing activities consistent with development objectives. • USAID can enhance prospects for sustainable results from private sector-led partnerships by carefully assessing the quality of the partner’s business plan and the prospects for achieving the long-term financial viability necessary to sustain the activities. When sustainability requires building the capacity of a private sector partner to take over the contributions of USAID, then the project must include a realistic strategy for building this capacity. • NGO partners are motivated by social responsibility rather than commercial viability. Nevertheless, NGOs cannot continue to support a development program forever. Sustainability in this context requires that the NGO is able to build sustainable community organizations. This requires a broad focus on community development rather than a narrow focus on agricultural productivity. To enhance prospects for sustainable, NGO-led partnerships, USAID should carefully assess the capacity and track record of the proposed partner in broad-based community development. The proposed partner should also have an understanding of agricultural interventions sufficient to implement relevant activities, and be sensitive to the unique requirements of working within smallholder agricultural systems. The RGCT project is more likely sustainable than the WV project because its approach includes building the capacities of the SHGs to manage their business internally, while the WV project has no SHG capacity-building element. WV also had little understanding of household agricultural production systems. Achieving the conditions necessary for sustainability in SHGs requires a sustained effort – likely longer than two years –and will thus require an ongoing commitment from the partners. 40 To make one comparison, a donor-driven FAO project in western Pakistan that follows a value-chain, community-development approach has engaged 50,000 farmers over a six-year period at a cost substantially above that of the PIKA program. To make a fair comparison, the FAO project has to contend with security concerns that are not factors in India. 41 UW (M&M), UW (Tasty Bite), UW (RGCT), IFPRI (HKB), IFPRI (ITC), MSU, and World Vision. Partnerships for Innovation and Knowledge in Agriculture: Final Evaluation 21 • While it may seem to promote exclusivity, starting projects with a single-firm approach may be necessary in order to demonstrate to industry that the suggested innovations are commercially viable and sustainable. Once this is demonstrated in actual practice, it is much easier to convince others in a particular industry to adopt the project-sponsored innovations. The initial single-firm approach is also more cost effective, in that the project start-up investment is limited to the requirements for supporting the adoption of the innovations by a single firm. It may also be easier to convince a single firm to test untried innovations than to attempt to involve an entire industry from the start. In all cases, a value-chain approach should be followed in order to produce optimum results. These theses have been demonstrated in the case of at least one very successful previous project in India (the USAID-sponsored India Growth Oriented Micro-Enterprise Development [GMED] Program). • The potential to scale up successful activities depends on the vision, network, and resources a partner has to expand a financially-viable model. For example, Tasty Bite is unlikely to scale up the model beyond its own limited needs; RGCT has already demonstrated a commitment to dramatically scaling up the SHG project; at least one of MSU’s private-sector partners is already substantially scaling up activities; and ITC and HKB have achieved significant scale. Soil tests were effective in reducing fertilizer application rates and reducing production costs – Almost all of the farmers the evaluation team interviewed who had obtained soil tests had been over￾applying basic fertilizers, particularly urea and to some extent DAP, and under-applying trace elements. Farm-specific information provided by the soil tests, coupled with outreach to help farmers interpret test results, has allowed farmers to calibrate fertilizer use to soil needs and, in most cases, lower production costs. Mobile connectivity – The introduction of mobile technology for providing current market and weather information to small farmers introduced by ACDI/VOCA under the World Vision project failed in the first instance due to use of the wrong system. By linking with a tested and successful commercial service in the second instance, ACDI/VOCA was able to initiate the application of mobile connectivity successfully for these purposes. The sustainability of this innovation is questionable, however, since the Rs. 90 monthly cost has been subsidized by the project for the first three months and there is some doubt that small farmers will be willing to pay for the services in future. Gender objectives are often misunderstood and inadequately addressed – People often confuse gender development with women’s development or women’s empowerment—all of which may be valid development objectives in different contexts. USAID solicitations need to be clear about the appropriate concept in a given context and ensure that bidders understand the perspective and submit proposals consistent with objectives. Women’s roles in agriculture – Interviews with farmers confirmed that women perform most of the tasks associated with producing crops. Men generally prepare the land and harvest and market the crops. Women do most of the planting/transplanting, weeding, and post-harvest handling. When farmers hire labor for these tasks, women often manage the labor. However, with the exception of RGCT, the projects are training men in improved agricultural practices.