Malawi Strengthening Inclusive Markets for Agriculture (MSIKA) Program Final Evaluation 11/26/2020 DISCLAIMER: This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. MSIKA Final Evaluation Program: Food for Progress Agreement Number: FCC-612-2016/006-00 Funding Year: Fiscal Year 2016 Project Duration: September 28, 2016 – September 30, 2021 Implemented by: Land O’Lakes Venture37 Evaluation Authored by: Kadale Consultants (UK) Jason Agar Yamikani Chabwera Patience Mtembezeka McPherson Chatama David Namanjasi MSIKA Final Evaluation Page iii kadale@africa-online.net Table of Contents List of Tables...................................................................................... iv Acknowledgements............................................................................ vi Acronyms .......................................................................................... vii Executive Summary ......................................................................... viii 1 Introduction and Background ................................................ 1 1.1 Objectives of the Final Evaluation ....................................................... 1 1.2 COVID Impact on Evaluation Design and Implementation................. 1 1.3 Background to MSIKA Project.............................................................. 2 1.4 Key Changes to MSIKA ......................................................................... 4 2 Methodology and Implementation ......................................... 5 2.1 Methodology .......................................................................................... 5 2.1.1 Overview........................................................................................................... 5 2.1.2 Household Survey............................................................................................. 6 2.1.3 Farmer Qualitative Interviews..........................................................................10 2.1.4 Key Informant Interviews.................................................................................10 2.1.5 Review of MSIKA Documents and Data ..........................................................11 2.2 Limitations............................................................................................12 3 Results and Findings............................................................ 14 3.1 Farmer Based Organisations..............................................................14 3.1.1 Background and outline of work with FBOs .....................................................14 3.1.2 FBOs summary data .......................................................................................15 3.1.3 Training at FBOs .............................................................................................18 3.1.4 Lead Farmers..................................................................................................25 3.1.5 Sales Agreements and Sales ..........................................................................26 3.1.6 Processing and Storage ..................................................................................27 3.1.7 Access to Market Information ..........................................................................28 3.1.8 COVID Impacts ...............................................................................................29 3.2 Participant Producers..........................................................................30 3.2.1 MSIKA Producer Sample Frame .....................................................................30 3.2.2 Producer Totals...............................................................................................30 3.2.3 Final Evaluation Sample Profile.......................................................................31 3.2.4 Agricultural Practices.......................................................................................33 3.2.5 Crop Inputs .....................................................................................................39 3.2.6 Land Area Under Improved Practices..............................................................40 3.2.7 Production.......................................................................................................40 3.2.8 Post-Harvest Handling Knowledge and Application.........................................43 3.2.9 Post-Harvest Losses .......................................................................................50 3.2.10 Gender........................................................................................................53 3.2.11 Farm Management Knowledge and Application...........................................54 3.2.12 Sales Volume ..............................................................................................55 3.2.13 Sales Value ................................................................................................. 57 3.2.14 Access to Finance.......................................................................................59 3.2.15 Investment...................................................................................................59 3.2.16 Employment ................................................................................................60 3.2.17 COVID effects .............................................................................................60 3.3 Banks, MFIs and VSLs.........................................................................61 3.3.1 Overall Performance .......................................................................................61 3.3.2 Finance for SMEs............................................................................................62 3.3.3 Finance for Producers.....................................................................................63 MSIKA Final Evaluation Page iv kadale@africa-online.net 3.4 Processors ...........................................................................................64 3.4.1 Training and Application of Improved Techniques and Technologies ..............64 3.4.2 MBS Certification ............................................................................................66 3.4.3 Other Support to Processors...........................................................................66 3.4.4 Performance data............................................................................................67 3.5 Government..........................................................................................68 3.5.1 Policy Level.....................................................................................................68 3.5.2 Training District Extension Staff ......................................................................69 3.5.3 Malawi Bureau of Standards ...........................................................................71 3.5.4 Bunda/MSU.....................................................................................................71 4 Conclusions, Lessons and Recommendations .................. 72 4.1 Key Conclusions and Lessons...........................................................72 4.1.1 Methodology ...................................................................................................72 4.1.2 Overall Progress .............................................................................................73 4.1.3 Farmer Based Organisations...........................................................................74 4.1.4 Producer Households......................................................................................76 4.1.5 MFIs/Banks/VSLs............................................................................................77 4.1.6 Processors ......................................................................................................78 4.1.7 Government ....................................................................................................78 4.2 Recommendations...............................................................................79 Annex 1: Table of Program Indicators............................................. 81 Annex 2: MSIKA Results Frameworks............................................. 98 Annex 3: Terms of Reference/Scope of Work ................................. 99 Annex 4: Instruments...................................................................... 108 Annex 5: Comparison of Practices by Crop.................................. 149 List of Tables Table 1: Minimum and Actual Responses by Crop ............................................................... 8 Table 2: Usable Responses by Crop and by District............................................................. 9 Table 3: KIIs Conducted by Crop, District and Sex .............................................................10 Table 4: KIIs Planned and Conducted by Category ............................................................11 Table 5: MSIKA’s Performance Rating of FBOs, FE, 2020 .................................................15 Table 6: MSIKA’s Performance Rating of FBOs, MTE, 2019 ..............................................15 Table 7: PMM Assessment, MSIKA-Established FBOs at FE, 2020 ...................................16 Table 8: PMM Assessment, Pre-existing FBOs, 2020 ........................................................17 Table 9: MSIKA Activities Reported by FBO Members in the MTE & FE HH Surveys.........17 Table 10: FBOs Trained in Agricultural Production, by Type of Training, at MTE & FE.......19 Table 11: FBOs Trained in Marketing, by Type of Training, at MTE & FE...........................20 Table 12: FBOs Trained in PHH and Storage, by Type of Training, MTE & FE...................21 Table 13: FBOs Trained in Financial Records, Literacy and VSL, MTE & FE .....................22 Table 14: FBOs Trained in Governance, Leadership & Cooperatives, MTE and FE...........23 Table 15: Proportion of Lead Farmers in the FE Survey .....................................................26 Table 16: Lead Farmer Activities by Sex at FE, 2020 .........................................................26 Table 17: Sources of Market Information, FE 2020.............................................................28 Table 18: Indicator 18, Producers Accessing Market Info via their FBO, FE 2020 ..............29 Table 19: Beneficiary Producers, by District, Sex & Crop, FE 2020....................................30 Table 20: Comparison of Baseline vs FE Samples .............................................................32 Table 21: Check Balance of Confounders Using Propensity Score Matching .....................32 Table 22: MSIKA Activities Reported by FBO Members in MTE & FE HH Surveys.............33 Table 23: Application of New Agricultural Practices per Category, FE Survey, 2020 ..........34 Table 24: Application of New Practices by Crop at FE, 2020..............................................35 MSIKA Final Evaluation Page v kadale@africa-online.net Table 25: Use of Practices for the First Time, FE Survey, 2020..........................................37 Table 26: Why Other Practices are not Used, by Crop, FE 2020 ........................................39 Table 27: Land area in ha under improved practices, MTE and FE HH surveys .................40 Table 28: Producer Yields in kg/ha, FE HH survey, 2020 ...................................................41 Table 29: Producer Yields in kg/ha, Revised Baseline 2017...............................................42 Table 30: Yield Comparison, FE vs baseline by Crop.........................................................42 Table 31: Producers Affected/Not Affected by COVID, Yield Comparison vs Baseline .......43 Table 32: Yield Comparison by Sex, FE vs baseline...........................................................43 Table 33: Use of PHH Practices by Crop, FE HH survey 2020 ...........................................45 Table 34: Use of PHH Practices Across the Four FE crops, at Baseline.............................46 Table 35: First Used New PHH Practice, at FE, 2020.........................................................47 Table 36: First Used New PHH Practice, Across FE Crops at Mid-term, 2019....................48 Table 37: Post-harvest Losses by Crop and by Sex, FE, 2020 ...........................................50 Table 38: Post-harvest Losses by Crop and by Sex, Baseline, 2017 ..................................51 Table 39: Point at which Harvest was Spoiled, FE, 2020....................................................51 Table 40: Point at which Harvest was Spoiled, Baseline, 2017...........................................51 Table 41: Place of Storing Crops, FE and Baseline ............................................................52 Table 42: Building or refurbishing storage calculation for FE population.............................53 Table 43: Work on the Tasks by Sex at MTE Across for the Four FE Crops.......................53 Table 44: Use of Improved Farm Management Practices, Baseline vs FE..........................54 Table 45: Indicator 8: Application of improved farm management practices, FE.................55 Table 46: Volume sold in kg/ha, Revised Baseline vs FE ...................................................56 Table 47: Value of Crop Sold in $/ha, Baseline vs FE.........................................................57 Table 48: Prices in Malawi Kwacha and USD $, Baseline vs FE.........................................57 Table 49: Growers that Sold Nothing, FE, 2020..................................................................58 Table 50: Primary Means to Sell, FE, 2020 ........................................................................58 Table 51: Sales location, FE HH Survey, 2020...................................................................59 MSIKA Final Evaluation Page vi kadale@africa-online.net Acknowledgements The Kadale mid-term evaluation team acknowledges the valuable assistance received from the Land O’Lakes Venture37 team. This study required the active support and contributions of the headquarters staff and the Malawi Strengthening Inclusive Markets for Agriculture (MSIKA) team based in Lilongwe, Malawi. Our considerable thanks to Boniface Msiska and Lyson Kadzakalowa, who assisted us on all aspects of the work. They responded to our many questions and provided both data and insights for our evaluation activities. We are particularly grateful for the technical inputs of Damiano Chipeta, Benson Kasekera, Nyarai Makwiza and Freeman Ngoma that guided us in the understanding of the technical aspects of the work. We also recognise the contributions of Meredith Saggers and Joe Carvalho who provided overall direction and guided us on a range of issues. Kadale benefitted from its dedicated interviewing team who undertook over a thousand phone interviews with all the frustrations of out of service numbers, dying phone batteries, poor signal and persuading farmers to give of their time. The Final Evaluation Team was led by Jason Agar as Team Leader, Yamikani Chabwera as Research Manager, Patience Mtembezeka as Assistant Research Manager, McPherson Chatama as Fieldwork Co-ordinator, Mercy Butao as Advisory Agronomist with additional support from David Namanjasi on data analysis, all of whom worked long hours under great pressure to deliver this evaluation. We are grateful to our 49 key informants, but most of all we wish to recognise the 979 Malawians who agreed to be interviewed, some of them for qualitative as well as quantitative interviews, to tell us about their experiences of working with the MSIKA program. These producers are the intended beneficiaries of the program, who have to tackle the day to day struggle to improve their livelihoods in difficult and unpredictable circumstances. Jason Agar Director, Kadale Consultants, Lilongwe, Malawi. November 2020 MSIKA Final Evaluation Page vii kadale@africa-online.net Acronyms AEDC Agricultural Extension Development Co-ordinator AEDO Agricultural Extension Development Officer COVID Coronavirus disease (2019) DADO District Agricultural Development Office/Officer DAES Department of Agricultural Extension Services EPA Extension Planning Area FBO Farmer Based Organisation GAP Good Agricultural Practices GoM Government of Malawi Ha Hectare HH Household HHH Head of Household Kg(s) Kilogramme(s) KII Key Informant Interview LoP Life of Project LUANAR Lilongwe University of Agriculture and Natural Resources MBS Malawi Bureau of Standards MEL Monitoring, Evaluation and Learning MFI Micro-finance Institution MoAIWD Ministry of Agriculture, Irrigation and Water Development MoIT Ministry of Industry and Trade MSIKA Malawi Strengthening Inclusive Markets for Agriculture (Program) MSU Michigan State University mT Metric Tonne (1,000 kgs) MTE Mid-Term Evaluation NGO Non-governmental Organisation PHH (Harvest &) Post-Harvest (Storage) and Handling PMM Performance Measurement and Management PMP Performance Management Plan PPI Progress out of Poverty Index PPS Proportional to Population Sample SPV Special Purpose Vehicle TNS/PFS TechnoServe/Partners in Food Solutions T&T Techniques and Technologies – also known collectively as ‘practices’ ToR Terms of Reference USD United States Dollar ($) USDA United States Department for Agriculture V37 Land O’Lakes Venture 37 VSL(A) Village Savings and Loan (Association) MSIKA Final Evaluation Page viii kadale@africa-online.net Executive Summary The final evaluation (FE) of the Malawi Strengthening Inclusive Markets for Agriculture (MSIKA) was conducted from June to October 2020 by Kadale Consultants Ltd. Overview of the Project The MSIKA program was originally a five-year value-chain development program targeting 36,000 producers, 210 Farmer Based Organisations (FBOs) and 24 processors. It operated in five districts in Central and Southern Malawi (Dedza, Lilongwe, Mangochi, Mchinji, Ntcheu) and originally across seven crop value-chains (tomato, onion, potato, mango, citrus, guava and chili). MSIKA was implemented by Land O’Lakes Venture37 (‘V37’). MSIKA is a multi-faceted program with integrated components notably: training beneficiary producers in agricultural, post-harvest handling (PHH) and farm management practices, along with finance and marketing; strengthening FBOs; working with MFIs and banks to increase credit for producers and small and medium enterprises (SMEs); working with processors to enhance processing and obtain certification; working with Ministry of Agriculture on a new Horticulture Policy and training of district extension staff, establishing new product standards with Malawi Bureau of Standards (MBS) and researching improved growing practices. Following a reduction in available funds from monetization in 2019, the length of the programme was reduced to four years and the focus was narrowed to four crops (tomato, onion, potato and mango). The number of focal FBOs was reduced from 217 to 50. There was also a reduction in the number of demonstration plots from 85 to 15. Evaluation Approach The evaluation approached changed when it became clear that face to face interviews were not feasible due to COVID. The original household (HH) survey of producers was changed to a telephone survey with reduced content, but across more farmers (target of 1,060) to reach the desired confidence levels and margin of errors overall and for each crop. The sample frame for the survey was 30,922 producers trained in agricultural production as at May 31st, 2020. In addition, 52 key informant interviews (KIIs) were planned across the range of stakeholders from farmer based organisations (FBOs) to government extension staff to processors. In addition, the planned16 focus group discussions (FGDs) were changed to 60 phone qualitative interviews with individual producers. Data is reported against the Life of Project (LoP) targets and the impact baseline. Due to the reduced scope of the project and progress at the mid-term evaluation (MTE), the LoP targets were revised. As noted in the MTE, several baseline targets were recalculated to enable consistent treatment for outliers. The consultant conducted 1,074 interviews with 979 respondents (some were interviewed for two crops separately), 49 KIIs, and 63 qualitative interviews, all by remoted means. The producer sample exceeded the required confidence levels and margins of error. Overall Progress Across its many indicators, the MSIKA program achieved or made considerable progress towards many of its LoP targets as summarised in the tables in Annex 1). Several factors have impacted on the program in the last two years, in particular: 1. The life of the program was shortened from five to four years and several planned activities were dropped due to a shortfall in the expected resources. It meant that the full incremental effects of the activities, such as on yields and particularly sales, could not be realized due to the shorter period of implementation. 2. COVID began to affect Malawi towards the end of March 2020 and over the next two quarters (Apr-Sep 2020). The Government lockdown restricted movement and markets MSIKA Final Evaluation Page ix kadale@africa-online.net and closed tourism and hospitality venues and institutions that are major users of fruit and vegetables. This impacted on producer production and sales, and restricted the operation of FBOs, processors and the MSIKA team. 3. The Presidential and Parliamentary election in May 2019 was heavily disputed with disruption in the cities that limited activities by MSIKA and disrupted progress with governmental bodies. 4. In March 2019, Cyclone Idai hit Mozambique, with the effects reaching Malawi. The effects of Idai were seen in the MTE, impacting yields. While farmers could plant in a later season, it sapped their resources. The above factors are not intended to provide reasons for shortfalls in performance, which was good overall; rather they are a reminder that implementation can be disrupted by multiple factors beyond the control of the project team. For MSIKA, these have been multiple factors, some of them concurrently, such that more could have been achieved without them. The lesson for USDA and V37 is that while it is not possible to predict the specific nature and timing of disruptions, there can be consecutive or concurrent disruptions in a volatile operating environment like Malawi. This should be considered when planning what is realistic to deliver. Key Findings FBOs: MSIKA has worked with 217 FBOs across the five target districts, which is above the LoP target of 210. Of these, 167 (77%) were formed by MSIKA. For Year 4 of MSIKA, based on fewer resources and the MTE recommendation to focus on better performing FBOs, MSIKA planned to work with 50 better performing FBOs. This plan was disrupted by COVID, though the MSIKA team was able to partially implement this narrower focus. From the performance data and FBO KIIs, the consultant’s conclusion is that it is difficult to improve the performance of low performing FBOs and that working with these dilutes project efforts. The lesson for USDA and V37 is that similar projects are likely to make more progress and achieve better returns by moving early to focus on better performing FBOs. However, it is noted that determining performance can only be done following some engagement and time to assess performance. FBOs are useful mechanisms for engaging with producers and some may ultimately become sustainable organizations. However, the conclusion drawn from the findings is that it is unrealistic to expect that many have the potential to become sustainable entities. The lesson is to be realistic about the potential to turn most FBOs into well-functioning, viable and self￾sustaining registered entities. This is unlikely in most cases, at least based on the MSIKA FBOs. MSIKA made some progress on governance capacity building and in capacity development; however, using V37’s PMM assessment tool, suggests that it would take many years and considerable investment to develop them, with a very uncertain success rate. For projects like MSIKA, FBOs may best be seen as temporal mechanisms to engage efficiently and effectively with farmers than to build into a project’s design plans and resources to try to turn them into sustainable entities. If there are a few FBOs that have potential, then a more focused support package for the few would be the most realistic approach. MSIKA has been very successful at increasing individual producer yields and has made progress on PHH losses. Individual producers have generally increased their sales. However, progress with collective selling via the FBOs has been limited. There is potential interest in links between producers and two new processors, but this started late in the program and was disrupted by COVID, so did not reach fruition. Linking producers to markets is a strong theme for MSIKA and something that many projects that work with producers want to achieve to convert success in increasing production volumes (and quality) into sales and incomes. This aim is important to both V37 and USDA. MSIKA Final Evaluation Page x kadale@africa-online.net The lesson from MSIKA is that linking producers to more formal markets with expectations on quality, volumes, timeliness and ‘packaging/presentation’ is very difficult and may depend heavily on the ongoing facilitation efforts by a project. FBO and member expectations were that MSIKA would help them to find attractive formal markets with better prices, however, these expectations look unlikely to be met. Although this may be disappointing, producers have still found markets for their produce even under COVID, mainly because they have to sell and there are retail and wholesale markets they can reach, even if these are sub-optimal in demand and prices. With substantially increased production, these producers are still likely to be better off even with lower prices from these informal markets. Over time, markets may become more structured with the better organised FBOs finding ways to aggregate to supply more formal buyers. That process can be facilitated by a project, but it takes time and expectations of supply to formal buyers, such as processors and bigger retailers/hospitality outlets should be kept low. Producers: The FE found that MSIKA succeeded in reaching large numbers of producers (39,744) with training in improved agricultural and PHH practices that is valued, efficiently delivered and that is effective. The adoption rate of practices has been very high, with 99.5% of producers applying at least one new practice. As noted, COVID had an effect on the overall numbers and the target would have likely been over-achieved, if it had not occurred. The extent of application of practices was impressive, with a mean 15.8 practices per producer per crop. This widespread application is likely to lead to higher yields. With better weather conditions at FE, another year of training, and opportunity to apply practices under lead farmer guidance, and with the narrower focus on the four crops, the results on yield at FE have been very good with a 54.4% aggregate yield improvement against a target of 26.0%. At FE, the post- harvest handling (PHH) applications rates have improved further from the MTE level, with tomato scoring over 90% application on most practices. Mango, like all the tree crops in the MTE still lags the field crops, reflecting that tree crops are not seen by most producers as crops to actively manage and invest in. The overall effect was that PHH losses reduced from 14.8% at baseline to 13.1% at FE. Losses are difficult to measure and so caution is needed on any PHH loss data. Somewhat surprisingly given the disruption of COVID on markets, sales volumes increased substantially compared to the baseline, with the highest increase in mean sales volume for tomato (+70.0%), followed by potato (+28.8%), onion (+21.1%) and mango (+6.1%). The low increase in mango is not surprising, but the big increase in tomato seems to be a function of the increase in volume produced, meaning production has translated into increased sales. The sales value generated interesting results in comparison with the adjusted baseline. The sales of the four crops fall within a similar range with tomato at $1,229/ha followed by onion at $1,226/ha potato at $1,195/ha and mango at $996/ha. However, comparing these to the baseline there were big increases in sales value for mango (+99.7%), followed by potato (+92.7%), onion (+63.3%) and tomato (+44.1%). In summary, there has been good progress since the baseline and MTE in application of agricultural and PHH practices, with large increases in yields, sales volumes and sales value. There has been modest progress on PHH losses. MFIs/Banks/VSLAs: MSIKA continued with its SME lending mechanism with a Malawi Bank. The initial results at mid-term were positive overall. While there was hope for bigger and more flexible second loans, with the winding up of MSIKA and a Presidential declaration on a moratorium on SME loan repayments, the Bank made the terms much less attractive and deterred several SMEs that had been approved. In the end, the total number and value of loans disbursed through the facility was much lower than originally intended, highlighting the difficulties of establishing a successful SME loan facility. The lesson is that facilities for SME lending are not attractive to all banks/lenders and that banks have their own requirements that override what the project is seeking. MSIKA Final Evaluation Page xi kadale@africa-online.net MSIKA’s work with a microfinance institution (MFI) to increase agricultural lending has been very positive with both parties regarding the initiative as a success. The MFI has been the biggest source of lending to MSIKA participants ($413,100) compared to banks ($193,343) and VSLs ($74,592). The approach is low cost for MSIKA and proving to be very effective, as the MFI is using its own capital. The MFI has been encouraged by the high repayment rates from the well-organised and trained MSIKA beneficiary producers. It has seen that there is lower risk in horticulture due to the relatively short growing periods meaning there can be up to three cycles a year if there is access to irrigation. The MFI is a well-established MFI and will continue to offer these loans to these and potentially other MSIKA beneficiary producers after the end of MSIKA. This means the result is sustainable for the immediate future. The lesson for projects like MSIKA is that MFIs may be a more fruitful partnership than banks. MSIKA has established 751 VSL associations (VSLAs_, improving access to finance for over 10,000 beneficiary producers. VSLs are self-sustaining community managed mechanisms with a high degree of continuity. Although the value of loans through VSLAs is small compared to the MFI and even the bank, the number of loans is higher than these partners indicating that many producers (5,444) are accessing small loans in their localities and are able to get a share of the profits on the lending when the VSLAs distribute their funds. The lesson is that projects like MSIKA should continue to establish VSLAs and train members in VSL. Processors: MSIKA has established relationships with 18 processors. There is high uptake of good manufacturing practices, increases in storage, certification to Malawi Bureau of Standards (MBS) standards, improvements in processing, adoption of marketing and financial management practices. However, the outcomes of this work are relatively limited at FE, particularly for purchases from FBOs and additional employment. MSIKA believes that with a fifth year that it might have been able to increase the purchases from FBOs, and this might happen based on the relationships with new processors. In conclusion, development of processing in these value-chains is beneficial overall, however, there is not yet a strong connection between the processors and the producers and their FBOs. At the outset, MSIKA identified 71 processors as potential partners, but most were micro￾enterprises and the pool of larger and medium processors was limited to less than ten. It has proven difficult to make much headway with processors due to their relatively small size. The lesson is that MSIKA was better focusing on the five processors that have shown willingness and capability to make progress, while recognising that there is still limited potential with processors in Malawi at this point in time. Government: MSIKA engaged GoM on policy, training of extension staff, work with MBS and research on crop practices/farmer field schools. Updating the Ministry of Agriculture’s Horticulture Policy, moved quickly in the early stages due to the support from MSIKA, but slowed because the Ministry lacked the resources to progress the process. This was compounded by disruption from the election, as well as the major effects on government due to COVID. MSIKA has moved the policy process along and there is a good chance that the policy will be adopted in the future, but the lesson is that policy change of this nature is often very slow, even with strong commitment within government and prone to delays. Some AEDOs/AEDCs have been involved and assisting on delivery of producer training, but the GoM extension staff are not a key driver in onward training compared to the lead farmers. The lesson is that projects like MSIKA have to include GoM staff, but should focus efforts on training and supporting lead farmers as the most sustainable, effective and efficient mechanism to reach producers. MSIKA worked with MBS to develop 10 new product standards. Despite some delay, the 10 new standards have been adopted. The lesson is that working on regulations or supporting measures may yield more results than working on national policies and strategies, and may be of more immediate relevance to the private sector. MSIKA Final Evaluation Page xii kadale@africa-online.net MSIKA’s engagement with MSU/Bunda on research was linked to the farmer field schools at the demonstration sites. Getting the most out of research requires maximising demonstration sites for the farmer field schools. It is unclear at this point how much MSIKA will be able to leverage this research to bring forward practices that can be readily adopted by many producers. The lesson is that research may not be able to bring results in the project period and it is better to focus on demonstrating under-utilized knowledge on proven practices. The following recommendations are made: #1 That to undertake a phone survey of producers requires access to a sufficient database of phone numbers, as representative enough of the population as possible, and may require additional efforts to get female producers included. Considerable thought should be given to question inclusion, such as those that may rely heavily on face to face interaction with respondents. #2 That in the design of projects of this type, USDA should consider in its expectations and ambitions that there is a high likelihood of consecutive or concurrent disruptive events. While these can be listed as risks and assumptions, the flexible response shown by USDA on MSIKA can avoid projects scrambling to make up ground and hit pre-determined numbers. #3 That projects like MSIKA should aim to identify and select out low performing groups and FBOs at an early stage using appropriate tools and drawing on experience. #4 That projects like MSIKA should not aim to turn FBOs into sustainable entities in most cases, but be highly selective in deciding which might have potential, and to see most FBOs as temporal entities for efficient and effective project delivery that may or may not continue post project. #5 That projects like MSIKA should seek to facilitate sales between better organised FBOs only where there is a realistic capacity to supply and members are committed to collective selling and that there are accessible markets. Where that is not possible, there is still merit in working to increase production for sale by individual producers into less formal markets. #6 That because Banks are regulated and relatively cautious, projects like MSIKA need to consider other more flexible and responsive lenders, such as well-established MFIs. #7 That VSLs should continue to be a focus for projects like MSIKA as they provide ready access to finance and over time, the VSL builds trust and cohesion between producers. #8 That projects like MSIKA should find an appropriate balance between working with enough processors, but of sufficient size and capacity to invest. Working with many micro- and small￾enterprises is resource intensive with limited potential to deliver results. Work with smaller enterprises should be limited to training and general information rather than more intensive support relationships. #9 That projects like MSIKA embarking on support for developing government policies and strategies need sufficient time and resource to see such processes through to the end and that it should be accepted at the outset that there is a risk that these processes may not yield results within the project life or at all. #10 That projects like MSIKA should focus on lead farmers as a more sustainable, effective and efficient mechanism for delivering services, particularly knowledge, than government extension staff. However, government structures should be engaged and government staff included in training and activities where they can support of lead farmers. MSIKA Final Evaluation Page 1 kadale@africa-online.net 1 Introduction and Background This report is the independent Final Evaluation of the Malawi Strengthening Inclusive Markets for Agriculture (MSIKA) undertaken by Kadale Consultants Ltd (‘Kadale’) from mid-June through to the end of October 2020. This evaluation has been conducted for the United States Department of Agriculture (USDA), with the active co-operation of Land O’Lakes Venture 37 (V37), which is implementing the MSIKA program. This section sets out the objectives of the final evaluation, an important note on the impact of COVID on the evaluation design and implementation, a brief background about MSIKA and some of the key changes to the project, notably following the mid￾term evaluation (MTE) in late 2019 and the advent of COVID from March 2020. 1.1 Objectives of the Final Evaluation The objectives of the final evaluation are as follows: 1. Assess the relevance of the project strategy and approach to project participants; 2. Measure progress the project has made toward key results, including effectiveness and efficiency of interventions in achieving established targets; 3. Identify enablers and constraints to progress (both internal and external factors) that have supported or limited success of the project; 4. Assess the sustainability of project outcomes, and the effectiveness of sustainability efforts that were undertaken by the project; 5. Assess operational aspects of the project, such as project management and monitoring and evaluation; 6. Document lessons learned, challenges and unanticipated effects; 7. Provide recommendations for best practices that were realized or areas of improvement for future USDA and Venture37 program in similar areas and value chains. 1.2 COVID Impact on Evaluation Design and Implementation During the implementation of the final evaluation, the impact of COVID on Malawi necessitated restrictions on movement, making it impossible to undertake face to face interviews. As a result, the methodology and instruments had to be redesigned to be based on a phone-based household (HH) survey of producers. The planned focus group discussions (FGDs) with producers were replaced by phone based qualitative interviews and all the other planned KIIs were also to be conducted by phone. This approach created new challenges, a major one was determining the phone numbers for producers, which took the MSIKA team considerable time and effort to gather from the FBOs. The implications were a delay in the original timescales for delivery, and removing questions in the data collection tools to limit the time for the interviews. This meant that the final evaluation is not be able to report on three of the project indicators. USDA provided approval for this change. This change was not ideal, but was necessary to make the data gathering manageable for the respondents. Although this was disruptive to the FE research process, the consultants believe that the results are still valid and robust. This has been an opportunity for the consultants to learn from taking a different approach in what has been a very challenging operating period. MSIKA Final Evaluation Page 2 kadale@africa-online.net 1.3 Background to MSIKA Project MSIKA was originally a five-year, $17.0 million project implemented by Land O’Lakes Venture 37 (V37), with funding from the United States Department of Agriculture’s (USDA) Food for Progress Program. MSIKA was implemented in partnership with TechnoServe and Michigan State University (MSU). The goal of MSIKA was to improve the commercialization of fruit and vegetable sectors for value addition, and ultimately increase domestic and regional trade. MSIKA had two strategic objectives, in line with USDA Food for Progress Results Framework. The first was to increase agricultural productivity in the fruit and vegetable sector by: increasing the availability of improved inputs, improving infrastructure to support on￾farm production, facilitating access to finance, and training farmers on improved agricultural and farm management techniques and technologies. The second was to expand domestic and then regional trade of fruit and vegetable products by: expanding access to finance; improving quality of post-production products, training producers and processors on improved post-production processes, facilitating improved linkages between buyers and sellers, improving market and trade infrastructure, and facilitating improved management of buyer/seller groups. Please see Annex 2 for MSIKA results frameworks. MSIKA’s theory of change involved strengthening the capacity and linkages between farmers, processors and input providers, such as financial institutions, extension services, research institutions, and agribusinesses to develop the target horticulture crop value chains. MSIKA built the capacity of processors to meet quality standards and improve their efficiency, and linked them to financial institutions and horticulture producers to be able to expand their production. MSIKA created a special purpose vehicle (SPV) with a Malawian bank to provide low costs loans to processors. MSIKA also worked with the Government of Malawi (GOM) to create and codify quality standards for horticulture to promote compliance amongst processors and improve reputation of Malawi products. With increased access to finance and raw materials, and improved quality and processes, processors were able to increase their production and sales. MSIKA also worked to create and strengthen horticulture Farmer Based Organizations (FBO) to provide an access point to engage and train tens of thousands of producers through lead farmer and extension staff training of trainers. MSIKA engaged with the research community to test improved horticulture varieties and techniques through demonstration plots to create the curricula for this training. MSIKA also linked producers to local agribusinesses to secure improved inputs, microfinance and formed village savings and loans associations to improve their access to finance. Improved knowledge, access to inputs, and access to finance to purchase the inputs, producers improved their practices and ultimately productivity of the target crops. MSIKA also linked the FBOs to market so that they could sell their additional product and ultimately increase their sales. There were eight specific areas of activity planned (see Scope of Work – Annex 3): 1. Training: In improved agricultural production techniques Provision of training, technical resources, and cutting-edge research to fruit and vegetable farmers to improve technical and business skills. Build capacity of lead farmers and extension agents from the Ministry of Agriculture, Irrigation and Water Development(MoAIWD). Build capacity through Farmer Field Business Schools (FFBS) and demonstration plots (Yankho PlotsTM) on topics such as orchard management, budding and grafting, nursery establishment, pruning, and weeding. Ensure that FFBS and Yankho PlotsTM build core climate smart agronomic and business skills of target producers, in partnership with MSU. Engage private sector partners to ensure increased MSIKA Final Evaluation Page 3 kadale@africa-online.net utilization of irrigation technologies and increased access to improved inputs. E ngage private sector partners to co-invest in Yankho PlotsTM and to establish for-profit nursery enterprises. 2. Infrastructure: Post-harvest handling and storage Improve PHH and storage infrastructure by providing training and technical assistance, facilitate market linkages, and support access to finance to improve on-farm and off-farm post-harvest infrastructure and to increase the use of improved post-production processing and handling practices. Facilitate increased adoption of established standards, such as Global Good Agricultural Practice (Global GAP), by market actors, such as traders, processors, and producers. Facilitate Farmer-Based Organizations (FBO) and private enterprises to develop wholesale markets, and aggregation centers, and to utilize technology and practices such as optimal packing technology, improved storage practices, and sorting, grading, and washing technology. 3. Training: Post-harvest processing Improve the efficiency and profitability of value-added post-harvest manufacturing and promote the utilization of value-preserving practices and technologies throughout the fruit and vegetable value chains, in partnership with Technoserve. Provide targeted technical assistance to small, medium- and large-scale fruit and vegetable processors. Technical assistance will focus on processing line operational efficiency, improved packaging and labelling, modernization of physical equipment and plant facilities, and improved business and financial management practices. Train fruit and vegetable processors on improved product quality standards and facilitate Malawi Bureau of Standards (MBS) certification. Facilitate FBO access to improved PHH techniques. Facilitate improved phytosanitary practices by agricultural enterprises and FBOs engaged in small-scale processing of fruits and vegetables into products. 4. Capacity Building: Producer groups and cooperatives Apply cooperative development tools, including V37’s AgPrO products, to improve farm management and to increase the capacity of FBOs in agricultural production, post￾harvest handling, and processing. Link FBOs to business development service providers and finance providers, and collaborate with national-level producer associations and district-level FBOs (such as the Mchinji Horticulture Association) to strengthen the delivery of member services and facilitate the transformation of FBOs into agricultural enterprises. 5. Market Access: Facilitate buyer-seller relationships Improve market access by facilitating buyer-seller linkages, increasing access to market information, and strengthening the capacity of key organizations in the trade sector. Facilitate buyer-seller networking events, linkages to transporters the establishment and strengthening of equitable outgrower schemes between producers and processor, stronger linkages to retail supermarkets for standards and specification trainings and procurement, promotion of village-based input sales agents, and targeted support for women- and youth-owned small and medium enterprises (SMEs) engaged in food processing. 6. Financial Services: Facilitate agricultural lending Build the capacity of a local financial institution to conduct risk assessments and develop financial products for value addition in the fruit and vegetable sectors, such as working capital and trade finance for processing, aggregation, irrigation, export, and transportation. Facilitate increased lending for climate smart irrigation; work with MSIKA Final Evaluation Page 4 kadale@africa-online.net financial institutions to develop loan products appropriate for horticulture value-adding activities; facilitate the development of input credit relationships between processors and producers or FBOs; improve the financial literacy and credit- worthiness of FBOs; and facilitate the development of capital investment plans by fruit and vegetable processors for plant modernization and expansion, including improvement of business and financial managementsystems. 7. Financial Services: Provide SME finance Stimulate increased agricultural lending by establishing a Special Purpose Vehicle (SPV). The SPV will make loans to financial institutions that will on-lend those funds to enterprises, farmers and farmer organizations. The SPV may catalyze further investment in value-adding activities and trade, such as loans from the Overseas Private Investment Corporation (OPIC). Ensure that loans originating from the SPV, prior to the award’s date of completion, will be used for purposes such as increasing access to working capital for farmers and SMEs for input purchases, transport for space arbitrage of fruits and vegetables, market and trade infrastructure, processing equipment, and equipment upgrades. After the award’s date of completion, ensure that loans originating from the SPV will be used to expand agricultural production and trade in African Least Developed Countries, as defined by the World Bank. For the life of the SPV, principal and interest repaid to the SPV for continued lending and investment. 8. Government Capacity Building: Improve Enabling Environment Build the capacity of key government research and extension agencies, including MoAIWD and the Department of Agricultural Extension Services (DAES). Work closely with the Lilongwe University of Agriculture and Natural Resources (LUANAR), in partnership with MSU, to conduct research on soil fertility and other value chain specific trials. Venture37 will build the capacity of processors, traders, national associations, and FBOs to advocate for improved regulations and policies. Work with the MBS to jointly establish and disseminate international quality standards. Conduct a gap analysis and recommend improvements to bring MBS’s standards, certification processes, and quality audit processes up to international levels. The key targets were to improve yields and sales through market linkages for 36,000 farmers through 210 FBOs across five districts. 1.4 Key Changes to MSIKA In the course of implementation, there was a budget reduction to $12.2 Million due to a shortfall in expected funded through monetization, which necessitated re-strategizing and modification of the approach. It was decided that MSIKA would come to an end at the end of Year 4, a year earlier than the original plan. Following the recommendations of the Mid-Term Evaluation (MTE) and related to the budget reduction, there was a de￾prioritization of guava, chili and citrus; increased focus on continuing activities 1, 4, 5 and 7; scaling down of activity 6 to focus on the MFI and Village Savings and Loan Associations (VSLAs) only; and wrapping up of activities 2, 3 and 8 in Q1 of Year 4. The MSIKA team also scaled down the number of participants worked with in Year 4, focusing on 50 FBOs representing around 8,000 farmers, and scaling down the farmer field schools and demonstration plots from 85 to 15. The modification resulted in revised indicator targets to reflect the changed focus and resources and these are the indicators against which MSIKA is evaluated in this final evaluation. Making such major changes part way through a program brings challenges. These were compounded by the advent of COVID-19 (COVID), which began to impact on MSIKA Final Evaluation Page 5 kadale@africa-online.net Malawi from mid-March 2020, at the end of the project’s Year 4 Q2 and then through to the end of the project. From the end of March 2020, the Government of Malawi (GoM) introduced measures to reduce the spread of COVID, primarily restrictions on movement within Malawi particularly from the cities to the districts, limits on the size of meetings/gatherings and closing of institutions, many of which were the main markets for fruit and vegetable producers. A further consequence was the restricted movement of project staff, hence most of the project activities from March onwards were coordinated remotely. The need to keep project staff and participants safe took precedence. This presented MSIKA with further considerable operational challenges, and resulted in further changes to the revised plan of activities. This provides the context against which this final evaluation took place. 2 Methodology and Implementation This section sets out the methodology and how the final evaluation was conducted. 2.1 Methodology 2.1.1 Overview A key factor for the final evaluation (FE) was the effect of COVID on the original plans. With the arrival of COVID in Malawi, it became necessary to move from the planned face to face household (HH) surveys, focus group discussions (FGDs) and key informant interviews (KIIs) to a revised approach based around remote interviewing (by phone, mainly). This also changed the sample size and selection. These points are covered in the respective sections below. Overall, and as planned, this FE adopted a mixed method, non-experimental approach, combining quantitative data from a household survey of participant farmers, with qualitative interviews (replacing the original intended FGDs) and KIIs with the MSIKA team and a wide range of stakeholders. The stakeholder groups interviewed were: high-, medium- and low-performing Farmer Based Organisations (FBOs); processors and input suppliers small and medium enterprises (SMEs); GoM staff in the districts; and financial institutions. The consultants also reviewed MSIKA documents and monitoring data, the impact assessment baseline report and data and the mid-term evaluation (MTE) report and data. 1 For the quantitative work, the team adopted a non-experimental, pre-post design to measure the key performance indicators. The pre-intervention data came from the impact baseline study that interviewed a random sample of expected MSIKA participants, as well as producers outside the target areas as a control group2 . The consultant compared the expected MSIKA participants interviewed at baseline with a random sample of actual MSIKA participants randomly selected from a database of all participants provided from the project. The quantitative design compares estimates of the two independent samples of the MSIKA at baseline and at FE. It was agreed with USDA and Venture37 that the consultants also should look at the effects of the COVID-19 pandemic that had reached Malawi and which led to measures by GoM to limit movement and business. Questions around the impact of COVID replaced the questions around the impact of Cyclone Idai that were asked in the MTE. 1 Kadale conducted the MTE and provided the data collection to TANGO who undertook the baseline. 2 The initial evaluation plan called for a quasi-experimental design, where the final evaluation would have followed up with both participants and a control group, however, due to the budget shortfall, the design was updated to a non-experimental design. MSIKA Final Evaluation Page 6 kadale@africa-online.net 2.1.2 Household Survey The key activities for the design and delivery of the FE HH survey were: Tool Development The consultants conducted KIIs with MSIKA staff to discuss activities implemented by the project and how these should be captured through the FE HH survey. The consultants revised the HH survey used for the MTE, which was based on the baseline instrument. so that accurate comparisons could be made. The main changes in the survey were 1) the removal of questions related to the three crops (citrus, guava and chilli) that were deprioritized in Year 4; 2) the removal of questions on knowledge of practices and hectares per practice, and asking about only one crop per interview to reduce the length of the survey; 3) updating the practices to focus on those in the MSIKA training packages. Note that not all practices in the baseline were in the MSIKA training package, and there are practices that are asked about in the FE that are not in the IB. It is not surprising that there were changes, as a baseline is anticipating what will be done in the future. The consultants sought to maintain as much comparability as possible with the IB and with MSIKA’s own semi-annual surveys. Out of the four focal crops in year four, the three field crops (tomato, onion, Irish potato) have up to three growing/harvesting cycles per year depending on the producer and their circumstances. For the remaining prioritized tree crop (mango) there is only one growing/harvesting cycle. The HH survey instrument had already been expanded during the mid-term evaluation to capture land area, yields, losses and sales for all growing cycles for each of the field crops. The team translated the instrument into Chichewa and coded it into Open Data Kit (ODK) for tablet-based data collection. The instrument was piloted through telephone interviews to check the flow of the questions and to determine whether the interviewing time would be appropriate. The instrument was further refined based on the results from the piloting. MSIKA staff were fully consulted on the technical aspects of the instrument and provided inputs at each stage. Overall, the consultants’ view is that there is good alignment between the FE and baseline instruments and data. The final instrument is included in Annex 4. Training Enumerators were trained for three days from 24-26th August 2020. This included a telephone-based interviewing practice and on the final afternoon. The training consisted of the background to MSIKA, running through the survey section by section to clarify questions and responses, mini-quizzes and practicing questions and responses. An agronomist provided technical explanations for all the practices so that enumerators would understand the agronomy of the crops. There were sessions on use of ODK and tablet collection, conduct during the phone interview, sampling, substitution protocol, quality control/checking procedures and data uploading. The telephone based interviewing practice tested that the enumerators could administer the instrument and record data on the tablets. It also tested whether the interviewing time was within the planned 45 minutes and if there were any issues with telephone￾based interviewing. At the end of the training, Kadale selected 14 of 16 trainees for two teams of one supervisor and six enumerators. Sampling MSIKA shared an initial database with 38,384 project participants. The MSIKA team and the consultants discussed the sample frame for the producer HH survey at the inception meeting and decided that it should be all producers trained in agricultural production as of May 31st, 2020. This was confirmed by the MSIKA team to be 30,922 MSIKA Final Evaluation Page 7 kadale@africa-online.net participant producers. The MSIKA team indicated that the difference between the two totals is accounted for by some producers that have received other forms of training, such as financial literacy, marketing and quality standards, plus non-farmer participant groups that were trained, such as project staff, government extension staff and other stakeholders. The sample for the producer HH survey was therefore based on the 30,922 farmers. MSIKA requested that the sample of producers represent the sample frame with 95% confidence overall and be representative at the value chain level with 90% confidence. Due to the restructuring of the evaluation from field based to telephone interviewing, which resulted in conducting an interview for one crop per farmer, the focus of the sample size was directed towards achieving the required minimum number of interviews for each crop, as that meant the overall sample would exceed the required 95% confidence level. Across the crops, it was agreed that the sampling approach for each crop would be calculated at 90% confidence level and 5% margin of error (MoE) according to the population of the farmers growing that particular crop. The sample was not designed to be able to report results by districts, but by crop as significant differences between districts were not expected. Getting a broadly proportional district split was to ensure that the sample was indicative of the districts that MSIKA works in. The consultants used a two-stage approach to sampling. First, Kadale randomly selected two FBOs (with a minimum of 12 members) per EPA. If an EPA only had two FBOs, they were automatically selected. Second, Kadale randomly sampled 12 participants from each FBOs using a random number generator. Kadale oversampled individuals from FBOs to make sure there would be back-ups if the originally selected participant could not be reached. The maximum number of interviews to be conducted per FBO was to ensure that the achieved sample would not be skewed more towards a few FBOs or EPAs. With the change from face-to-face to phone interviews, the consultant needed to obtain phone numbers for potential respondents. The MSIKA team only had phone numbers for about 1/3 of the participants in their database. To reduce the bias that this might call, MSIKA made concerted effort to get phone numbers for more participants. Kadale used the above method to draw a short list of 8,000 participants. The 8,000 participants were drawn from a population of 30,922 farmers that had undergone basic agricultural training as of 31st of May 2020 (as shared on 2nd July 2020). MSIKA staff worked with the FBOs to get numbers for as many of those participants as possible. In the end, MSIKA was able to get phone numbers for 2,973 participants from the list of 8,000 listed participants. This became the final list of participants to attempt to interview.3 Lastly, Kadale needed a method to select which crop to ask each participant. During the actual data collection, supervisors were given printed random crop selection matrices which clearly elaborated on the crop each farmer was to be interviewed for depending on the crops the farmer had mentioned to have grown and harvested within the specific time period of interest. During each successful call, the enumerators asked the farmers to list the crops that s/he had grown. Once these had been listed, the enumerators randomly selected which crop to ask about based on the matrix. All farmers that only grew one of the focus crops were interviewed on that crop and this was considered as a random selection. To ensure an appropriate sample size for each of the crops. Kadale used a quota method. Once the random selection method resulted in enough interviews on a crop, that crop was removed from the random selection list. In summary, for each district, the sampling approach involved identification of EPAs, selection of FBOs in each EPA, selection of farmers per FBOs, identification of crops 3 It was expected that some of the numbers would be unreachable. MSIKA Final Evaluation Page 8 kadale@africa-online.net grown by each farmer in the selected FBO, random selection of one crop from the stated crops and interviewing. Sample size The sample size calculation formula for the specific crops was: n = N*X / (X + N – 1), where X = Zα/22 *p*(1-p) / MOE2 Through this sample size determination formula, the minimum sample at 90% confidence level and 5% MoE for tomato, onion, Irish potato and mango was calculated at 268, 264, 266 and 262 respondents respectively. This was adjusted to a target of 276, 272, 274 and 270 for tomato, onion, Irish potato and mango farmer interviews respectively, to allow for responses that might be unusable/incomplete. The interviews were split by crop based on the explanation above, and set out in Table 1 below. Survey Implementation The survey was implemented over a 17-day period (27th August to 17th September 2020), which is similar timing to the IB. MSIKA staff notified village leaders and FBO leaders during the phone number collection period to enable participants to be mad aware that they could be called, but were otherwise not involved in the survey. All the participants for whom there was a phone number were called. Many were not reachable, despite at least three attempts to reach each participant. This was problematic, especially in Mangochi District partly due to network problems and that most participants in the district were registered collectively under a phone number for someone else, so if a number could not be reached, it meant all those listed for that that number could not be reached. Similarly, for those whose number could be reached, the owner of the phone sometimes had challenges finding all the individuals which had registered under that number. In total after calling all numbers that had been listed, 979 participants were reached and interviewed. This number of respondents was sufficient for the overall sample to reach the desired confidence level and MoE, however it was insufficient to reach the target level for each crop. Therefore, h 95 individuals, who reported a second crop of the type which had a shortfall, were interviewed twice on different crops. This resulted in 1,074 crop interviews in total from 979 respondents. Table 1 below sets out the planned interviews against the achieved interviews per crop. Due to the above issues, the field time was extended to complete the work across the value chains, whilst also ensuring that each district had a reasonable number of interviews. Overall, the sample is valid and robust. Table 1: Minimum and Actual Responses by Crop Interviews per crop Crop Minimum Actual Difference Tomato 268 276 8 Onion 264 264 - Irish Potato 266 266 - Mango 262 268 6 Total 1060 1074 14 MSIKA Final Evaluation Page 9 kadale@africa-online.net Each crop’s planned target was met at the 90% confidence level and 5% MoE. The actual sample by district is set out in Table 2 below. Table 2: Usable Responses by Crop and by District District Tomato Onion Irish Potato Mango Total: all crops Dedza 41 32 76 29 178 Lilongwe 84 58 43 35 220 Mchinji 56 82 79 68 285 Ntcheu 53 53 51 91 248 Mangochi 42 39 17 45 143 Male Producers 158 149 145 129 581 Female Producers 118 115 121 139 493 Total 276 264 266 268 1074 Min sample 90% confidence, 5% MoE 268 264 266 262 1060 Min sample 90% confidence, 5% MoE 68 68 68 68 272 Data Cleaning, Analysis and Comparison with the Baseline The data was cleaned and tabulated to provide a first-cut analysis. The IB sample was drawn from registered farmers who were expected to become MSIKA participants, while the FE survey was drawn from actual participants who had at least undergone basic agricultural training, so some differences were expected. Kadale tested the comparability through a regression analysis based on key characteristics, such as land size, education level, and household size. The analysis showed that there was an acceptable level of comparability between the samples. The profiles of the baseline and FE samples are compared in the findings (section 3.2.1). When the consultants reviewed the Impact baseline analysis files in the MTE, it identified that the baseline analysis had removed high-end outliers for tomato, but not for the other six crops. To ensure comparability and to determine the level for outliers, the consultants used the Food and Agriculture Organisation’s (FAO) annual maximum crop yields based on the application of Good Agricultural Practices (GAP) in kilograms/hectare (kgs/ha)4 for vegetables and kilograms/tree for fruits. These maxima were applied, involving the removal of outliers above the specified level and resulting in re-calculating the baseline. Similarly, during the FE, Kadale applied the same maxima as those of the baseline to ensure comparability of the findings and to ensure that only realistic yields were taken into account during analysis. The consultants also noted that the baseline asked only for the “most recent harvest” cycle, not all cycles for the previous 12 months. Although the Word version of the baseline instrument referred to the previous 12 months, the final Excel/ODK version referred to the most recent harvest. It is the latter that was implemented, though it is not clear how the change occurred. As a result, the consultants make comparisons between the baseline and FE samples based only on the most recent growing/harvesting cycle. For the FE, all knowledge related questions were taken out so as to shorten the interviewing time. Therefore, information to be derived from knowledge related questions will make reference to the findings of the MTE. In the MTE, the consultants noted that there were price outliers in the baseline that had not been removed. These had resulted in some high sales values, beyond what 4 These are set per crop in the context of Malawi as a maximum yield in kgs/ha or kgs/tree. MSIKA Final Evaluation Page 10 kadale@africa-online.net is realistic for these types of households. The consultants adjusted the baseline values in the MTE to remove these, so that comparisons are made with this adjusted baseline. There is a considerable amount of data in the dataset. Kadale has sought to set out in the findings (section 3) what it thinks is most useful, particularly focusing on Life of Project (LoP) target achievement for project indicators. 2.1.3 Farmer Qualitative Interviews The original plan was that the Kadale team would facilitate FGDs with participant producers to collect qualitative information to support the quantitative survey and project data. However, due to the restructuring of the FE from field to remote interviewing, it was agreed with the MSIKA team to replace these with 60 qualitative KIIs of beneficiary producers growing the four target crops in Year 4. Kadale, with inputs from the MSIKA team, developed the guide for the qualitative farmer interviews derived from the MTE FGD guide shortened to be suitable for a phone interview. The guide is included in Annex 4 In each district, 12 farmers were to be sampled, ideally with a split of six males and six females, and three interviews per crop. This was to ensure that the responses would cover each district, sex and crop. These farmers were sampled purposively. The farmer qualitative interviews were conducted by members of the consultant team with qualitative experience. Despite considerable challenges finding mango farmers who had received training on tree-crop techniques and technologies, the consultants managed to conduct 63 KIIs which surpassed the planned interviews by three. Overall, the consultant was satisfied with the quality of discussion in the KIIs and were able to gather the diversity of views and qualitative inputs that were necessary. Table 3: KIIs Conducted by Crop, District and Sex Crop Sex District Dedza Lilongwe Mchinji Ntheu Mangochi Total Tomato Male 1 3 2 2 2 10 Female 2 1 2 1 2 8 Onion Male 2 1 2 1 2 8 Female 1 2 1 2 1 7 Irish Potato Male 2 1 2 1 2 8 Female 1 2 1 2 1 7 Mango Male 1 2 1 3 1 8 Female 2 1 2 1 1 7 Overall Male 6 7 7 7 7 34 Female 6 6 6 6 5 29 Total 12 13 13 13 12 63 The consultants analysed the interviews by themes. This was done by summarizing the information provided by the farmers using a separate matrix template for each crop. Once the write-ups or summaries were finalized, a separate column was added to further summarize the common themes that emerged under each question. 2.1.4 Key Informant Interviews The consultants conducted key informant interviews (KIIs) with stakeholders to understand their engagement with MSIKA, obtain feedback on the implementation, and determine the outcomes that they have seen or achieved resulting from the program. MSIKA Final Evaluation Page 11 kadale@africa-online.net KIIs were planned with the following key stakeholder groups: 1. MSIKA monitoring and evaluation staff, technical staff and management; 2. FBOs: one high-, medium- and low-performing FBO per district; 3. Processors and business partners that have been supported; 4. Agricultural Extension Development Coordinators (AEDCs) and Agricultural Extension Development Officers (AEDOs) in the target districts; 5. Financial institutions that have partnered with MSIKA; 6. Govt. Ministry Headquarters and Research Scientists; and 7. USDA. The consultants developed different KII guides for the discussion with each type of key stakeholder. These were tailored to the specific details of the stakeholders as required, but shortened so as to conduct these remotely. See Annex 4 for the KII instruments. The KIIs were conducted by the team leader and the central team members, as most appropriate based on logistical and language reasons. The consultants were able to conduct most of the intended KIIs. The exceptions were 1) one processor who kept changing appointments, despite making multiple attempts and specific efforts to contact them; 2) the Ministry of Agriculture HQ as the information on policy progress was obtained from the MSIKA team and 3) Bunda academics, as the research team’s assessment was that the report on the research was sufficient. In the latter cases, the consultants drew on interviews conducted with those organisations in the mid-term. The KIIs with the FBOs and processors were not intended to be representative but to gather qualitative insights and validate monitoring data collected by MSIKA. Table 4: KIIs Planned and Conducted by Category Organization Planned Actual Comment FBO Comments 15 15 All 5 districts over the phone Ministry of Agriculture, District-based 10 11 All 5 districts over the phone Ministry of Agriculture, HQ 1 0 Not completed - a sufficient progress update obtained from MSIKA staff LUANAR, Research scientists 1 0 Not completed - a sufficient progress update obtained from MSIKA staff Financial institutions 2 1 MFI completed as the priority Malawi Bureau of Standards 1 0 Unable to complete Processors 12 11 One processor kept changing the meeting time, while another provided part information Other SMEs 2 3 Input suppliers MSIKA M&E team 1 1 Multiple engagements MSIKA management 2 2 Multiple engagements MSIKA technical team 4 4 Single and multiple engagements USDA 1 1 Completed prior to fieldwork commencement Total 52 49 2.1.5 Review of MSIKA Documents and Data The MSIKA team provided the consultants with key documents and data at the start of the evaluation. These included • MSIKA program description (original and updated) MSIKA Final Evaluation Page 12 kadale@africa-online.net • Evaluation plan (original and updated) • Performance monitoring plan (PMP) (original and updated) • Internal Monitoring Evaluation and Learning (MEL) Plan (original and updated) • Lists of participants and key stakeholders for the sample frames • List of FBOs and training conducted at FBOs • Semi-annual reports to USDA • Semi-annual data collection tools and data • MSIKA annual workplans • the baseline report, database and instruments • FBO capacity assessment SOW, tools and data The evaluation team reviewed these documents and data to inform the evaluation. Insights from this information that support the evaluation’s primary data collection are reported in the findings section. The role of this documentation and data was to inform the development of the instruments and the sampling frame, as well as to provide information on other results not directly measured by the survey, farmer qualitative interviews and KIIs. Throughout the process, clarifications were requested by the consultants and were addressed by the MSIKA team. 2.2 Limitations This was an unusual evaluation process because the COVID pandemic led to the introduction of restrictions on movement, meetings and operation of institutions that affected the data collection. The research was substantially redesigned part way through, with the major changes being a shift to a telephone survey for participant producers and other KIIs and converting FGDs into qualitative interviews. This contributed to the following limitations: 1. Reduced information on certain topics The consultants and the MSIKA team had to adapt to a phone-based interview by revising all tools, primarily to shorten them after having prepared them based on face to face. There had to be several revisions to reduce the length to be manageable in a phone interview, particularly with participant producers. The result was that the questionnaire only tackled one crop for each participant, and sections on knowledge, gender and finance were omitted, while others were reduced in scope. For some of the topics omitted or compressed, the ultimate effect is limited. For example, knowledge is a stepping-stone to application/use, so as long as application and use is measured, there is a reasonable conclusion that producers would ‘know’ the practice concerned. It would be useful to have the knowledge measured, but inferences can be drawn from the MTE results and the qualitative producer interviews. For some topics, the missing information would be helpful, like gender and access to finance, but these are not measured by specific indicators or the information can be drawn from other sources. Overall, the consultants felt that there was sufficient information, including drawing on MSIKA data and the recent MTE, to inform this final evaluation. 2. Potential producer survey bias For a phone-based survey, potential respondents need to have a cell-phone or access to one, perhaps shared with others. Ownership of a phone might have biased the research, perhaps through respondents being better off, and possibly with a male bias. The exercise to collect phone numbers was challenging, with numbers obtained for 37% of respondents. The consultants sought to remove any bias within this group by MSIKA Final Evaluation Page 13 kadale@africa-online.net ensuring a sampling protocol that enabled them to work through all the FBOs that were in the short list. Ultimately, all those with a number were called, with around 36% of them being reached. Reasons for non-response were rarely refusals, but that the numbers were not reachable. The profile of the FE sample was comparable with the baseline, though there is over￾representation of male respondents relative to the sample frame. From the MTE, farming was commonly a joint household activity that typically involves both men and women in production, even if the amount of input varies, and that men tend to be more prominent in sales and use of the income. The over representation of men (54% in sample vs 45% in the population) probably reflects the ownership and control of the cell phone. Overall, this difference in male:female representation was significant and is not ideal, but is not regarded as substantively affecting the results, partly because men and women tend to operate jointly (see MTE findings on gender). There were difficulties getting respondents from Mangochi District and for the mango value chain, requiring a degree of purposive contacting/contact chasing. In the end, the consultants managed to get sufficient responses for each crop, but there were fewer responses from Mangochi compared to the plan. Overall, this may have introduced some minor limitations on the mango and Mangochi data compared to other crops and districts. 3. Challenges of producers estimates Producers were asked about the size of new and refurbished storage units installed in the last 12 months. Following analysis of this data, it became clear to the consultants that there was likely to have been an over-estimation of the size of the storage by producer participants. Achievement was much higher than the trend from MSIKA reports and the MTE. Steps were taken to address this, but the final data still appears to be much higher than is likely, though it is expected that the indicator was genuinely exceeded. This exercise highlights that there can be difficulties in getting valid responses for certain types of questions, such as measurement/estimations through a phone interview. If the enumerators had been face-to-face with the respondents, they could have viewed the storage that was built or paced out the floor area and asked the respondent to show the height visually. This would have reduced the chances of over￾estimation. The consultants’ conclusion is that this did affect the responses on storage and possibly some other questions, such as land area, where it might have helped to have face to face time to clarify responses. Overall, the results, other than storage area, were not substantially different than could be expected. The lesson is that some topics, e.g. that require measurement, may be unreliable in this type of interviewing and that some type of further cross-checking at the interview stage should be considered. MSIKA Final Evaluation Page 14 kadale@africa-online.net 3 Results and Findings This section sets out the findings of the research, set out according to different stakeholder groups, starting with FBOs, followed by Producers, Financial Institutions, Processors and Government/Government bodies. A summary of the specific project indicator achievements, including disaggregates is included in Annex 1: Table of Program Indicators. 3.1 Farmer Based Organisations This section is based on project information and KIIs conducted with the committees of 15 FBOs5 , covering high-, medium- and low-performers across the five districts. The KIIs sought to verify information provided by MSIKA about the performance of specific FBOs and to generate qualitative data on FBOs engagement with MSIKA, progress, successes, challenges and sustainability. The section also draws on an exercise conducted by MSIKA in August 2020 to review progress of its FBOs using V37’s Performance Measurement Management (PMM) tool through phone interviews with 49 FBOs. Finally, this section also draws out lessons learned. 3.1.1 Background and outline of work with FBOs Central to MSIKA’s approach has been to work with, and through, FBOs to strengthen their capacity to operate and to deliver services to their members. This approach has the merits of providing a mechanism for reaching large numbers of farmers with training in a relatively efficient manner and as a means for sustaining the project activities and impacts beyond the life of the project. At the outset of the program, MSIKA undertook a mapping exercise to identify potential participants and any established FBOs that the team could work with. This was undertaken in collaboration with the Ministry of Agriculture’s district-level structures, the District Agricultural Development Offices (DADOs), and with other NGOs operating in the target districts. The aim was to identify Extension Planning Areas (EPAs) where the target crops were more commonly grown and to determine where there were potential participant producers not supported by projects and NGOs. MSIKA worked with some existing FBOs, and supported the establishment of new FBOs. For the new FBOs, once the target EPAs were agreed, MSIKA called meetings for producers of the target crops at Group Village Head (GVH) level, typically covering two to three villages. The aim was to introduce the MSIKA project and motivate producers to form clubs of 15-30 members. From that point, a minimum of five and up to 10 clubs were formed into an FBO within approximately a 5km radius (maximum 8km), so that it would be practical for members to get together. The aim was to have from 75 to 300 producers in one FBO. Each club has five leadership positions (Chairperson, Vice￾Chairperson, Secretary, Vice-Secretary and Treasurer) and the leaders of the clubs (Chairpersons) form the committee for their FBO. The FBOs elected officials in the same five roles out of this group of leaders, with elections held annually. The MSIKA team also engaged with existing FBOs in the target EPAs, if they were focused on the target crops or had a substantial number of members that grew the target crops. Working with existing structures has the advantages of faster commencement of activities and not requiring all the initial capacity building support. However, there can be a disadvantage if a pre-exiting FBOs is not functioning well, as it is difficult to change embedded behaviours/practices. 5 FBOs included registered co-operatives and associations plus unregistered groups. Some FBOs call themselves co-operatives or associations, but are not formally registered as these. For this report, the term ‘FBO’ encompasses any group of producers whether formally registered or not that MSIKA has worked with. MSIKA Final Evaluation Page 15 kadale@africa-online.net MSIKA encouraged FBOs to form the following four sub-committees: Markets, Finance, Production (chaired by a Lead Farmer), and Disciplinary (usually chaired by an elder in the community). There were other committees in some longer established groups. These sub-committees included FBO leaders, club leaders and members. A key point to note was that the focus for Yr 4 was on 50 FBOs, which would have had some effect on coverage and activities with FBOs, and was compounded by the impact of COVID on FBO activities (see 3.1.8). 3.1.2 FBOs summary data MSIKA reports that it has worked with 217 FBOs as at September 30th, 2020, consisting of 56 in Dedza (25.8% of all FBOs), 36 in Lilongwe (16.6%), 40 in Mangochi (18.4%), 41 in Mchinji (18.9%) and 44 in Ntcheu (20.3%). Dedza has the most FBOs with the other four districts relatively evenly split. The total was 219 at mid-term, but two FBOs were amalgamated into other FBOs covering similar producers. Out of these 217 FBOs, 167 (77%) were established by MSIKA, and 50 (23.0%) were pre-existing. For the purposes of the evaluation, MSIKA categorized its FBOs6 into high-performing - 59 (27.2%); medium performing - 84 (38.7%) and low performing - 74 (34.1%). Overall, 65.9% are medium- or high-performing. Compared to the MTE, the number of high and low performing FBOs at FE has reduced, resulting in a shift to medium-performing, which now representing 38.7% of FBOs compared to 28.3% at mid-term. There has also been an improving trend in performance in Mangochi and Lilongwe, while it has weakened in Mchinji and Ntcheu. The details for the FE and the mid-term are set out in the two tables below: Table 5: MSIKA’s Performance Rating of FBOs, FE, 2020 Performance rating as % of each district total Districts High Medium Low Total FBOs % of All FBOs # % # % # % # % Dedza 10 17.9% 31 55.4% 15 26.8% 56 25.8% Lilongwe 13 36.1% 7 19.4% 16 44.4% 36 16.6% Mangochi 11 27.5% 17 42.5% 12 30.0% 40 18.4% Mchinji 12 29.3% 19 46.3% 10 24.4% 41 18.9% Ntcheu 13 29.5% 10 22.7% 21 47.7% 44 20.3% All districts 59 27.2% 84 38.7% 74 34.1% 217 100.0% Table 6: MSIKA’s Performance Rating of FBOs, MTE, 2019 Performance rating as % of each district total Districts High Medium Low Total FBOs % of All FBOs # % # % # % # % Dedza 15 27.3% 15 27.3% 25 45.5% 55 25.3% Lilongwe 10 28.3% 6 17.1% 19 54.3% 35 16.1% Mangochi 10 25.6% 14 35.9% 15 38.5% 39 18.0% Mchinji 19 46.3% 14 34.1% 8 19.5% 41 18.9% Ntcheu 18 36.7% 13 26.5% 18 36.7% 49 22.6% All districts 72 32.9% 62 28.3% 85 38.8% 219 100.9% Source: MSIKA internal data 6 MSIKA has internally set criteria for assessing performance that draws on the PMM tool. These cover questions on governance and leadership, market access, production capacity, adaptive and operational capacity and capacity needs. This enables the MISKA team to assess the level of performance and prioritization of needs. MSIKA Final Evaluation Page 16 kadale@africa-online.net MSIKA undertook a formal assessment of FBO capacity development in August 2020 using V37’s PMM tool, which it has developed based on its years of working with farmer cooperatives. The PMM tool covers six performance areas: leadership (goals, strategy); adaptive capacity; management capacity; operational capacity; supply, processing and marketing; and productivity and financial performance. Each of these has several sub-areas of performance. For each sub-area of performance, there are four descriptive statements corresponding to four levels of performance that enable the interviewer to place the FBO’s performance in the one that best describes the current situation. Level one is the lowest level (‘base’)7 which equates to the starting level of capacity with very limited capacity to function as an organisation, make decisions, manage finances, etc. up to level four which is the highest-level of capacity which equates to well-functioning leadership and governance, robust systems throughout, ownership of premises, effective decision-making. The tool can be used for assessing progress if there is a baseline to compare to, and for helping individual FBOs to consider the areas on which they need to improve as the basis to develop an action plan. The assessment exercise set out the results for 38 FBOs established by MSIKA as well as for 11 pre-existing ones. For the newly established FBOs, the baseline would be 1.0 since they did not exist before. The current mean score for the newly established FBOs across all six areas (not weighted) was 1.77. 8 This equates to FBOs being around three quarters of the way between level one and level two. The highest scores were on leadership (1.99) and management capacity (1.98), followed by Supply, Processing and Marketing (1.81), and adaptive capacity (1.79). The lowest scores were for Operational Capacity (1.58) and Productivity and Financial Performance (1.32). The results are set out in the table below. Table 7: PMM Assessment, MSIKA-Established FBOs at FE, 2020 FBOs established by MSIKA Performance area Mean Score Leadership 1.99 Adaptive Capacity 1.79 Management Capacity 1.98 Operational Capacity 1.58 Supply, Processing & Marketing 1.81 Productivity & Financial Performance 1.32 Across all areas 1.77 N=38 newly established FBOs For the pre-existing FBOs, MSIKA undertook an assessment of the baseline and the current scores to determine what change had occurred. The baseline score, determined retrospectively, was a mean of 1.45, with a current mean score of 2.04 and a change of 0.59 points. The current mean scores for the six performance areas were relatively similar and between 1.94 to 2.12. 7 Meaning the lowest possible score is 1.0. 88 The consultants, with agreement of MSIKA, updated the scoring to replace zero scores with ‘n/a’ meaning ‘not assessed’. Zero scores meaning not assessed, incorrectly reduced the means. MSIKA Final Evaluation Page 17 kadale@africa-online.net Table 8: PMM Assessment, Pre-existing FBOs, 2020 FBOs established prior to MSIKA Baseline Current Difference Performance area Mean Score Mean Score Leadership 1.59 2.12 0.53 Adaptive Capacity 1.55 2.06 0.52 Management Capacity 1.42 2.07 0.65 Operational Capacity 1.73 2.09 0.36 Supply, Processing & Marketing 1.34 1.94 0.60 Productivity & Financial Performance 1.31 1.6 0.65 Across all areas 1.45 2.04 0.59 N=11 pre-existing FBOs The data from the internal scoring of FBOs by MSIKA and the recent more extensive PMM assessment set out above, highlight that there has been progress in strengthening the capacity of FBOs. However, the progress has been limited with most FBOs not yet at level 2, meaning limited capability. In the FE HH survey, participant producers were asked about the MSIKA activities they participated in over the last 12 months, all of which are accessed via an FBO. The most common were ‘training in VSL’ (87.8%), ‘training in basic agri-production’ (86.2%), ‘training in PHH’ (81.5%) and ‘training in gender equality’ (80.0%). The lowest scores were ‘participated in international/district fair’ (15.8%) and ‘been linked to markets’ (49.8%). Of the scores, several were lower than the MTE, notably training in PHH (MTE: 91.8% = -10.3% points), ‘linkage to market’ (MTE: 61.9% = -12.1% points) and ‘participated in international/district fair’ (MTE: 25.1% = -9.3% points). The lower scores at FE could be due to disruption from COVID as it particularly affected training and market activities, based on the qualitative interviews with FBOs. There were increased scores in the FE survey on gender training (MTE: 73.4% = +6.6%), suggesting an increased focus on this, in the period when this was possible. The FE results are set out in the table below: Table 9: MSIKA Activities Reported by FBO Members in the MTE & FE HH Surveys Which of the following activities from Land O’Lakes /MSIKA have you participated in? Activity MTE FE Male Female Total Male Female Total % % % % % % a. Training in improved horticulture practices by MSIKA staff or a lead farmer 93.8 92.8 93.2 85.8 86.7 86.2 b. Training in horticulture post-harvest handling by MSIKA staff or a lead farmer 91.1 92.3 91.8 79.8 83.6 81.5 c. Training in marketing 81.7 83.5 82.8 79.1 79.3 79.2 d. Training in financial literacy 73.5 74.0 73.8 74.4 72.5 73.5 e. Training in gender equality 77.4 70.7 73.4 81.3 78.4 80.0 f. Training/membership of a village savings and loan group 85.6 89.5 87.9 85.6 90.5 87.8 g. Been linked to an MFI 62.3 66.1 64.6 63.0 63.3 63.1 h. Been linked to markets to sell horticulture products 59.5 63.5 61.9 47.3 52.9 49.8 i. Participated in international/District trade fair 28.4 22.9 25.1 17.0 14.4 15.8 j. Any other? 0.4 - 0.2 1.9 0.2 1.1 Total 257 389 646 535 444 979 N=646 & 979, multiple response possible MSIKA Final Evaluation Page 18 kadale@africa-online.net 3.1.3 Training at FBOs As noted earlier, MSIKA’s strategy is to develop the capacity of FBOs as a sustainability mechanism, and because it is efficient and effective to work through a community level organisation that can reach many farmers. MSIKA places a high priority on training of FBO committees and members. The training falls into two broad categories: 1. Training the FBO leadership to make the FBO an effective organization; and 2. Training farmer members using the FBOs as a grouping mechanism covering the key impact areas for MSIKA notably agricultural production, PHH and storage, financial management and marketing/sales. This section focuses on both of the above categories, as delivered through the FBOs. The knowledge and application of training by the producers is covered in the section that follows (see section 3.2 below). Training is delivered by MSIKA’s technical specialists and by its Farmer Business Co￾ordinators (FBCs), of which there are two per district. It was reported at MTE by MSIKA that the FBCs are very stretched in undertaking the training and all their other roles. The revised plan for year 4 was to focus on 50 FBOs that had higher potential. As the final year/year 4 has unfolded, training conducted up to the end of February was recorded by the FBCs, but from March it has been difficult for training to be conducted by FBCs/MSIKA technical teams and by FBOs/lead farmers. Although the pandemic has waned in Malawi from September, it was too late to pick up further training. MSIKA provided a training tracker up to the end of February 2020 from which the consultants have analysed the training that MSIKA reported providing to 213 FBOs.9 MSIKA’s training for FBOs and for producers falls into five categories: 1. Agricultural production; 2. Marketing; 3. PHH and storage; 4. Financial Management; and 5. FBO Development. Each category of training is considered in turn below. 3.1.3.1 Agricultural Production Training Agricultural production training covers: basic agri-production training, soil fertility management, integrated pest management, climate smart agriculture (CSA) and irrigation. MSIKA intended to train all participant producers in basic agricultural production as a foundation for all other agricultural training. The aim is to introduce producers to good agricultural practices (GAP) that can be applied widely across field and tree crops, along with some crop specific training where there are commonly produced crops. This is followed by refresher and additional training for more in-depth agricultural production topics. Training is intended to be ongoing and progressively building on earlier training through a modular approach. From analysis of the training tracker data, all FBOs have had training for members in basic agri-production practices, an increase to 100.0% from 97.2% at MTE. The next most common types of training were integrated pest management (FE: 68.1%, MTE:49.3%) integrated soil fertility management (FE: 62.0%, MTE:41.3%) and climate 9 The training tracker has data for 213 out of 217 FBOs. The difference is attributed to different dates of preparation of the two documents/sources. MSIKA Final Evaluation Page 19 kadale@africa-online.net smart agriculture (FE 61.5%, MTE: 23.5%). Irrigation was still lagging (FE: 31.9%, MTE: 0%), but not all locations are suitable for irrigation. The consultants expect that the results would have been higher if COVID had not occurred. Table 10: FBOs Trained in Agricultural Production, by Type of Training, at MTE & FE Agricultural production training % trained at MTE % trained at FE Basic agri-production 97.2% 100.0% Integrated soil fertility mgt 41.3% 62.0% Integrated pest management 49.3% 68.1% Climate smart agriculture 23.5% 61.5% Irrigation 0.0% 31.9% All FBOs 213 213 Multiple response possible Qualitative interviews with the FBO leaderships supported the above data with evidence of widespread adoption of a range of agricultural practices, even in lower performing FBOs. The committee members were asked why some practices have been adopted and others not. They provided a range of reasons: i. Some practices are less costly, so easy to adopt; ii. Some practices were better understood than others and easier to follow; iii. One FBO said the training was not “first-hand” or “comprehensive” so not well understood; iv. Some farmers are more self-motivated and confident, so they adopt practices more than those who are not as motivated or confident; v. Some farmers were practicing these before MSIKA, so they continued; vi. Some practices were affected by factors such as agrodealers supplying mixed variety seed, animals ruining storage sheds and a low supply of manure; and vii. Lack of markets demotivated some from applying more effort. FBOs offered a range of suggestions on how to improve the uptake of practices: i. Continuation of the training and practical application through demonstrations; ii. More training and demonstration time allocation; iii. Greater encouragement among the farmer members; iv. Ready market linkages to motivate them to supply; v. Frequent extension visits; and vi. Provision of input loans to farmers Across all three FBO performance categories, there was evidence of a range of good results from the use and adoption of the improved techniques and technologies, specifically higher yields, higher quality, reduced on-field and transportation losses, and higher selling prices: "From an acre, farmers used to produce 60 (20L pail), but now are able to produce as much as 200 (20L) pails of Irish potato.” (Equates to an increase from 960kg to 3,200kg). High performance FBO “There has been a great rise in production levels and quality in a short period because these (Land O Lakes) extension officers follow things up more closely than the government ones who are busier and under-resourced sometimes.” Medium performance FBO MSIKA Final Evaluation Page 20 kadale@africa-online.net “Before MSIKA came, we were not as many who were farming Irish potatoes, but now so many farmers are into Irish potatoes, onions and tomatoes." Low performance FBO The conclusion by the consultants is that the agricultural training delivered via the FBOs has been effective overall, with an uptake of a range of practices by farmers that have positively impacted on their production yields and quality. While there may be some gaps in training coverage, and in farmer knowledge and uptake, these were impacted by the challenges faced by MSIKA over the second half of the program year. 3.1.3.2 Marketing Training A key focus for MSIKA is improving links to markets and increasing sales to enable increased crop volumes to be sold, which in turn motivates and enables farmers to invest in increasing production and improving quality. MSIKA offers a range of marketing training modules: 1. Introduction to marketing; 2. Marketing principles; 3. Market searching; 4. Costing and pricing; and 5. Customer care management. The marketing training is mostly delivered as a block of five modules. MSIKA reports training that between 81.7% to 83.6% of FBOs were trained in these modules. This compares to the range of 49.8% to 53.1% of FBOs at mid-term. It was a clearly articulated aim for the MSIKA team to train many more FBOs in marketing and this appears to have been achieved. Table 11: FBOs Trained in Marketing, by Type of Training, at MTE & FE Marketing training % trained at MTE % trained at FE Introduction to marketing 53.1% 83.6% Principles of marketing 52.6% 83.1% Marketing searching 50.7% 82.2% Costing and Pricing 49.8% 81.7% Customer care management 51.6% 82.6% All FBOs 213 213 Multiple response possible Out of five FBOs in each category, four high performance, four medium performance and four low performance FBOs reported having been trained in Market Access by MSIKA. This verifies with what MSIKA had in its records. The four high performance FBOs reported having benefitted from applying marketing principles/practices such as negotiating prices, knowing own ability to supply enough volume, understanding the quality sought by buyers, running the farming as a business, and offering competitive quality. Three medium performance FBOs reported having applied/benefitted from activities/practices such as: linked to markets by MSIKA, negotiating prices, ability to calculate profits, analyzing markets, advertising, and ability to produce and deliver high quality crops. Only one low performance FBO reported having applied/benefitted from the training by knowing price trends before going on the market. The other two reported not having benefitted from the training, because they still have not secured the expected good markets. MSIKA Final Evaluation Page 21 kadale@africa-online.net Overall, nine of the 15 FBOs reported one or more links to buyers, including retailers (five FBOs for fruit and vegetables) and processors (four FBOs for chilies). "They just linked us to a market, SPAR supermarkets." Medium performance FBO "At first, members reported their crops to us to inform MSIKA. However, MSIKA did not help us find a market and since then we have been resorting to vendors like pre MSIKA times...." Low performing FBO In one case, the FBO had an agreement, but pulled out as they felt they could not supply: "We actually gave up, we were afraid that the volumes being aggregated for supply of the very first consignment were very low compared to that in demand..." High performance FBO With the advent of COVID, the marketing training switched to being remote with FBOs being provided with Bluetooth speakers. This was a flexible response, though likely to have been less effective than face to face training. From the consultants’ assessment, marketing/access to markets is often the most challenging aspect of a program. The training coverage is much higher than in the mid￾term, where it was just over 50% of FBOs and was over 80% when it was stopped in March. There are expectations by FBOs that MSIKA will find them markets, and this may encourage a degree of passivity on the part of the FBOs and members. MSIKA is seeking to both improve links, but also for FBOs and farmers to build their capacity to find and access markets for themselves. However, a reduction in facilitating linkages due to COVID means that some of the expectations remain unfulfilled. There would likely have been more benefits and more opportunity to consolidate learning and capacity if the marketing training had been completed at an earlier stage, accepting that COVID was not predictable and that originally there was to be a fifth year to the project. 3.1.3.3 PHH Training The third set of modules are around reducing losses through improved post-harvest handling (PHH) and storage. The PHH and storage training modules are: 1. Post-harvest handling; 2. Storage requirements; and 3. International handling standards. From the FBO training summary, 34.7% of FBOs had been trained in PHH (MTE: 24.4%), 33.3% trained in storage requirements (MTE: 23.5%) and 25.8% in international standards (MTE: 18.8%). Detail of the training by module is set out below. Table 12: FBOs Trained in PHH and Storage, by Type of Training, MTE & FE Post-Harvest Handling and Storage training % trained at MTE % trained at FE Post-Harvest Handling 24.4% 34.7% Storage Requirements 23.5% 33.3% International handling Standards 18.8% 25.8% All FBOs 213 213 Multiple response possible The plan at mid-term was to increase PHH and storage training. This has happened, but falls short of complete coverage. The timing of the training is important as it needs to be close to when it will be used, rather than well in advance of it. As March to May are peak times for harvesting, then the advent of COVID affected the training plans for this set of modules. The overall reduction in the budgets has also played a part. MSIKA Final Evaluation Page 22 kadale@africa-online.net In the KIIs, more of the FBOs reported being trained, than MSIKA reported they had training. MSIKA clarified that the PHH training was commonly tied in with the marketing training, hence it not appearing in the training record separately. International handling standards training is related to indicator 31: Percentage of FBOs who are aware of international production and handling standards, as a result of USDA assistance. MSIKA conducted a phone survey of FBOs trained as at 31st December 2018 (MTE), and found 71.8% achievement against the LoP target of 75.0%. As at 30th September, 2020, MSIKA reports 78% achievement thereby exceeding the target for indicator 31. The validity of raising awareness of international standards for FBOs that have, or whose members have, very limited capacity and potential to export was questioned by the consultant in the mid-term report. The same question remains, as there is little scope for most of the target FBOs and farmers to contemplate formal exports at this point. The issue of international standards was only relevant for FBOs with members that grow chilis, paprika and red cayenne, which are primarily for the export market, but which were deprioritized. The consultants’ conclusion is that indicator 31 was not appropriate for MSIKA, as none of the 16 FBOs visited in the MTE or the 15 FBOs called in the FE were capable of directly exporting. 3.1.3.4 Financial and VSL Training MSIKA sees financial literacy skills and knowledge for producers as important to their development. There are three financial modules: 1. Keeping financial records; 2. Financial literacy; and 3. Village savings and loans (VSL). Since the MTE, MSIKA has increased the proportion of FBOs with financial and VSL training from between 47.4 to 49.3% across the three modules at MTE to 62.4 to 64.3% by FE. As with the other training, this had to be cut short due to COVID, meaning that fewer FBOs were trained than planned. As with marketing, FBOs that are trained in finance tend to have had all three modules. There are exceptions due to the needs of FBOs, particularly the well-established ones, who may already have been trained in one or more aspects. Table 13: FBOs Trained in Financial Records, Literacy and VSL, MTE & FE Financial & VSL training % trained at MTE % trained at FE Keeping financial records 49.3% 64.3% Financial literacy 47.4% 62.4% Village savings and loans 49.3% 63.4% All FBOs 213 213 Multiple response possible Of the five FBOs in each category, four high-performance, three medium-performance and three low-performance FBOs reported having been trained in Improved financial management by MSIKA, verifying what MSIKA had reported. The one other high￾performance FBO reported being trained in Improved Financial Management by a consortium of other projects, so it was not necessary. On Improved Financial management activities, three of five high-performance, two medium-performance and no low-performance FBOs reported having MSIKA support in implementation of the activities. MSIKA offered support to high-performance FBOs in form of facilitating MFI loan applications, coaching, providing stationary and MSIKA Final Evaluation Page 23 kadale@africa-online.net recommending better banks for loans. The medium-performance FBOs reported support in form of books for recording VSL information and for other records. Promoting VSLs has been a key part of MSIKA’s work. The training model relies on village agents to mobilize VSLs, who have already been trained in VSL methodologies by other organizations. In relation to MSIKA’s FBOs, training in VSL is undertaken by a mix of these agents and by MSIKA’s FBCs. FBO members are invited to form new VSLs where these do not exist, however it is not compulsory for MSIKA members to join a VSL. There is a rule that MSIKA VSLs do not allow non-FBO members to join, although FBO members are not stopped from joining non-FBO VSLs. VSLs are quite common in rural areas, so restricting access would be problematic. 3.1.3.5 Governance, Leadership and Cooperative Development The final area of training is governance, leadership and cooperative development. The modules are: 1. FBO Governance Structures 2. Leadership Skills 3. Cooperative Education MSIKA focused on training the FBO leadership in the early stages, however because there is a regular change in leaders, there needed to be wider membership awareness of governance issues. At FE, MSIKA reports that 77.9% of FBOs have been trained in FBO governance (MTE 56.8%, change +21.1%), 64.3% of FBOs trained in leadership skills (MTE 35.2%, change = +29.1%) and 1.4% trained in cooperative education (MTE 0%, change 1.4%). MSIKA team noted at MTE that more training in governance was required to assist sustainability, as well as performance. The FE levels show that there has been a substantial increase in training, noting also that some of the pre-existing FBOs would not require some of the governance and leadership training. However, the plans for further training were affected by the reduction in budgets and the COVID restrictions. On cooperative education, MSIKA report that six FBOs were linked to the Ministry, and three of these to another V37 project called Co-operative Development Program (CD4) for training. This is an additional three to the three that MSIKA reports that have been trained in the table below. Details on governance training are set out in the two tables below for the FE and the mid-term: Table 14: FBOs Trained in Governance, Leadership & Cooperatives, MTE and FE Governance, leadership & cooperative training % trained at MTE % trained at FE FBO Governance Structures 56.8% 77.9% Leadership skills 35.2% 64.3% Cooperative Education 0.0% 1.4% All FBOs 213 213 Multiple response possible According to MSIKA, formal cooperative training by the Ministry of Trade and Industry is relatively long. The consultant’s finding is that cooperative training is not suited to many FBOs, as converting to a registered cooperative requires sufficient levels of membership and the capital to sustain it, as well as strong and effective leadership. At mid-term, MSIKA staff estimated that around 10% of FBOs have potential to become cooperatives. From an analysis of the summary data on FBOs, only 12 were identified with membership over 300, while over half have less than 150 members. The larger FBOs tend to be the longer established, pre-existing farmer entities. The consultant’s MSIKA Final Evaluation Page 24 kadale@africa-online.net finding is that most of the MSIKA FBOs do not have potential to become functioning cooperatives. This does not mean FBOs do not have a valuable function as effective groupings for accessing training, inputs and potentially for aggregation and selling. From the qualitative KIIs and discussion with the MSIKA team, the implementation by FBOs of the governance training is mixed, as it was at MTE. Two high-performing, two medium-performing and one low-performing FBOs found Leadership to be the most useful topic because good leadership sustains and grows consistent membership. It also supports confident duty execution and harmonious management of affairs due to clear awareness of everyone’s respective roles. From the MTE and wider work, the consultant’s finding is that leadership is the most challenging area for FBO functioning and development, particularly if the leaders are embedded and are not performing well. "Knowledge of the things from the training made leadership easy to work with members and steer forward wisely"…Low performing FBO One high performing FBO found By-law formulation the most useful, one medium performing found book-keeping to be most useful and one low performance found Good governance to be the most useful topic. In the latter case, they said that it helps them carry out critical commitments, such as loan repayments without fail. "Recently we were assisted to access loans from the MFI for capital. People we faithful through compliance to the FBO's bylaws. Everyone paid back on time as required..." High performing FBO "Through record keeping, we are also able to have an account of where we are coming from and what we are aiming to achieve..."…Medium performing FBO There were some references in the FBO KIIIs to governance training that had not been undertaken, which the consultant presumes relates to the COVID situation. From the KIIs, there is evidence of activities, such as regular meetings and elections in many, but not all FBOs. COVID appears to have reduced the number of governance related activities by the FBOs, which may be down to the ceasing of regular engagement with the FBOs by MSIKA’s FBCs. This may imply that ongoing operation of some FBOs may partly depend on regular engagement to remind, encourage and motivate the FBO leaderships. FBOs are required by MSIKA (and by the by-laws they have adopted) to keep records, such as minutes of meetings, market information (buyer product specifications), production and sales records. However, because the research was conducted by phone, it was not possible to verify if the records existed or to see the state of the records. In some interviews, it was clear that the FBO committees were giving information from the records, while in others, the respondents did not appear to have the information available. 10 It should be noted that some records may not be kept, e.g. sales records, due to lack of transactions. In the staff KIIs, MSIKA highlights the need for ongoing and repeat trainings for leaders, since there is going to be committee membership turnover, as these are annually elected positions. This is a sustainability challenge for some FBOs. Indicator 33 is the number of FBOs trained in improved financial and organizational management. MSIKA reported progress at 183 trained against a revised LOP target of 135 (136% achievement), as at 30th September 2020. From the FBO KIIs, of the five in each category, four high performing, three medium performing and three low performing FBOs reported having been trained in improved financial management by MSIKA. This matched what MSIKA had on record. One high 10 The researcher made an appointment with the committee and requested them to have their records to hand. MSIKA Final Evaluation Page 25 kadale@africa-online.net performing FBO reported being trained in Improved Financial Management by other projects, so there was no need for MSIKA to also train them. On improved financial management activities that MSIKA supported and were implemented11, three of five high-performing, two medium-performing and no low￾performing FBOs reported having MSIKA support in implementation of the activities. Examples were MSIKA support to high performing FBOs in facilitating MFI loan application, coaching, providing stationary and recommending better banks for loans and for medium performing FBOs in the form of books for recording VSL information and other records. One high performing FBO had implemented three elements of the training, including MFI) loan application and management, record keeping in exercise books, and getting advice on which banks the FBO could open an account with. In contrast to those that have implemented learning, the challenge with some FBOs is captured in the quote below: "MSIKA did not help us get markets, hence we could not implement the systems we learnt from them..." Low performing FBO The implementation by the high performing FBO and the quote from the low performing FBO illustrates the range of responses by FBOs about working with MSIKA, and likely with other future projects. Some are willing and able to take on elements of training that is relevant to them, while others have unrealistic expectations of what a project can or should do for them, noting that MSIKA is trying to help FBOs find markets sustainably. MSIKA staff may have encouraged the expectations over time based on some feedback at MTE; however finding a market for every FBO is not possible to fulfil and some FBOs may see this as only the responsibility of project staff, and not see training as something to improve their capacity to do more things for themselves. The MTE recommended that MSIKA focus on better performing FBOs, which was a recommendation that MSIKA took up by focusing on 50 FBOs, 10 per district. The full effect of that change in focus was disrupted by COVID. In the KIIs, the FBOs gave suggestions on how to improve the training that they had received. Training is something that FBOs expect and were clear that they want a continuation of. The improvements they proposed were: i. Good notice should be given prior to the training ii. Trainers should be punctual iii. More time should be allocated to the training iv. The training length should be extended v. Some parts should be repeated upon request from participants vi. Motivational real life stories should be part of the training content These are not responses from all FBOs, but give a sense of areas that could be enhanced in future. Several are to do with the organising of the training (i-iv) and others to do with how it is delivered (v-vi). 3.1.4 Lead Farmers Lead Farmers played an important role in the MSIKA approach with 2-3 designated Lead Farmers at each FBO. MSIKA utilized lead farmers as training of trainers, training the lead farmers on agricultural practices and PHH, and the lead farmers were expected to train other FBO members in these practices, establish demonstration plots and provide ongoing advice and extension to members. Many lead Farmers were 11 This relates to Indicator 30, though this indicator is not reported on in Yr 4. MSIKA Final Evaluation Page 26 kadale@africa-online.net selected prior to MSIKA by the communities as part of a wider government initiative, so it was appropriate for MSIKA to work with these individuals. However, MSIKA also picked others that were appropriate based on their experience of the crops and performance. In general, MSIKA found that the latter were generally better performing. Based on the FE HH survey, 36.0% of respondents said they were lead Farmers. This high proportion is likely to be a function of doing a phone survey and with the Lead Farmers numbers known to MSIKA and the FBOs who provided many of the numbers. Of the lead farmers, 41.3% were female and 58.7% were male, though the higher number of male lead farmers is in line with the higher proportion of men in the sample. Table 15: Proportion of Lead Farmers in the FE Survey Are you a Lead Farmer trained by Land O Lakes/MSIKA? Response Male Female Total Count % Count % Count % No 328 61.3 299 67.3 627 64.0 Yes 207 38.7 145 32.7 352 36.0 Total 535 100.0 444 100.0 979 100.0 These Lead Farmers were asked about the activities they had undertaken (prompted) for MSIKA. A high 91.3% (mid-term 83.3%) said they had established a demonstration plot, 87.1% stated they had trained others (mid-term 94.0%), and 58.4% (mid-term 79.8%) had submitted training forms. Training and demonstrations are key for MSIKA. The lower score for submitting training forms suggests there is more training that has occurred than is reported. The responses on activities are set out in Table 16 below Table 16: Lead Farmer Activities by Sex at FE, 2020 As a lead farmer, have you done any of the following? Activity Male Female Total % % % Conducted trainings of other farmers 89.9 83.1 87.1 Submitted training forms to MSIKA about the training 58.1 58.8 58.4 Established a demonstration plot 88.9 94.9 91.3 Any other? 1.5 1.5 1.5 Total 198 136 334 N=352 Lead Farmers 3.1.5 Sales Agreements and Sales Indicator 29 is the number of sales agreements with FBOs. MSIKA works with FBOs to link them to potential buyers and to make sales agreements for FBO members to fulfil. At mid-term, the cumulative achievement was 52, but this increased substantially in Yr 4 by a reported 209 to reach a cumulative LoP total of 261 against a target of 62, representing 421.0% achievement. The additional target for year 4 was 10 agreements, which looks modest against the actual achievement. From the FBO KIIs, five FBOs gave examples covering six sales agreements, with two for Tingadalire (chillies), two for North East Foods (chillies) and one FBO with two large supermarkets (Spar and Shoprite for vegetables). Two FBOs gave examples of where they were linked, but either did not reach an agreement or were not able to supply. There were examples where MSIKA said there was an agreement and the FBO did not confirm it, but also the other way around where sales agreements were reported by the FBOs, but not by MSIKA. This may be a function of changes in leadership where earlier agreements were not known or recalled. It could also be a limitation of the interview process, as time was short and records could not be checked and were not always available despite being requested in advance. MSIKA Final Evaluation Page 27 kadale@africa-online.net "That was the only time when we were surprised to enjoy a selling price as high as K600 per kilogram of tomato….. The Spar offer was unprecedented…." Medium performing FBO. Sales agreements are not necessarily followed through to make actual sales: "We actually gave up, we were afraid that the volumes being aggregated for supply of the very first consignment were very low compared to that in demand..." High performing FBO MSIKA provides FBOs with sales books and the FBOs ask participant producer members to provide sales data to their club, which provides this data to the FBO level for summarizing and reporting to MSIKA. MSIKA report sales against indicator 21 volume of commodities sold by project beneficiaries and indicator 23 value of sales by project beneficiaries. For indicator 21, volume sold, MSIKA reports sales of 62,963 mT against an LOP target of 86,330 mT, thereby achieving 72.9% of the target. For indicator 23, value sold, MSIKA reports US $18,440,386 against an LoP target of US $21,972, 909, which is an achievement of 83.9%. Year 4 sales were around 5% of the Year 3 sales, with the impact of COVID affecting the result in two ways. Firstly, the COVID pandemic began during the main production season for the field crops. The closure of governmental institutions, restrictions on hotels/lodges, including meetings/conferences, and restrictions on general movements, resulted in a significant loss of business from formal buyers for fruit and vegetables. Producers only had local markets to supply rather than formal outlets that could buy large quantities at relatively good prices. See the section 3.1.8 on COVID below for more detail. The second effect has been difficulties for MSIKA staff to gather information on sales, both from FBOs and from processors, as they were not able to go to the field and gather data in person. There also appears to have been less reporting of their own sales by farmers to their FBO. This likely resulted in sales for the second half of Yr4 being understated. The sales data reported in the FE HH survey (section 3.2.12) suggest that farmers were still able to sell, but primarily to local markets. From the MTE and FE KIIs, MSIKA staff highlight the difficulties of getting FBOs to keep sales records, particularly as most farmers sell individually rather than collectively. Part of the aim of FBOs KIIs was to verify sales agreements and data. There was some confirmation, but also some differences. Because the researchers could not check the records, it proved difficult to verify sales volumes. Overall, there is evidence of sales agreements and sales facilitated by MSIKA through the FBOs as a mechanism for their members to sell their produce. Some FBOs seem to have grasped the opportunities, while others have expected MSIKA to keep providing links. The latent or existing capacity of the FBOs is a key factor in whether sales can be made and delivered. 3.1.6 Processing and Storage From the qualitative interviews, none of the FBOs in the MTE or FE KIIs had warehousing or were processing crops. The consultant’s finding is that storage and processing are unlikely to happen at the present state of development of most of the FBOs. Particularly with the newly established FBOs, these are relatively small groups with limited capacity and resources. As noted in the PMM assessment, capacity is still weak in many areas. MSIKA Final Evaluation Page 28 kadale@africa-online.net 3.1.7 Access to Market Information Indicator 18 is the percentage of producers who have access to current market information through their cooperatives or producer associations. MSIKA aims to increase access to market information for participant producers through their FBOs. Market information includes location of buyers, names of possible buyers, where to buy inputs, where to access finance, buyers’ requirements, weather, and market prices. This access was measured in the FE HH survey by a question about the source of market information (unprompted) on each type of market information. The findings are set out in Table 17 below. Across the range of types of market information, the two most commonly sources were ‘MSIKA extension workers’ (31.3%-52.7%) and ‘radio’ (22.5%-59.2%). MSIKA extension workers were the most common source for ‘location of market’ (45.1%), ‘names of buyers’ (37.1%), ‘where to access finance’ (52.7%), and buyers product needs’ (41.6%). Radio was the most common source for ‘where to buy inputs’ (47.9%), weather that affects crops (59.2%) and ‘market prices for crops’ (39.4%). MSIKA Lead Farmers are also regarded as a significant source, mentioned by 12.6-18.6% of respondents across the different types of market information. For FBOs as a source of market information, the relevant responses were ‘fellow FBO members’ (2.8%-4.9%) and from ‘FBO leadership’ (3.4%-5.9%). Table 17: Sources of Market Information, FE 2020 Information Sources Response Location of Market To Sell Names of buyers Where to buy inputs Where to access finance Buyers’ product needs Weather that can affect crops Market prices for your crops % % % % % % % Neighbor/friend 10.4 10.2 11.8 6.3 5.8 2.8 9.2 Govt ext’n officer 7.8 4.6 9.4 5.5 6.1 9.5 6.5 Land O’Lakes/MSIKA extension worker 45.1 37.1 43.7 52.7 41.6 36.0 31.3 LOL/MSIKA lead farmer 18.6 16.3 15.7 18.1 16.8 14.8 12.6 Private buyer ext’n officer 0.9 0.9 0.4 0.5 0.4 0.1 0.8 NGO/Project ext’n officer 2.7 1.2 1.6 3.0 0.9 0.9 1.3 Vendors/traders 12.0 10.3 4.6 0.7 19.1 0.5 20.8 Radio 36.3 27.4 47.9 22.5 23.6 59.2 39.4 SMS 2.6 1.6 1.2 0.5 0.9 2.6 1.6 FBO Leadership 5.9 5.6 4.7 5.5 5.0 3.4 4.9 Fellow FBO member 4.3 4.2 4.9 4.0 3.6 2.8 4.1 Posters 0.3 0.5 1.2 0.1 0.3 0.1 0.6 Agricultural shows 0.2 0.3 0.4 0.2 0.2 0.7 0.3 Trade fairs 0.1 0.2 0.4 - 0.2 - 0.3 Other sources 3.4 3.7 3.0 2.5 3.3 1.7 3.2 Did not get this info 7.5 17.3 5.1 16.4 12.6 8.3 9.0 All respondents 100.0 100.0 100.0 100.0 100.0 100.0 100.0 Multiple response, n= 979 MSIKA Final Evaluation Page 29 kadale@africa-online.net The proportion of respondents in the FE HH survey that obtained market information from these two sources, accounting for any overlap, was 27.6%, with similar results for men and women. This shows progress from the baseline at 12.9% and the mid-term at 14.9%, but is short of the LoP target of 48.0%. The LoP achievement is 57.4%. Table 18: Indicator 18, Producers Accessing Market Info via their FBO, FE 2020 % of producers with access to current market information via their FOBs IB% MTE% FE FBO Governance Structures 12.9% 14.9% 27.6% Leadership skills 11.0% 14.1% 27.4% Cooperative Education 15.0% 16.0% 27.7% Composite of responses, n= 979 Although there has been an increase in market information that farmers receive through their FBOs, they still rely more on MSIKA for information than their FBOs. MSIKA is not a sustainable source of information and it is unclear if the FBOs will be able to sustain the flow of information post project as much of this has been supplied by MSIKA to the FBOs. In conclusion, there has been progress from the baseline on FBOs as a source of information, however this is short of the target. 3.1.8 COVID Impacts In the FE KIIs, the FBOs were asked about the effect of COVID on the FBO and its members. All FBOs reported COVID effects in their KIIs, notably: i. Low market prices locally due to export restrictions ii. Training suspension by V37, resulting in limited information access iii. Training suspension by government iv. Hampered data reporting activities v. Low sales and prices due to reduced market demand vi. Higher costs of transport and inputs which have raised production costs vii. More individualistic selling of crops by members The FBOs said: "Some buyers buy large volumes and export the vegetables but now they are grounded, they can’t buy because they can't export.” High performance FBO “Although we have had no COVID-19 case here in our community, we are still affected in our farming business. If I'm not mistaken, the last time the (MSIKA) extension officer came here was in October, 2019.” Medium performance FBO "Prices dropped for chillies to K 1,000 instead of the agreed K 1,500 per kilogram because their demand in Blantyre had also dropped due to lockdown restrictions there.” Low performance FBO Most FBOs did nothing in response, as they could not think what they could do. A few took some steps to mitigate the effects, as follows: i. Individual selling has increased due to fewer opportunities via the FBOs ii. Using cellphones to share information (two medium performing FBOs) iii. Listening to pre-recorded messages on the radio that MSIKA provided (one high performing FBO) MSIKA Final Evaluation Page 30 kadale@africa-online.net iv. Limiting meetings to small groups or leaderships only (one high performing FBOs, one medium performing FBO, four low performing FBOs) v. Waiting until buyers come to the farmgate (two high performing FBOs, one medium performing FBO) vi. Continuation of training and practical application through demonstrations According to one FBO: "We are now just meeting in more small groups so that we still progress forward." Medium Performance FBO The effects on FBOs have been noted in the reporting of several indicators, notably those on sales (indicators 21 and 23), where performance has been much lower than expected due to loss of formal buyers. FBOs have made some adaptions, however, the loss of the link with MSIKA, mainly through the reduced contact with FBCs, has affected planned activities such as training, access to information and market linkages. 3.2 Participant Producers Producer HHs that grow the target fruit and vegetable crops are the main participants of the MSIKA program. To be included as a participant producer for the FE survey, an individual must have been trained in basic agricultural production by MSIKA as of 31st May 2020. Based on this criterion, MSIKA provided a list of 30,922 trained producers. In addition to the HH survey, 63 qualitative KIIs were conducted with farmers, replacing the originally intended FGDs. These provide insights into some of the key changes. 3.2.1 MSIKA Producer Sample Frame This section provides a profile of the producers in the sample frame that were trained by MSIKA. Comparisons are made with the baseline when appropriate. Table 19 summarizes the profile of the producers by district, sex and crop. Producers are spread across five districts, with the proportional split per district ranging from the lowest at 16.9% in Ntcheu to the highest at 24.6% in Mchinji. At 31st May 2020, the beneficiary producers were members of 217 FBOs. MSIKA has trained more female producers (55.6%) than male producers (44.4%). Table 19: Beneficiary Producers, by District, Sex & Crop, FE 2020 District # of FBOs Individuals trained in basic agri-production Total Female Male Tomato Onion Potato Mango Dedza 56 5,890 3,666 2,224 3,166 1,454 4,630 763 Lilongwe 36 5,747 2,506 3,241 4,259 1,856 1,474 203 Mangochi 40 6,472 4,592 1,880 5,171 768 868 1,241 Mchinji 41 7,601 3,589 4,012 5,663 3,771 4,167 3,132 Ntcheu 44 5,212 2,827 2,385 3,981 1,240 1,347 2,202 Total 217 30,922 17,180 13,742 22,240 9,089 12,486 7,541 3.2.2 Producer Totals According to MSIKA data, the program reached 41,185 individuals throughout the four years of the MSIKA project implementation, which is 114.4% of the LoP target (Indicator 2). Of these, 18,433 were male (91.4% of LoP target) and 22,752 were female (143.6% of LoP target). The proportion of female participants is 55.2%. Therefore, the overall LoP target for indicator 2 has been met with over achievement on female beneficiaries, and a shortfall on male beneficiaries. MSIKA Final Evaluation Page 31 kadale@africa-online.net Indirect beneficiaries (indicator 3) are a function of the direct beneficiaries, so the achievement on indicator 3 is a multiple of indicator 2. The reported achievement is 189,451 against the LoP target of 165,600 (114.4%). Most of the project participants (39,744) are producers, who received short term agricultural productivity training (Indicator 4). MSIKA has overcome a slow start in reaching participants and has surpassed the target of 36,000 beneficiary producers achieving 110.4% of the target. 3.2.3 Final Evaluation Sample Profile This section provides a summary of information about the producers in the FE sample, which is compared to the baseline where that information is available and appropriate. These are summarised in Table 20 below, followed by an analysis of the differences on age and HH size in Table 21. Except for the sex of the respondent and relationship to the head of household (HHH), the demographics of the baseline and FE samples are very similar. Both baseline and FE respondents were on average between 40 and 43 years, with some education (93.8% FE versus 88.0% at baseline), and were married (86.2% FE versus 81.9% at baseline). Households in the FE sample were nearly all headed by males (96.2%) and had 5.8 members, while households interviewed at baseline had 5.6 members. Slightly more respondents (74.7%) identified as the HHH, compared to the baseline (70.3). For those respondents that were not the HHH, the relationship was mostly that of spouse. The majority of the FE sample was male (54.5%). This was a higher proportion of males than in the sample frame (44.4%) or in the baseline (50.3%). Overall, the datasets were similar for respondents’ marital status, household status and education. A further analysis was conducted on the sex of the respondent, where there was a superficial difference in the balance (baseline 50.3% male, FE 54.6% male), along with age and HH size. The analysis found that the FE (treated) and untreated (Impact baseline-treat arm) differed by about 0.02 standard difference (SD) in age, and by 0.15 SD in household size. Both ANOVA and linear regression analysis of the baseline and FE datasets indicate that sex was a significant factor [p-values<0.05] in predicting yield outcomes with the analysis showing that that there was positive association of yield and being male. The analysis further showed that only about 3-4% [R-squared] of the variance in both the baseline and FE could be explained by the factors listed in the table above strongly suggesting that these factors (age, sex and HH size) would not account for the high percentage of the changes that may be observed even if they were significant. The evaluation team undertook a further review to determine if there were any factors that explained why yields were higher for men and lower for women. The first analysis was to see if there were differences in the proportion of males and females that received various training types, however no significant differences were found. The second analysis looked to see if socio-cultural structural factors might have contributed to this. It was found that a higher proportion of male respondents (98.3%) were married compared to 71.6% of female respondents. Of the female respondents, 28.4% were divorced or separated, but very few men. Due to the small number of men who were not married, it was not possible to compare their relative performance, so the yield performance of married men was compared with married women. The gap in yield performance was still evident and significant, so cannot be explained by marital status. Overall, there is no explanation of the differences in yield between men and women from the data in this evaluation. MSIKA Final Evaluation Page 32 kadale@africa-online.net Table 20: Comparison of Baseline vs FE Samples Demographic Baseline FE Male Female All N Male Female All N Sex of respondent 50.3% 49.7% 100.0% 590 54.6% 45.4% 100.0% 979 Age(years) 43 40 42 590 42 41 42 979 Household size n/a n/a 5.6 590 n/a n/a 5.8 979 Marital status Married 93.3% 70.3% 81.9% 590 98.3% 71.6% 86.2% 979 Divorced, separated, widowed, never married 6.7% 29.7% 18.1% 590 1.7% 28.4% 13.8% 979 Education level of respondent Standard 1-8 63.0% 62.8% 62.9% 590 61.5% 69.8% 65.3% 639 Form 1-4 27.3% 15.0% 21.2% 590 32.1% 16.0% 24.8% 243 Further education 1.3% 0.3% 0.8% 590 0.9% 0.0% 0.5% 5 None 6.7% 17.4% 12.0% 590 2.6% 10.6% 6.2% 61 Adult literacy 1.7% 4.4% 3.1% 590 2.8% 3.6% 3.2% 31 Head of household Head of household is respondent 95.3% 45.1% 70.3% 590 97.0% 47.7% 74.7% 731 Head of household is other 4.7% 54.9% 27.7% 590 2.6% 46.6% 22.6% 221 Head of household is joint n/a n/a n/a 590 0.4% 5.6% 2.8% 27 relationship to H0fH-spouse 7.1% 91.3% 84.6% 590 79.0% 97.0% 97.0% 214 relationship to H0fH-other 92.9% 8.7% 15.4% 590 21.0% 1.9% 3.0% 7 N=979 Table 21: Check Balance of Confounders Using Propensity Score Matching Covariates Mean in treated Mean in untreated Standardized diff Age 41.84 41.54 0.02 HH size 5.82 5.29 0.15 3.2.3.1 Producers’ Engagement with MSIKA To provide a picture of their engagement with MSIKA, FE survey respondents were asked (prompted) to state which MSIKA activities they have been involved in, in the last 12 months. 86.2% respondents reported receiving training in improved horticulture practices from MSIKA or a MSIKA lead farmer in the last 12 months This compares to 93.2% at the MTE. In addition, 81.5% reported receiving training by MSIKA/lead farmers in horticulture PHH (MTE 91.8%) and 87.8% were trained in VSL (MTE 87.9%). Financial literacy training was reported at 73.5% (MTE 73.8%), gender equality at 80.0% (MTE at 73.4%), and marketing at 79.2% (MTE 82.8%). A majority of respondents reported that they were linked to a micro-finance institution (MFI) at 63.1% (MTE 64.6%) and close to half (49.8%) were linked to markets (MTE 61.9%). It appears likely that the lower numbers of producers linked with finance and market in the final year might be attributed to COVID restrictions in the country and ceasing of MSIKA fieldwork. Summary responses are set out in Table 22 below. MSIKA Final Evaluation Page 33 kadale@africa-online.net Table 22: MSIKA Activities Reported by FBO Members in MTE & FE HH Surveys Which of the following activities from Land O’Lakes /MSIKA have you participated in? Activity MTE FE Male Female Total Male Female Total % % % % % % a. Training in improved horticulture practices by MSIKA staff or a lead farmer 93.8 92.8 93.2 85.8 86.7 86.2 b. Training in horticulture post-harvest handling by MSIKA staff or a lead farmer 91.1 92.3 91.8 79.8 83.6 81.5 c. Training in marketing 81.7 83.5 82.8 79.1 79.3 79.2 d. Training in financial literacy 73.5 74.0 73.8 74.4 72.5 73.5 e. Training in gender equality 77.4 70.7 73.4 81.3 78.4 80.0 f. Training/membership of a village savings and loan group 85.6 89.5 87.9 85.6 90.5 87.8 g. Been linked to an MFI 62.3 66.1 64.6 63.0 63.3 63.1 h. Been linked to markets to sell horticulture products 59.5 63.5 61.9 47.3 52.9 49.8 i. Participated in international/District trade fair 28.4 22.9 25.1 17.0 14.4 15.8 j. Any other? 0.4 - 0.2 1.9 0.2 1.1 Total 257 389 646 535 444 979 N=646 & 979, multiple response possible 3.2.4 Agricultural Practices 3.2.4.1 Knowledge Due to the change in interviewing methodology, it was agreed with USDA that the questions on knowledge of agricultural practices would not be asked. Gathering the information on use/application of the practices was more important and, applying a practice implies that there was knowledge of the practice(s) applied. This decision affects Indicator 14 (Percentage of producers who can recite five or more improved agricultural techniques and technologies). At mid-term when data for indicator 14 was collected, the level of unprompted knowledge of five or more practices was 56.7%. The revised target for LoP was 66.0%. The qualitative interviews with farmers focused more on training and application than knowledge, so the detail is covered in the application of practices section below. 3.2.4.2 Application of Practices Indicator 7 is the number of individuals who have applied new techniques or technologies, as a result of USDA assistance. To measure this indicator required a review of the baseline instrument to enable an appropriate comparison, as MSIKA adopted a different approach to train producers than anticipated at the baseline. This change was to train producers in a wide range of practices applicable across many crops, plus some specific practices for particular crops, rather than training specific to each crop. There were additional practices introduced into the training than were measured at baseline, so a direct comparison of these is not possible. This question in the baseline and the FE survey was prompted, with respondents asked about all the practices for one selected crop (28 practices for onion and tomato and 19 practices for potato and mango). The full list of practices and percentage of respondents applying can be found in Annex 5. MSIKA Final Evaluation Page 34 kadale@africa-online.net The finding on application of practices is that 99.5% (all but five out of 1,074) of responses across the four crops had applied one or more new practice in the period August 2019-July 2020. This is a very high achievement level equating to 30,778 of the project population, which is all except 144 beneficiary producers. The application rates of men and women were very similar with 99.7% of men applying one or more new agricultural practice, and 99.4% of women applying them. The LoP target for applying one or more new practice (indicator 7) is 31,680 based on an expected 36,000 individuals trained in agricultural practices and food security (Indicator 4). This is a rate of 88.0% of those trained using at least one new agricultural practice. Although the actual application rate is very high (99.5%) for those that were trained, the number of producers trained in basic agriculture at 30,922 is lower than anticipated, as the reported achievement of indicator 4 includes marketing and financial training for producers, as well as training of non-farmers in basic agriculture (extension staff). As the FE population was lower than the total of individuals trained in basic agricultural production, then the total number of people applying new practices (Indicator 7) is also relatively lower, at 30,778. This is 97.2% of the overall LoP target. The male target was exceeded (107.1%) and there was a shortfall on female (90.5%). Based on 99.5% of the sample applying the practices, a slightly higher achievement on the number of trained farmers overall would have resulted in the LoP target being met. Further training was disrupted by COVID, but training subsequent to finalising the sample frame suggests the LoP target for indicator 7 would likely have been exceeded. Table 23 below shows the application of practices across the eight disaggregate categories. The achievement ranged between 68.8% to 98.3% of respondents applying one or more. The highest application rates were for disease management (98.3%), pest management (97.6%), and soil related fertility and conservation (96.6%). The lowest rates were for crop genetics (79.9%) and climate mitigation and adaptation (68.8%), though these are still good scores. The results for men and women were very similar. Details of the individual practices are in Annex 5. The baseline and FE surveys had practices that were common to both surveys, but there were some practices in the baseline that were not asked in the FE survey, such as green manuring, and some practices asked in the FE survey that were not in the baseline, such as staking young trees and intercropping. This means it is not valid to compare the FE and baseline disaggregated categories. Table 23: Application of New Agricultural Practices per Category, FE Survey, 2020 # of individuals that applied one new techniques & technologies as a result of USDA assistance Category/Disaggregate FE # applying FE % FE population applying Total applying 1,069 99.5 30,778 Male 579 99.7 13,542 Female 490 99.4 17,236 New 821 76.4 23,638 Continuing 248 23.1 7,140 Crop Genetics 858 79.9 24,703 Pest Management 1,048 97.6 30,173 Disease management 1,056 98.3 30,404 Soil-related fertility and conservation 1,038 96.6 29,886 Irrigation 878 81.8 25,279 Water management (non-irrigation based) 963 89.7 27,726 Climate mitigation or adaptation 739 68.8 21,277 Other 903 84.1 25,999 Total w/one or more improved techniques or technologies 1,069 99.5 30,778 MSIKA Final Evaluation Page 35 kadale@africa-online.net An overall achievement rate of 99.5% of respondents applying one or more new practices is very high, though it masks other aspects of MSIKA’s performance. The consultants calculated the mean number of new improved practices per beneficiary producer was 15.8. This compares well to the MTE with a mean of 9.8 practices, suggesting further deepening of the application of practices. This may also be a factor in the relatively high increases in yields reported in section 3.2.7 below. Table 24 indicates the number of new practices applied by crop. This ranged from implementing no practices to a maximum score of 28 new practices applied on tomato and on onion, as well as a maximum score of 19 new practices for potato and for mango. The mean number of new practices applied per producer was highest for tomato (20.9), onion (20.6) and potato (13.5). Mango had the lowest number of new practices applied per producer (8.1). The low value for mango is attributed to most farmers not managing mango because they inherited these trees and they produce without any effort. Field crops more clearly require their effort and time in to produce. Table 24: Application of New Practices by Crop at FE, 2020 Crop Total Producers Total Practices applied by all producers Mean practices applied per producer Minimum practices applied per producer Maximum practices applied per producer % applying none Tomato 276 5775 20.9 11 28 - Onion 264 5429 20.6 9 28 - Irish Potato 266 3600 13.5 4 19 - Mango 268 2177 8.1 - 19 1.8 Total 1060 1074 15.8 - 28 1.8 The detailed application by crop is set out in Annex 5 and provides the detail for each practice, including the percentage that are using it and the comparison with the baseline usage rates. This table covers 41 practices, across the seven crops with baseline, FE and difference scores. Across the field crops, ridge and plant spacing, irrigation, selection of best seedlings and use of nursery generally scored the highest. For mangoes, pruning scored the highest, followed by composting water capture and digging of planting holes prior to planting which all had a same score. Use of fish soup, testing soil acidity, and use of ash/lime generally scored poorly across all the four crops. Specifically, for mangoes, scores were lower for grafting, budding, draining excess water and deflowering of first flowers scored poorly. While these are practices MSIKA was to promote more following the mid-term findings which are similar to the FE findings, it is a possible that the low scores were affected by the abrupt reduction in activities and the inability of farmers to conduct meetings at FBO level due to COVID. There were roughly ten practices where there has been an increase in use of 20% or greater between baseline and FE across at least for the three field crops. These practices were: composting, manuring, mulching, irrigation, soil and water conservation, selecting varieties and spraying. 20% or greater also applied water capture and pruning for mango. Overall, there has been an improvement from baseline to FE across all the crops except ridging for potatoes with a 5.5% decrease from the baseline. The latter may be a function of it already being a very common practice and statistical variation. Participant producers may have been using practices before MSIKA or leant them from other sources, so that the impact of these is not exclusively attributable to MSIKA. MSIKA Final Evaluation Page 36 kadale@africa-online.net Asking about practices used for the first time in the last year, combined with the information on source, enables clearer attribution to MSIKA for the subsequent results. Respondents were asked in the survey what practices they had used for the first time in the last year (August 2019-July 2020). In general, the scores for first use of practices were high, across the four field crops. It is not surprising that for some of the practices, respondents reported to have not used it for the first time in the last 12 months since the sample population took into account all the individuals trained since project commencement. This means, it is likely that some individuals have implemented the practices for the first time after MSIKA training, but not within the last 12 months. Table 25 below sets out the first use of practices for the four field crops. Across the field crops, composting, soil and water conservation, choosing the variety, uprooting infected plants and use of fertiliser scored consistently well, with fish soup and testing soil acidity scoring generally lower. For tomato, 11 practices have been applied for the first time by over 40% of respondents. Composting was applied for the first time by over 50% of respondents. There was a high uptake of practices relating to uprooting infected plants, choosing the variety and use of recommended plant and ridge spacing. The linked practices of testing for acidity and use of lime/ash scored poorly, along with the use of fish soup. For onion, 11 practices were applied for the first time by over 40% of respondents. Over 50% of respondents applied scouting for pests and diseases as well as sterilizing nursery beds before planting out. As with tomato, practices relating to composting, use of recommended plant and ridge spacing scored highly. Testing for acidity and draining excess water scored poorly, as did use of fish soup. For potato, 5 practices were reportedly applied for the first time by over 40% of respondents, with the highest scoring being selection of the best seed potato (44.4%), uprooting infected plants and burning (42.9%) and choosing the variety (42.9%). No practices were used by over 50% of respondents. The scores for potato were generally lower than for tomato and onion. The use of fish soup scored poorly. For mango, 2 practices were used for the first time by over 40% of respondents, but no new practices were used for the first time by over 50% of the respondents. The lowest uptake by respondents was for budding, fish soup and scouting for pests and diseases, all at less than 20%. MSIKA Final Evaluation Page 37 kadale@africa-online.net Table 25: Use of Practices for the First Time, FE Survey, 2020 Practice used Tomato Onion Potato Mango % % % % 1 Composting 51.1 49.5 37.1 40.4 2 Manuring 42.5 41.5 25.9 31.1 3 Ridging 36.7 33.0 31.7 n/a 4 Mulching 43.0 40.1 26.8 25.7 5 Crop rotation 36.7 38.2 27.3 n/a 6 Minimum tillage 31.7 29.2 n/a n/a 7 Planting seeds in a nursery before planting out 38.5 40.1 n/a n/a 8 Staking 37.6 n/a n/a n/a 9 Succession planting 33.0 n/a 23.4 n/a 10 Using irrigation 30.8 34.0 24.9 24.6 11 Soil and water conservation 39.4 36.8 34.1 42.1 12 Draining excess water 30.3 24.5 n/a 24.0 13 Testing soil acidity 18.1 21.2 n/a 19.7 14 Adding lime or ash to soil before planting to reduce acidity 24.0 27.8 n/a 25.7 15 Choosing the variety 47.1 42.5 42.9 n/a 16 De-suckering or removing unwanted side-shoots 41.2 n/a n/a n/a 17 Spraying for pests and disease 35.3 33.5 27.8 25.1 18 Using recommended plant and ridge spacing 46.2 44.8 42.0 n/a 19 Using recommended fertilizer and application rates 43.4 44.3 38.0 n/a 20 Sterilizing nursery beds before planting 38.7 81.8 n/a n/a 21 Scouting for pests and diseases 32.1 69.3 40.0 17.5 22 Uprooting infected plants and burning 48.9 38.2 42.9 25.7 23 Sowing seed in row/groove nursery 42.1 42.5 n/a n/a 24 Using fish soup / sugar solution as bait for insects 15.4 17.0 18.0 15.8 25 Planting in sunken beds 36.7 41.0 n/a - 26 Hardening off before planting out 40.3 38.7 n/a n/a 27 Selecting the best seedlings for planting out 43.4 38.7 44.4 n/a 28 Intercropping 30.8 29.2 22.9 29 Green manuring n/a n/a n/a n/a 30 Water capture n/a n/a n/a 39.9 31 Earthing up n/a 34.9 25.4 n/a 32 Clipping plant ends n/a 35.8 n/a n/a 33 Using raised beds n/a 34.0 n/a n/a 34 Chitting n/a n/a 33.7 n/a 35 Pruning n/a n/a n/a 42.6 36 Grafting n/a n/a n/a 21.9 37 Budding n/a n/a n/a 15.8 38 Staking young trees n/a n/a n/a 32.8 39 Digging of planting holes prior to planting n/a n/a n/a 31.7 40 Deflowering of first flowers n/a n/a n/a 26.2 Total 221 212 205 183 From the qualitative interviews, growers of all four crops were able to identify many of the practices they were trained in and that they were able to apply. Between them, tomato growers identified 25 practices they had applied of which spraying for pests and diseases was the most commonly applied and crop rotation, intercropping, identifying the type of the seed that buyers are looking for, applying lime and ash to control salinity, uprooting infected plants and burning. MSIKA Final Evaluation Page 38 kadale@africa-online.net Onion farmers identified 28 practices in the qualitative interviews. The most commonly mentioned practice was application of recommended fertilizer with the least applied practice being making beds sunken or raised, uprooting infected plants, choosing the variety for the growing characteristics and market that aiming for, clipping plant ends to allow seedlings make fresh sprouts, weeding and earthling up. The Irish potato growers noted 13 practices from the training that they were able to apply. The most commonly applied practice was using recommended fertilizer, while soil and water conservation by planting vetiver, selection of best seed potatoes, uprooting infected plants, and land preparation by removal of wastes were the least commonly applied. The mango farmers gave 12 practices that they had applied. The most commonly applied practice was manuring and the least applied were scouting for pests and diseases, fertilizer application, weeding, and grafting. It was clear from the KIIs that the farmers had applied a wide range of practices and that these have benefited them. "I have seen that these practices are of great profit to my farming.” Ntcheu, male tomato farmer "I was able to use less fertilizer because I depend mostly on the use of manure". Mangochi male onion farmer. "I like the benefits am getting through practicing what I was trained on, I'm making profits and can support my family now." Ntcheu, female potato grower "The training proved that mango production can be taken as a business and the proceeds from the business can be used to buy food.” Ntcheu farmer, female. Respondents were asked why they had not used the ‘other’12 practices between August 2019 to July 2020. Due to the shortening of the questionnaire, respondents were asked the overall reasons why they did not use some of the other practices, rather than on each practice not used, as was asked at mid-term. ‘Other’ therefore refers to the practices not used. This was a multiple response question for which the enumerator classified the single or multiple responses given rather than reading out the list of possible responses. Respondents were asked to give more than one response (“any other reasons?”). The question was asked for the overall practices relating to a specific crop and are reported below by crop. The most common reason across all crops was that the respondent ‘did not know it’, with all crops having more than 44% of respondents saying they did not know some of the practices. This is an improvement on the mid-term where over 90% of respondents said they did not know some of the practices, suggesting that there has been an increase in knowledge overall. This was followed by ‘did not fully understand it’ (13.9 - 30.2%), ‘need did not arise’ (12.0 – 25.4%, ‘inadequate resources’ (6.8 – 20.5%) and it was ‘expensive to use’ (9.8% - 15.3%). The ‘did not know’ response might indicate that some practices were not covered in the training, or that these were forgotten by the respondent. It would be surprising if respondents knew all the practices, as there were many and respondents may have been growing two crops,13 so there was a great deal to ‘know’. Respondents may have lost concentration during the training or forgotten the practice, especially as many had limited education. 12 Described as ‘other’ as the respondents had been read a list of practices and stated the ones they used, so this refers to the ones they did not use. 13 The mean number of crops per farmer was 2.18. MSIKA Final Evaluation Page 39 kadale@africa-online.net The second most common response was that ‘did not fully understand it. The third most common response was that the ‘need did not arise’ which can be the case for pest and disease management. The ‘inadequate resources’ and ‘expensive to use’ responses suggest that that some practices are too expensive for some respondents to apply. This is inevitable, given the poverty of many respondents. The least commonly given reasons were: ‘inefficient’, ‘too difficult for farmers’, ‘not confident to use’ and ‘not confident it will work’. This means that these practices are not seen as inefficient, too difficult, or confident that they will work or to use them. In other words, the practices are mainly appropriate. Table 26 sets out the reasons given for not using practices by crop. Male and female responses were very similar. Table 26: Why Other Practices are not Used, by Crop, FE 2020 Overall, what are the reasons why you did not use the other improved practices? (Unprompted) Reasons Tomato Onion Potato Mango % % % % Did not know it 56.9 56.5 44.0 59.0 Did not understand it 23.2 25.2 13.9 30.2 Need did not arise 22.1 22.1 12.0 25.4 Inadequate resources 17.4 19.8 6.8 20.5 Expensive to use it 13.8 15.3 9.8 13.1 Other 8.7 9.9 3.0 6.3 Time consuming 7.6 7.6 4.5 6.0 Not confident it will work 7.2 6.9 22.2 5.2 Too difficult for farmers to do it 6.2 6.1 21.8 4.5 Not confident I can use it properly 3.6 3.8 24.8 4.1 Inefficient 1.8 3.4 11.3 3.4 Sample size/ difference 276 262 266 268 N= 1,074 3.2.5 Crop Inputs The FE HH survey did not cover crop inputs so as to reduce the survey length. Information on inputs was included in the MTE. In summary from the MTE, a high proportion of respondents (60-80% range) were using fertilizers, insecticides, watering cans, sprayers, certified seed, compost/manure and fungicide, particularly for tomato, onion and potato. These showed improved uptake in nearly all areas. The most frequent source of inputs was ‘own/personal’, followed by ‘agro-dealers’ at 74-80% of respondents for tomato, onion and potato. In terms of ease of getting inputs, for tomato, onion and potato, between 67-70% of respondents at mid-term found accessing inputs ‘somewhat hard’ or ‘very hard’. Growers of mango reported easier access to inputs, but this was partly because around a third did not buy any inputs at all. Overall, the mid-term reinforced the importance for MSIKA to work with agro-dealers to continue improving access. MSIKA Final Evaluation Page 40 kadale@africa-online.net 3.2.6 Land Area Under Improved Practices Indicator 6 is the land area under improved growing techniques and technologies (‘growing practices’) as a result of USDA assistance. The FE survey asked for the improved practices applied and the area of land on which those practices were applied. The land area was measured in hectares (ha) for the 12-month period from August 2019-July 2020, over up to three separate planting and harvest cycles for tomato, onion and potato and one cycle for mango. All cycles were totalled to give the total land area. It was agreed after consultation with USDA to exclude questions related to the disaggregates for this indicator to reduce interviewing time. The total land area for respondents on which one or more improved practices was applied across all growing/harvest cycles14, was 276 ha. The mean land area on which improved growing practices were applied was 0.26 ha per respondent from August 2019 – July 2020. If this mean is applied to the FE population of 30,778, improved growing practices were applied on an estimated 7,909 ha. The reduction in total land, as well as the mean land area since mid-term due could be due to the reduced number of crops in the fourth year. To enable a comparison with the FE, the MTE was also revised by taking into account the land under improved practices for the four crops, rather than the original seven. The mean land area on which improved growing practices were applied was 0.36 ha per respondent from July 2018 – June 2019. When this mean is applied to the MTE population of 8,442, improved growing practices were applied on an estimated 3,045 ha. Overall, there has been progress from the MTE to the FE and the indicator has surpassed its LOP target. This reflects the decisions made to reduce activity due to the shortfall in funding. Table 27 sets out the land area under improved practices. MSIKA reported 6.3 ha of demonstration plots on which practices were applied in Year 4, down from 12.77 in Year 3. This reflects the decisions made to reduce activity due to the shortfall in funding. Table 27: Land area in ha under improved practices, MTE and FE HH surveys Number of hectares of land under improved techniques or technologies as a result of USDA assistance Crop MTE FE Overall 3,045 7,909 Tomato 1,362 2,506 Onion 276 2,048 Potato 1,265 2,744 Mango 142 610 3.2.7 Production A key purpose of the training and other activities is to increase yields for producers. This is measured through Indicator 1, which is the Percentage change in yield of beneficiary producers as a result of USDA assistance. The baseline values were tomato at 2,617 kgs/ha, onion at 3,575 kgs/ha, potato at 3,329 kgs/ha and mango at 4,347 kgs/ha. Following the MTE, the LoP targets were 14 Land under each cycle was counted separately and summed to give the total. MSIKA Final Evaluation Page 41 kadale@africa-online.net revised to overall (+26%), men (+29%), women (+24%), tomato (+25%), potato (+29%), onion (+30%) and mango (+21%). The baseline only had data for one growing cycle, so the FE survey data is reported for the comparable production and cycle only. One of the recommendations which was made in the MTE was to focus on individual crops when reporting the yields, as it is difficult to aggregate them and compare them over time due to different weightings relating to number of farmers growing that crop at that time. The consultants therefore report on the individual crop comparisons. As noted in the methodology section, the baseline yield values were revised to use a consistent methodology. The baseline calculations for six of the seven crops (all but tomato) did not remove outliers, which skewed the yield values. The consultants set the maximum yields using FAO maximum expected yield assuming application of GAP. These were used in the analysis of the FE, so the baseline and FE were comparable. The FE HH survey findings on yield are standardised into kgs/ha to allow comparisons between crops and with the baseline. Median yields15 have been calculated as well as means, as these tend to moderate more extreme values and can be useful to review. The crop with the highest yield in kgs/ha was mango with a mean of 7,078 kgs/ha followed by potato at 4,931 kg/ha. Onion and tomato had a mean 4,896 kg/ha and 4,418 kg/ha respectively. All median values were lower than their equivalent means, notably tomato at 2,635 kg/ha. The comparison of median values shows the range for all crops is between 2,635 kgs/ha and 5,216 kgs/ha Comparing male and female mean yields, males generally had higher yields than female producers. If median values are used, the gaps are reduced, but still a bit higher in favor of men, except mango with women having higher yields. Table 28 sets out mean and median yields by crop standardized to kgs/ha and weighted. Table 28: Producer Yields in kg/ha, FE HH survey, 2020 Yield Kg/ha FE Male Female Overall Mean Median Sample Mean Median Sample Mean Median Sample Tomato (kg/ha) 5,369 3,162 154 3,207 1,897 121 4,418 2,635 275 Onion (kg/ha) 5,871 4,320 146 3,647 1,956 114 4,896 3,260 260 Potato (kg/ha) 5,639 4,323 133 4,133 2,849 118 4,931 3,751 251 Mango (kg/ ha) 7,327 4,890 121 6,853 5,433 134 7,078 5,216 255 N=1,074 As noted in the methodology, the baseline results were updated during the mid-term to remove outliers above the FAO maximum yields per ha. Table 29 sets out the revised baseline yields, with a focus on only the four crops which were of focus in year four of MSIKA project implementation and sets out the comparison between the FE survey and baseline with the outliers removed from both using the same methodology. 15 Medians can be useful measures of the average in a dataset that has zero as the lowest value, but an unlimited maximum value, such that higher values can distort the mean average. MSIKA Final Evaluation Page 42 kadale@africa-online.net Table 29: Producer Yields in kg/ha, Revised Baseline 2017 Yield in Kg/Ha Impact Baseline Male Female Overall Mean Median Sample Mean Median Sample Mean Median Sample Tomato (kg/ha) 3,305 1,537 173 1,858 880 157 2,617 1,217 330 Onion (kg/ha) 4,358 2,748 58 2,061 797 30 3,575 2,005 88 Potato (kg/ha) 3,946 2,983 136 2,684 1,637 130 3,329 2,341 266 Mango (kg/ ha) 4,017 2,580 112 4,660 2,886 118 4,347 2,737 230 N=686 Compared to the revised baseline, the FE had higher mean yields across all the crops. The percentage changes for the crops were as follows; tomato (+69%), onion (+37%), potato (+48%) and mango (+63%). Similarly, the percentage changes for median values are higher, notably tomato at 116%. These are set out below. Table 30: Yield Comparison, FE vs baseline by Crop Crop Baseline FE - Overall FE vs Baseline Mean Median Mean Median Mean Median Tomato (kg/ha) 2,617 1,217 4,418 2,635 69% 116% Onion (kg/ha) 3,575 2,005 4,896 3,260 37% 63% Potato (kg/ha) 3,329 2,341 4,931 3,751 48% 60% Mango (kg/ ha) 4,347 2,737 7,078 5,216 63% 91% COVID potentially had an effect on producer yields, so comparison with the baseline is also made based on those producers that said they were not affected by COVID. Table 31 below sets out the comparison of baseline and FE yields for those not affected by COVID. The FE mean yields for those not affected by COVID16 compared to the baseline show increases in mean yields for tomato (+72%), onion (+42%), potato (+49%) and mango (+64%). The change in the yields compared those using the whole sample is quite small, though for all four crops the yields for those not affected by COVID were higher by up to five percentage points. This is consistent with the expected impact from COVID, which occurred in the period when some farmers were planting and growing, such that governmental restrictions made access to inputs more challenging due to movement restrictions and market closures. Median values for those not affected by COVID have a similar pattern to the whole FE sample. Overall, the comparison of the FE group not affected by COVID, shows progress in yield increases for the crops. The evaluation team reviewed the yield difference between those affected and not affected by COVID-19 and related this to the baseline yields. Overall, the yields were higher for those not affected by COVID compared to those affected by COVID for tomato, potato and mango. The differences are set out in the table below and were significant. 16 936/1074 (87.2%) were not affected by COVID in their production (excluding outliers). The main effect of COVID was on sales. MSIKA Final Evaluation Page 43 kadale@africa-online.net Table 31: Producers Affected/Not Affected by COVID, Yield Comparison vs Baseline Crop Comparison of % changes from Impact Baseline Not affected by Covid-19 Affected by Covid-19 Overall Mean Median Mean Median Mean Median Tomato (Kg/ha) 72.1% 118.7% 49.1% 116.5% 68.8% 116.5% Onion (Kg/ha) 42.3% 73.5% 72.9% 64.1% 36.9% 62.7% Potato (Kg/ha) 49.4% 60.4% 37.9% 47.7% 48.1% 60.2% Mango (Kg/ha) 64.3% 86.6% 30.5% 98.5% 62.8% 90.6% Table 32 below compares male and female yields for those not affected by COVID. There was a slight positive yield changes across all the crops for men, while for women, tomato and potato had reduced slightly. Table 32: Yield Comparison by Sex, FE vs baseline Crop FE Not affected by Covid 19 vs Baseline Males Females Mean Median Mean Median Tomato 67% 122% 71% 101% Onion 39% 60% 91% 151% Potato 44% 45% 52% 74% Mango 83% 90% 49% 84% As there are only minor differences between those not affected by COVID and the overall sample, the consultant reports means for the overall population in the summary indicator table (see Annex 1) along with the revised baseline with outliers removed. It is worth noting overall that tomato (31.7%), onion (25.9%) and potato (34.7%), account for more than 92% of the production by land area where new practices were applied within the reporting time period (August 2019 to July 2020), and so are the most important crops for MSIKA. With positive increases in mean yields from 37% to 69% across the four crops, MSIKA has surpassed the LoP revised targets for each crop. 3.2.8 Post-Harvest Handling Knowledge and Application PHH knowledge was not covered in the FE HH survey so as to shorten the interview. Knowledge is a steppingstone to application, and can be assumed to be present from a practice(s) being applied, so more reliance is placed on the application than knowledge in this final evaluation. Knowledge was explored in the MTE and the details below are drawn from the MTE findings. The qualitative interviews with farmers mainly covered the application of practices, so are covered in the next section. The findings on PHH knowledge are set out in the section immediately below, followed by those on PHH use in section 3.2.8.2. 3.2.8.1 Knowledge of PHH This section summarised key points from the mid-term HH survey. For tomato, onion and potato, the mid-term unprompted knowledge scores were around 20%, suggesting that there was scope to improve knowledge. The mid-term unprompted PHH practices scores for mango were generally very low and below those for the field crops. MSIKA Final Evaluation Page 44 kadale@africa-online.net At mid-term, ‘MSIKA Extension worker/lead farmer’ was consistently the highest source of PHH information for field crops with a wide margin over the next highest source, which was either ‘always known’ or a ‘relative/other farmer’. For mango, ‘always known’ was the most common source for respondents, which is interpreted to be passed down knowledge. As noted earlier, the baseline question was asked as a prompted question, so it is not valid to compare the mid-term findings to it. 3.2.8.2 Use of PHH practices Respondents were asked in the FE HH survey which practices they used from a prompted list of all PHH practices per crop. The findings at FE are set out in Table 33 with the baseline findings in Table 34 below. The highest scores on PHH use across all the four crops were for ‘grading by size, color and shape’ ‘ sorting by variety, size/maturity, damaged, ripeness’ and ‘packing - loading into appropriate containers for storage or transportation’. The lowest score was ‘lifting potatoes with a fork/prong, not a hoe’ which only 11.3% of the sample had used in the last year. This is not surprising as most farmers would most likely prioritize buying a hoe as it can be used for several crops compared to crop specific tools like a fork or prong. As farmers specialise and want to increase quality and yield, these tools would likely become more relevant. There were several changes to the FE questions that were made at mid-term to reflect the practices MSIKA was teaching. These did not appear in the baseline or were worded differently, making direct comparisons more difficult on these specific practices. Having noted this, overall, the use of PHH practices for the four crops has increased across most PHH practices compared to the baseline, even though baseline scores were quite high to start with. For tomato, the scores are particularly impressive as they are all over 83%, with most over 90%. For onions, the lowest score is 73.5%, with most scores over 75%. For potato, apart from a very low 11.3% for lifting using a fork/prong, four scores are over 90% and the next lowest is 66.2%. The only score for mango lower than in the baseline was the use of a special harvesting pole (FE 48.5%, baseline 70.1%). Other than storing in crates (a big investment), and catching before it falls (which is difficult), all the other practices scored over 82% MSIKA Final Evaluation Page 45 kadale@africa-online.net Table 33: Use of PHH Practices by Crop, FE HH survey 2020 Practices used (Prompted) Tomato Onion Potato Mango % % % % Timely harvesting not too early so immature and not too late so over-ripe 90.2 n/a n/a n/a Harvesting & selling green/fresh onions with stalks on n/a 76.5 n/a n/a Harvesting when plant tops wilt and start to wither; slashing tops close to soil to cure in soil a day before harvesting n/a n/a 68.4 n/a Harvesting when ripening just started and color begins to change n/a n/a n/a 86.9 Grading by color, size and shape 89.5 87.1 96.2 87.3 Storing in cool places out of the sun prior to sale 96.0 n/a n/a 90.7 Sorting by variety, size/maturity, damaged, ripeness 92.8 87.1 96.6 90.7 Packing - Loading into appropriate containers for storage or transportation 83.3 73.5 92.1 88.4 Not over-filling baskets & bags or using very big bags, as this damages product at the bottom 93.5 84.1 n/a 82.5 Handling carefully to avoid damage 97.8 n/a 94.7 91.0 Not stacking more than three containers high to avoid crushing tomatoes in lower containers. 92.8 n/a n/a n/a Adding soft dry grass as soft litter 92.0 n/a n/a n/a Storing in crates to prevent damage in transporting n/a n/a n/a 53.4 Lifting when tops have bent over & dried; n/a 76.1 n/a n/a Drying (curing) onions in the sun or under a shade before storing to improve in keeping quality n/a 74.6 n/a n/a Trimming dried stems and roots before packing n/a 73.5 n/a n/a Lifting potatoes with a fork/prong, not a hoe n/a n/a 11.3 n/a Rubbing off surface dirt after harvest; or after drying & curing n/a n/a 72.9 n/a Keeping clean potatoes in a cool dry place to cure or heal surface damage, before packing, storage & selling n/a n/a 89.8 n/a Harvesting by climbing the tree n/a n/a n/a 87.3 Harvesting by using a special harvesting pole n/a n/a n/a 48.5 Catching fruit before it falls using bags or spreading sheets n/a n/a n/a 66.8 Using night vented sheds for short term storage for selling outside main season n/a n/a 66.2 n/a Washing to improve appearance n/a n/a n/a 84.0 Sample size 276 264 266 268 Multiple response possible MSIKA Final Evaluation Page 46 kadale@africa-online.net Table 34: Use of PHH Practices Across the Four FE crops, at Baseline Practices used (Prompted) Tomato Onion Potato Mango % % % % Timely harvesting not too early so immature and not too late so over-ripe 94.1 n/a n/a n/a Harvesting & selling green/fresh onions with stalks on n/a 60.6 n/a n/a Harvesting when plant tops wilt and start to wither; slashing tops close to soil to cure in soil a day before harvesting n/a n/a n/a n/a Harvesting when ripening just started and color begins to change n/a n/a n/a 86.8 Grading by color, size and shape 71.2 58.5 82.7 48.0 Storing in cool places out of the sun prior to sale 82.0 n/a 68.3 44.6 Sorting by variety, size/maturity, damaged, ripeness n/a n/a n/a n/a Packing - Loading into appropriate containers for storage or transportation 63.4 55.3 53.2 34.3 Not over-filling baskets & bags or using very big bags, as this damages product at the bottom 73.2 55.3 64.0 43.1 Handling carefully to avoid damage 76.1 n/a 70.1 52.0 Not stacking more than three containers high to avoid crushing tomatoes in lower containers. 69.9 n/a n/a n/a Adding soft dry grass as soft litter 70.6 n/a n/a n/a Storing in crates to prevent damage in transporting n/a n/a n/a 9.3 Lifting when tops have bent over & dried; n/a 60.6 66.5 n/a Drying (curing) onions in the sun or under a shade before storing to improve in keeping quality n/a 27.7 n/a n/a Trimming dried stems and roots before packing n/a 51.1 n/a n/a Lifting potatoes with a fork/prong, not a hoe n/a n/a 6.8 n/a Rubbing off surface dirt after harvest; or after drying & curing n/a n/a 36.7 n/a Keeping clean potatoes in a cool dry place to cure or heal surface damage, before packing, storage & selling n/a n/a n/a n/a Harvesting by climbing the tree n/a n/a n/a 88.7 Harvesting by using a special harvesting pole n/a n/a n/a 70.1 Catching fruit before it falls using bags or spreading sheets n/a n/a n/a 35.3 Using night vented sheds for short term storage for selling outside main season n/a n/a n/a n/a Letting lifted potatoes get a dry surface before packing and selling n/a n/a n/a n/a Washing to improve appearance n/a n/a n/a 43.1 Sample size 306 94 278 204 Multiple response possible Respondents were asked what PHH practices they used for the first time in the last year. The baseline did not have questions related to first time usage of PHH practices, as that would not have made sense in the context of new practices yet to be taught. Therefore, the mid-term data (Table 36) is compared with the FE data (Table 35). The findings on first used in the mid-term survey were relatively high with the uptake of practices such as ‘timely harvesting’, ‘grading’, ‘sorting’, ‘cool storage’, and ‘careful handling’ first used by over 60% of respondents at that point. Even for practices with relatively low scores, the uptake was often in the 30-60% range. This means there was limited room for further improving. MSIKA Final Evaluation Page 47 kadale@africa-online.net ‘Storing in crates and appropriate containers’, ‘lifting by fork’ and ‘climbing to harvest’ were the lowest scoring. For storage, this could be for cost reasons. For lifting by fork, this might be down to the long-standing use of hoes and/or unavailability/cost of forks. Prior to 2019, MSIKA team said they had not put as much emphasis on PHH training as on growing practices and this likely influenced the scores on mango and the generally lower scores on PHH than on improved growing practices. However, for the final year of the program, there was more PHH training undertaken. Across the practices used for the first time in the last 12 months in the FE HH survey, the pattern was similar to that of the mid-term. For instance, practices such as ‘timely harvesting’, ‘grading’, ‘sorting’, ‘cool storage’, and ‘careful handling’ were first used by over 50% of respondents. Even for practices with relatively low scores, the uptake was often in the 30-50% range. It should be noted for interpretation that if there was already a high first use in earlier years, the scores in later years, as a proportion of the whole sample, will necessarily be reduced. Table 35: First Used New PHH Practice, at FE, 2020 PHH practices used first time Tomato % Onion % Potato % Mango % Timely harvesting not too early so immature and not too late so over-ripe 60.4 n/a n/a n/a Lifting when tops have bent over & dried n/a 56.6 n/a n/a Lifting potatoes with a fork/prong, not a hoe n/a n/a 12.0 n/a Harvesting & selling green/fresh onions with stalks on n/a 53.8 n/a n/a Drying (curing) onions in the sun or under a shade before storing to improve in keeping quality n/a 58.2 n/a n/a Rubbing off surface dirt after harvest; or after drying & curing n/a n/a 48.7 n/a Trimming dried stems and roots before packing n/a 53.3 n/a n/a Handling carefully to avoid damage 57.3 n/a 58.2 42.4 Storing in cool places out of the sun prior to sale 62.2 n/a n/a 54.0 Grading by color, size and shape 60.4 64.3 58.9 52.5 Sorting by variety, size/maturity, damaged, ripeness 58.5 63.7 62.0 52.5 Packing - Loading into appropriate containers for storage or transportation 57.9 53.8 53.2 39.6 Not over-filling baskets & bags or using very big bags, as this damages product at the bottom 57.9 63.7 n/a 51.8 Not stacking more than three containers high to avoid crushing tomatoes in lower containers. 54.3 n/a n/a n/a Adding soft dry grass as soft litter 50.0 n/a n/a n/a Using night vented sheds for short term storage for selling outside main season n/a n/a 41.8 n/a Keeping clean potatoes in a cool dry place to cure or heal surface damage, before packing, storage & selling n/a n/a 58.9 n/a Harvesting when plant tops wilt and start to wither; slashing tops close to soil to cure in soil a day before harvesting n/a n/a 53.2 n/a Washing to improve appearance n/a n/a n/a 37.4 Storing in crates to prevent damage in transporting n/a n/a n/a 36.0 Harvesting when ripening just started and color begins to change n/a n/a n/a 49.6 Harvesting by climbing the tree n/a n/a n/a 29.5 Harvesting by using a special harvesting pole n/a n/a n/a 33.8 Catching fruit before it falls using bags or spreading sheets n/a n/a n/a 42.4 Total respondents 164 182 158 139 MSIKA Final Evaluation Page 48 kadale@africa-online.net Table 36: First Used New PHH Practice, Across FE Crops at Mid-term, 2019 Use of PHH practices for the first time Tomato % Onion % Potato % Mango % Timely harvesting not too early so immature and not too late so over-ripe 60.8 n/a n/a n/a Lifting when tops have bent over & dried; n/a 61.8 59.3 n/a Lifting potatoes with a fork/prong, not a hoe n/a n/a 8.7 n/a Harvesting & selling green/fresh onions with stalks on n/a 18.4 n/a n/a Drying (curing) onions in the sun or under a shade before storing to improve in keeping quality n/a 61.8 n/a n/a Rubbing off surface dirt after harvest; or after drying & curing n/a n/a 42.7 n/a Trimming dried stems and roots before packing n/a 21.1 n/a n/a Handling carefully to avoid damage 54.9 n/a 52.0 43.5 Storing in cool places out of the sun prior to sale 64.1 n/a 50.0 47.8 Grading by color, size and shape 66.3 21.1 69.3 54.3 Sorting by variety, size/maturity, damaged, ripeness 63.7 30.3 63.3 50.0 Packing - Loading into appropriate containers for storage or transportation 47.6 34.2 48.7 47.8 Not over-filling baskets & bags or using very big bags, as this damages product at the bottom 60.8 25.0 n/a 47.8 Not stacking more than three containers high to avoid crushing tomatoes in lower containers. 57.5 n/a n/a n/a Adding soft dry grass as soft litter 55.7 n/a n/a n/a Using night vented sheds for short term storage for selling outside main season n/a n/a 35.3 n/a Keeping clean potatoes in a cool dry place to cure or heal surface damage, before packing, storage & selling n/a n/a 50.0 n/a Harvesting when plant tops wilt and start to wither; slashing tops close to soil to cure in soil a day before harvesting n/a n/a 59.3 n/a Washing to improve appearance n/a n/a n/a 32.6 Storing in crates to prevent damage in transporting n/a n/a n/a 13.0 Harvesting when ripening just started and color begins to change n/a n/a n/a 28.3 Harvesting by climbing the tree n/a n/a n/a 10.9 Harvesting by using a special harvesting pole n/a n/a n/a 37.0 Catching fruit before it falls using bags or spreading sheets n/a n/a n/a 32.6 Total respondents 273 76 150 46 The qualitative interviews conducted at FE gathered feedback on the knowledge and application of PHHs by crop, as follows: Of the 18 tomato farmers interviewed, two said they did not receive any PHH training for tomatoes. One farmer who was not growing tomatoes before receiving the training from MSIKA has not yet started applying the practices as the tomatoes were in the field at the time of the interview. For those that were trained, 10 practices were specifically mentioned that have been applied. These include: timely harvesting; storing in cool places out of the sun prior to sale; grading by color, size and shape; adding soft dry grass as soft litter; not over￾filling baskets and bags or using very big bags as this damages product at the bottom; handling carefully to avoid damage; sorting by damaged and ripeness; packing - MSIKA Final Evaluation Page 49 kadale@africa-online.net loading into appropriate containers for storage or transportation; and rubbing off surface dirt after harvest. "One thing I liked about the training is that we were taught to grade, which makes selling easier.” FBO member, Dedza. The most applied practice is the timely harvesting not too early so immature and not too over ripe, which was applied by 13 farmers. The least applied practice was rubbing off surface dirt after harvest by one farmer. On average each farmer applied four practices with the lowest number applied being one and the highest being seven. Out of the 15 onion respondents, 12 received training on PHH and three said they did not. Seven practices were stated as having been adopted: harvesting and selling green or fresh onions with stalks on; making a shed to hang onions to prevent them from getting direct sunshine; packing in appropriate bags or baskets; grading by variety, size and appearance; sorting according to damaged; trimming dried stems and roots before packing; and harvesting/lifting when the tops have bent over & dried. "We realized that marketing of onions was not good because we did not know how to grade or sort, but now that we learnt how to grade and sort, marketing and pricing is not a problem."… FBO member, Lilongwe. The most applied practice by 10 farmers is making a shed to hang onions to prevent them from getting direct sunshine. The least applied practice is trimming dried stems and roots before packing that was applied by one farmer. On average, each farmer applied three practices with the lowest applying one practice and the highest applying five practices. Some practices were not applied as the farmers said that they sold the onions while they were still fresh: "I sold out my onions fresh, because I needed quick money for other projects." FBO member, Mangochi. Out of the 15 Irish potato respondents, 12 received training on PHH and three did not. For those that were trained, ten practices were adopted: harvesting when plant tops wilt and start to wither; rubbing off surface dirt after harvest or after drying and curing; keeping clean potatoes in a cool dry place to cure or heal surface damage before packing, storage & selling; handling carefully to avoid damage – remove all damaged potatoes to avoid rotting; packing - loading commodities into appropriate bags/containers for storage or transportation; grading by color, size and shape; sorting by variety; removing damaged potatoes; making a shed; adding grass as soft litter; and lifting potatoes using forks. The most adopted practice was harvesting when plant tops wilt and start to wither that has been applied by nine farmers and the least adopted practices were rubbing off surface dirt after harvest or after drying and curing and adding dry grass as soft litter which were both applied by one farmer each. On average, one farmer applied four practices with the lowest applying one and the highest applying six. Out of 15 mango respondents, two did not receive any of the PHH training for mangoes. One respondent did not apply the practices because he was trained after he had already harvested the mangoes. Eight practices have been applied: harvesting when ripening just started and the color begins to change; catching fruit before it falls; harvesting by climbing with bags; grading; storing in a cool place; harvesting by using a special harvesting pole; and cleaning before taking them for sale. The most applied practice is storing in a cool place applied by ten Farmers. The least adopted practices are catching of fruit before it falls and harvesting using a special MSIKA Final Evaluation Page 50 kadale@africa-online.net harvesting pole which have been applied by one farmer each. On average a farmer applied three practices with the lowest applying one and the highest applying five. "I liked the training because we are having few rotten fruits now." FBO member, Mchinji. Overall, these suggest high up take of PHH practices. 3.2.9 Post-Harvest Losses Indicator 26 measures change in post-harvest losses. The LoP target for post-harvest losses is 14.0%. This is calculated as the total weight of the losses across all growing/ harvesting cycles divided by the total production for those cycles for each crop and then aggregated to an overall loss. Table 38 sets out the revised weighted baseline for the four crops and the same for the FE results are set out in Table 37 below. The consultants removed the results of producers who gave a production value above the FAO maximum yield, as the losses are a proportion of the yield and so problematic to include when working out the losses. The removal of the three crops and introduction of weights has resulted in a revised aggregate loss at baseline of 14.8%, with 11.8% for men and 15.4% for women. The FE survey found that across the four crops, the losses were 13.1% of production, which is 107% of the LoP target set at 14.0%. In aggregate, MSIKA has met and surpassed its LoP target, however it should be noted that there are considerable challenges in accurately measuring crop losses. Measuring crop losses is a very difficult task, as losses occur at several points from field/harvest through to market. Measuring production is also difficult, but at least the producer can record whole units harvested, such as bags or pails. When it comes to losses, these can be part of a whole unit (a bag of produce all going rotten), but more likely losses are part units where the producer sees that some of the vegetables or fruits in a bag have rotted, so removes only those affected. This might be repeated several times. A further difficulty in measuring losses is defining the loss at harvest time. It is difficult to assess the loss for a potato plant that is dug up at harvest with a mix of usable and not usable potatoes. Rotten or unsellable fruit will be left in/on the ground. Ditto with fruits that have fallen before collection, as to how they/the weight can be count. Protocols for enumerators can help, but there are genuine difficulties for producers to estimate losses. Therefore, data for losses should be treated with caution. For the different crops at FE, tomato had losses of 12.7%, onion at 9.9%, potato at 8.9% and mango at 17.5%. The losses for men were 11.8% and for women were 15.4%. The differences vary by crop, notably in tomato (10.7% for men vs 17.0% for women), though the losses are similar for onion, mango and potato. Overall, the losses for men were lower than for women at 11.8% and 15.4% respectively. Table 37: Post-harvest Losses by Crop and by Sex, FE, 2020 FE mean yield and loss (kgs) Crop Males Females Total Yield Loss % Weighted calc Yield Loss % Weighted calc Yield Loss % Weighted calc Tomato 127,020 13,564 10.7 2.7 60,784 10,308 17.0 4.4 187,804 23,872 12.7 3.25 Onion 115,796 10,815 9.3 1.9 44,103 5,008 11.4 1.4 159,899 15,823 9.9 1.68 Potato 147,389 12,764 8.7 2.1 85,414 8,046 9.4 1.9 232,803 20,810 8.9 1.99 Mango 100,786 16,743 16.6 5.2 86,185 16,010 18.6 7.6 186,971 32,753 17.5 6.15 Total 490,991 53,886 11.8 276,486 39,372 15.4 767,477 93,257 13.1 MSIKA Final Evaluation Page 51 kadale@africa-online.net Table 38: Post-harvest Losses by Crop and by Sex, Baseline, 2017 Baseline Mean Yield and Loss (kg) Crop Males Females Total Yield Loss % Weighted Calc Yield Loss % Weighted calc Yield Loss % Weighted Calc Tomato 94,495 12,734 13.5 3.5 58,762 7,961 13.5 4.1 153,222 20,695 13.5 3.70 Onion 48,227 3,707 7.7 0.6 7,373 1,031 14.0 0.5 55,601 4,738 8.5 0.53 Potato 196,234 14,827 7.6 2.3 81,387 7,202 8.8 2.4 277,620 22.029 7.9 2.31 Mango 72,995 17,723 24.3 8.8 54,055 10,380 19.2 7.5 127,050 28,103 22.1 8.23 Total 411,952 48,991 15.2 201,577 26,573 14.5 613,493 75,564 14.8 Respondents were asked the point at which the crop was spoiled covering pre￾harvest for mid-term, and harvest and post-harvest periods for both evaluations. The findings are set out in Table 39 for FE and Table 40 for the baseline. The most common points at which losses occur were ‘damaged when harvesting’ and ‘on-farm storage’17, with at least 30% - 60% reporting them both during the baseline and FE. Damaged when harvesting includes produce which may get spoiled where it is physically difficult to harvest, notably for mango, or where the produce is delicate (tomato) or easily damaged at harvesting, such as digging up potatoes. As expected, ‘could not be sold prior to spoil’ was higher at FE with at least 30-40% stating it across the crops, most likely due to COVID which resulted in restrictions on markets. Lower scores were reported for damage ‘during transportation’ and ‘damaged at the market’ both during the mid-term and FE. Damaged during transportation includes losses in the field while transporting back to the farm/storage facility or while transporting to the market. Table 39: Point at which Harvest was Spoiled, FE, 2020 Point at which crop was lost Tomato Onion Potato Mango % % % % Damaged when harvesting 48.2 39.9 62.6 67.0 Damaged at the farm in storage 31.2 34.0 28.7 23.5 Damaged during transport to market 17.0 5.4 2.6 8.7 Damaged at the market 7.5 5.4 1.7 10.4 Could not be sold prior to spoiling 41.9 45.8 37.8 47.8 Total 253 203 230 230 Multiple response possible Table 40: Point at which Harvest was Spoiled, Baseline, 2017 Point at which crop was lost Tomato Onion Potato Mango % % % % Damage prior to harvest (pest/birds) 51.2 26.1 43.5 33.8 Damage at harvest 31.6 17.4 45.9 39.8 Damaged at farm (storage) 31.0 52.2 45.3 24.4 Damaged during transport 14.1 2.9 3.5 4.0 Damaged at market 7.4 1.4 1.8 4.0 Could not be sold prior to spoil 8.0 4.3 2.4 7.5 Total 326 69 170 97 Multiple response possible 17 On farm storage is the point after harvest where the crop is held on the farm prior to selling. MSIKA Final Evaluation Page 52 kadale@africa-online.net Respondents were asked how they store their crops (non-crop specific). The findings at FE are set out in Table 41. Unsurprisingly, the most common place is inside the house practiced by 49.9% of respondents, followed by a separate building/store at 45.8% compared to 26.8% and 20.8% at mid-term and baseline respectively. This could explain why the volume of storage built within the last 12 months has exceeded the revised LoP target. Only 1.2% stored at their FBO, which matches the KII responses for FBOs that they do not have any storage facilities. MSIKA reports that it added 450 cu/m of storage through construction and distribution of farm storage bins to FBOs. Similar to the mid-term, women were more likely than men to store in the house, while men were more likely to have a separate store. Compared to the baseline, more FE respondents were storing in a separate building/ store/shed (45.8% vs 20.8%). There is a clear shift from inside to outside storage. Table 41: Place of Storing Crops, FE and Baseline Storage location Baseline Final Evaluation All % Male % Female % Total % Inside my house 55.1 43.7 57.4 49.9 In a separate building/store/shed 20.8 53.5 36.5 45.8 In a Nkokwe/outside store 10.2 1.5 1.4 1.4 In baskets/bags outside or in a pile 8.0 4.3 4.1 4.2 FBO Warehouse 0.2 0.9 1.6 1.2 Do not store (direct to market) 8.0 8.4 10.8 9.5 Total 590 535 444 979 Multiple response possible Respondents were asked if they had built or restored their place of storage. A total of 23.3% said they had built a new storage building, room or shed while 13.8% said they had restored a storage building, room or shed. As at least 40% mentioned to have been storing in the house, it is possible that refurbishment or building relates to the house, as a multi-purpose building rather than specifically for storage, but it is not possible to say from the data. The building and/or restoring of storage contributes to Indicator 27, total increase in installed capacity (dry or cold storage). Table 42 below sets out the calculation for this indicator. Respondents gave estimates of the floor area in square metres and height in metres. This approach is prone to error, as measurements were estimated by respondents and given over the phone such that it was not possible to pace out the estimated storage physically. Based on the initial raw data, the consultant’s conclusion was that an upper value needed to be set to address unrealistically high values. After consideration, and consultation with MSIKA, upper values for the floor area were revised to a maximum of 24 square metres, while upper values for height were set at 2.5 metres. This was to ensure that the final calculation was closer to realistic measurements. The estimated areas for both categories (new and restored/refurbished) were totalled and divided by the total number of respondents in the sample, giving a mean average volume per sampled participant (979) of 8.7 and 5.5 cubic meters (cu/m) for new and restored respectively. This equates to total new storage of 269,680 cubic metres(cu/m) and total refurbished storage of 168,950 cubic metres (cu/m) when extrapolated to the FE population of 30,922 trained farmers. As noted in the MSIKA performance indicator table, while the data is probably overstating the extent of the change, the consultant's MSIKA Final Evaluation Page 53 kadale@africa-online.net finding is that the target is very likely to have been achieved even if there had been verification of the farmers’ estimated measurements of the storage. Table 42: Building or refurbishing storage calculation for FE population Storage built or refurbished (cu/m) Sample size 979 Population 30,922 Volume new storage/farmer 8.72 Volume refurbished storage/farmer 5.46 Total new storage 269,680 Total refurbished storage 168,950 Total storage 438,629 The revised LoP target was a total (new and restored) increase in storage of 45,770 cubic metres, so with this addition in 2019-20, MSIKA has met and exceeded the revised LoP target with an overall achievement of up to 1,059%. 3.2.10 Gender Questions on gender were not included in the FE HH survey to reduce its length. This section briefly summarises the main points from the mid-term for ease of reference. At mid-term, respondents were asked about who (men/women) does the work on the crop(s) that were grown by their HH. The most common response was that men and women do the work equally, ranging across tasks from 75-81% of respondents. This was followed by similar scores for ‘only by men’ ‘only by women’ and ‘more by men than by women’. The least common was ‘more by women than by men’. There was no discernible pattern between crops. Respondents were asked to provide the split of key growing tasks by sex, including selling and managing the money. The findings have been updated to cover the four crops at FE and are set out in Table 43 below. There were relatively even balances, but with men more involved in land preparation and managing the proceeds, and marginally more involved in planting and managing the crop. Table 43: Work on the Tasks by Sex at MTE Across for the Four FE Crops Work done by: Land preparing Planting Managing growing Harvesting PHH Selling Managing proceeds % % % % % % % Only by Men 9.1 4.9 5.4 2.7 4.9 9.9 13.7 More by men than by women 14.8 13.2 13.6 11.5 10.1 10.9 9.8 By men and women equally 59.9 65.0 65.8 69.2 65.8 58.5 56.8 More by women than by men 5.4 5.5 5.4 5.7 7.1 5.7 3.2 Only by women 13.6 13.7 13.4 14.0 17.5 20.3 19.1 Do not know 1.3 1.6 0.6 0.5 0.3 1.1 1.1 Total 634 634 634 634 634 634 634 Multiple response possible MSIKA Final Evaluation Page 54 kadale@africa-online.net 3.2.11 Farm Management Knowledge and Application Indicator 15 is the percent of agricultural producers in target region who can identify key characteristics of a well-managed farm. The questions related to the indicator were taken out to shorten the interviewing time. A summary of the MTE HH findings are provided for ease of reference. MSIKA trained producers in farm management. For unprompted knowledge of improved farm management practices, the baseline found that 65.0% of respondents could identify at least one characteristic and the mid-term survey found that all respondents (100.0%) could identify at least one characteristic. Respondents were also asked which management practices they used in the last year in the FE HH survey. There was much higher application (prompted), of farm management practices than at baseline, which suggests the training was effective in stimulating action. Similar to the mid-term, the two lagging practices are ‘sell together with other farmers’ and ‘keep good farm records’. From the consultant’s experience in Malawi, these are practices that producers have difficulty adopting. For example, keeping good farm records requires literacy, which is often very low. At baseline, 55.9% of respondents were applying at least one farm management practice which had increased substantially to 97.7% of participants at FE. Table 44: Use of Improved Farm Management Practices, Baseline vs FE Have you used these management practices in the last 12 months? Baseline Final Evaluation Management Practices Male % Female % Total % Male % Female % Total % Keep good farm records 31.7 14.3 23.9 64.6 53 59.4 Sell together with other farmers 10.9 15 12.7 30.2 28.8 29.6 Do costings for growing crops 41 40.8 40.9 75.7 66.5 71.5 Calculate profits after selling 68.3 57.8 3.6 88.6 82.3 85.8 Plan production for upcoming season n/a n/a n/a 90.9 87.2 89.2 Plan production for a specific market 21.9 27.9 24.5 77.2 73.7 75.6 Understand the specifications for specific buyers n/a n/a n/a 88.2 83.3 86 Separating commodities into the different grades n/a n/a n/a 78.5 81.2 79.7 Timely buying of inputs n/a n/a n/a 90.5 85.1 88.1 Run your farm as business 60.7 44.2 53.3 n/a n/a n/a Plan your farm and how to rotate crops 51.9 55.1 53.3 n/a n/a n/a Respondent 183 147 330 526 430 956 Multiple response possible. The effect of MSIKA’s training of producers in farm management practices is measured through Indicator 8: is the number of individuals who have applied improved farm management practices (i.e. governance, administration, or financial management). The calculations for this indicator are set out in Table 45 below. Based on the FE survey finding above that 97.7% of producers applied one or more improved farm management practices, this equates to 30,196 out of 30,922 producers. In terms of the disaggregates, the LoP actual for males is 93.5% (14,213 vs 13,286) while the LoP actual for females is 139.7% (12,107 vs 16,910). MSIKA Final Evaluation Page 55 kadale@africa-online.net Table 45: Indicator 8: Application of improved farm management practices, FE Indicator 8 and disaggregates # who applied % of sub￾sample Total # of producers applying # who have applied improved farm management practices 956 97.7 30,196 # who have applied improved farm management practices - Male 526 98.3 13,286 # who have applied improved farm management practices- Female 430 96.8 16,910 # who have applied improved farm management practices - Producers 956 97.7 30,196 3.2.12 Sales Volume According to the revised Performance Management Plan (PMP), surveys have not given accurate figures for sales when triangulated with other data. Therefore, the MSIKA team, with USDA agreement, determined that the most accurate way to measure sales is through FBO records. Nevertheless, this section will report on the findings from the FE HH survey for triangulation, though the information should be treated with caution. The outcome of all the training, organising of farmers, making market linkages and providing access to market information (see FBO section 3.1) is that sales are expected to increase in both volume (Indicator 21) and value (Indicator 23). Indicator 21 is the volume of commodities sold by producers in metric Tonnes (mT). Standardising sales data per ha enables an appropriate comparison between baseline and FE rather than sales per producer, as land area varies. 18 The original impact baseline value was 7.3 mT/ha, however, as noted in the MTE and in the yield section, there were outliers in the baseline sample that had not been consistently removed. As a result, the unadjusted baseline values considerably overstating the true value. The consultant also identified sales outliers in the baseline and FE where the volume of sales exceeded the volume of production minus estimated losses. The consultant allowed a margin of error, and removed those that still had excess sales. In the MTE, the consultant noted that the baseline was based on the mean sales per ha for vegetables (and chili) added to the mean sales per ha for fruits. Adding the sales of all crops together ignored that some crops produce a higher volume per ha than others. Therefore, there was need for weighting of the respective samples. For the FE, the consultant removed the yield and sales outliers and re-weighted the samples for comparison. The baseline and FE findings are set out in Table 46. The data for sales volume is reported by crop in kgs19/ha to enable comparisons between the crops and with the baseline. The highest mean volume of sales in the FE is for mango (5,270 kg/ha), followed by onion (4,267 kg/ha), potato (3,780 kg/ha) and tomato (3,616 kg/ha). Compared to the adjusted baseline, the highest increase in mean sales volume is for tomato at +70.0%, followed by potato (+28.8%), onion (+21.1%) and mango (+6.1%). The increased sales volume for tomato is important for producers, as it is the most commonly grown crop, so increases for tomato suggest widespread benefits. Potato is the second most widely grown and onion the third, so the highest increases were for the most commonly grow crops and the lowest for the least commonly grown (mango). 18 The baseline producers had relatively large land areas compared to mid-term and FE surveys. 19 Metric tonnes are split into 1,000 kgs so that the differences are clearer. MSIKA Final Evaluation Page 56 kadale@africa-online.net Table 46: Volume sold in kg/ha, Revised Baseline vs FE Crop Kg/Ha Impact Baseline Finals Evaluation-all Mean Median Mean Difference Median Difference Tomato 2,126 886 3,616 70.1% 2,213 149.7% Onion 3,524 2,198 4,267 21.1% 2,717 23.6% Potato 2,934 1,989 3,779 28.8% 2,766 39.1% Mango 4,966 3,568 5,270 6.1% 3,532 -1.0% N=1,074 As noted in earlier sections, the producers have had to cope with COVID, which is likely to have been a factor in the lower than target performance for Year 4 and the LoP sales volumes. Nevertheless, there have been positive changes from baseline to FE across the four crops. Producers were asked in the FE HH survey if sales for their crop were affected by COVID. Responses by participants on 607 out of 1,074 (56.5%) crop said sales of the crop were affected by COVID. It was therefore surprising to find in the analysis that sales of those who say their sales were impacted by COVID on average had higher sales than those who said they were not affected. There are several possible explanations. One is that they may be overstating that COVID impacted sales, perhaps thinking that there might be some benefit/compensation to them. Another is that they produced more, since increased yields would mean they have more to sell and they still managed to sell in local markets. A third is that COVID only affected one of the three sales seasons, so overall the effect was lessened. In the qualitative interviews, farmers were asked about their sales experiences on only one of their crops. Based on these, the following information was obtained: Out of 18 tomato farmers, 13 reported that they had sold alone/as individuals to vendors. Six had sold vendors, with two selling in groups. "Since our produces ripen differently with other farmers, we sell individually by calling or taking it to the vendors." Lilongwe, male farmer. One out of the 18 farmers was linked to a market by MSIKA, with the other 17 saying they were not linked or assisted in finding markets. "We were promised to be linked with markets, but it seems it was a challenge with LOL, so we sold our tomatoes to vendors." Dedza male farmer. Out of 15 onion farmers that were trained in onion production, three farmers reported selling through their FBOs and 12 farmers reported selling alone to vendors or selling themselves in their local area. "I sell alone at home to traders that come to me." Dedza, male grower Out of the 15, four say they were helped or linked with markets, while the other 11 sold individually to vendors and through their own selling. "LOL linked us to markets and all is working well. We just want to increase production so that we fully meet the needs of our customers." Mchinji, female. Out of 15 Irish potato farmers trained in Irish potato production, 12 said they were selling alone and three said they sold through their FBOs. "I sold alone, because we usually sell to vendors, so groupings would not work well since these people cannot be able to purchase a large stock at once. Only when a bigger market is found are we able to sell in groups." Lilongwe, male farmer. MSIKA Final Evaluation Page 57 kadale@africa-online.net Five out of the 15 farmers were assisted or linked to markets to sell their produce and eight said they were not assisted to access a market. "LOL should continue to link us with more markets because their prices vary and we might be able to find a market that will benefit (us).” Lilongwe, male farmer. Out of 15 mango farmers, 13 sold their produce individually to vendors or by their own retail in their local areas. Two reported that they were able to sell via the FBO. ''We were advised by LOL to sell as a group because you make more money than selling alone'' Lilongwe, male farmer. None of the farmers reported being linked to a market. "We were expecting to be linked to markets, but unfortunately we weren't and resulted in us selling our produce to vendors." Mangochi, female farmer. 3.2.13 Sales Value The method for determining Indicator 23, sales value in US $ was also revised to be captured through FBO reported data for the same reasons as for volume (section 3.2.12). Data on sales value was captured in the FE HH survey to support triangulation. To allow comparisons, sales are standardised into US $/ha. As noted with sales volume, the baseline had yield, price and sales outliers which were removed to enable comparison between the baseline and FE. For the overall FE sample, sales of the four crops fall within a similar range with tomato at $1,229/ha, onion at $1,226/ha potato at $1,195/ha and mango at $966/ha. Comparing these to the baseline there were big increases for mango (+99.7%), followed by potato (+92.7%), onion (+63.3%) and tomato (+44.1%). See table below. Table 47: Value of Crop Sold in $/ha, Baseline vs FE Crop Baseline - yield and price outliers removed Final evaluation - yield & price outliers removed Difference FE over baseline Mean $/ha Median $/ha N Mean $/ha Median $/ha N % Tomato 853 408 268 1,229 652 275 44.1% Onion 751 335 68 1,226 680 259 63.3% Potato 620 367 200 1,195 816 251 92.7% Mango 483 208 100 966 550 237 99.7% Kadale noted that there were different prices in MK in the baseline and the FE. To get to the US $ value, the exchange rate used is MK 726.82:$1 as used in the baseline. The MK and US $ equivalent prices are compared in Table 48 below. While the mean tomato price in US $ was 8.0% lower in the FE, the mean mango price was 76.1% higher, onion 24.8% higher and potato 15.6% higher. Table 48: Prices in Malawi Kwacha and USD $, Baseline vs FE Crop Impact Baseline Finals Evaluation-all Difference Price (MK) Price (USD) Price (MK) Price (USD) Tomato 268 0.37 247 0.34 -0.80% Onion 175 0.24 218 0.3 24.80% Potato 157 0.22 182 0.25 15.60% Mango 76 0.1 102 0.18 76.10% MSIKA Final Evaluation Page 58 kadale@africa-online.net In the FE survey, a very small number of growers20 (20 out of 1,074 – 1.9%) did not sell any of their crop. Details are set out in Table 49 below. The FE found that 3.4% of mango producers, 2.6% of potato producers, 1.1% of tomato producers and 0.4% of onion producers did not sell anything. The most common reason, especially by tomato, onion and potato producers, was that they ‘did not have a surplus to sell’, which likely means they lost most or all of their crop. The second most common reason was that they ‘could not get a buyer’, given most commonly by mango producers. ‘Poor quality’ was the third most common reason and was given mostly by potato producers. This may be related to pest and disease damage, and damage at harvest, on the farm and at the market. It would be surprising if no growers had faced problems at all, such as a crop failure in any season. Table 49: Growers that Sold Nothing, FE, 2020 Crop Producers # that sold nothing % of that crop Tomato 276 3 1.1% Onion 264 1 0.4% Potato 266 7 2.6% Mango 268 9 3.4% Total growers 1,074 20 1.9% Respondents were asked for the primary way they sold, including selling individually, selling with other producers in their club/group or selling with producers outside their club/group. The most common method was selling as an individual, ranging from 84- 90% of respondents depending on the crop. Selling with other members of a club/group ranged from 9-13%, with the highest responses for onion producers. Table 50: Primary Means to Sell, FE, 2020 Primary Sell Response Tomato Onion Potato Mango % % % % With farmers in my farmer club/group 9.1 12.9 8.8 9.3 With farmers not in my farmer club/group 5.1 3 3.8 0.8 On my own as an individual 85.5 84.1 87.4 89.6 Other 0.4 - - 0.4 Total 275 264 262 259 Respondents were asked about the locations where they sold their crop, with multiple responses possible (Table 51). The most common response for all crops was ‘traders who came to me’, ranging from 75-79% across the four crops. ‘Local market’ was the second most common response, ranging from 30-44% of respondents. The only other substantial response was for ‘neighbours’ at 22-34%. 20 Note that some were interviewed for two crops. MSIKA Final Evaluation Page 59 kadale@africa-online.net Table 51: Sales location, FE HH Survey, 2020 Where Did You Sell your produce? Response Tomato Onion Potato Mango % % % % Traders who came to me 79.3 78.0 79.4 74.5 Local market 44.0 30.7 29.8 38.2 Neighbors 34.9 22.0 25.6 23.6 Traders that I delivered to 20.4 14.4 14.9 16.2 Distant market 8.0 7.6 7.3 7.7 My Farmer group 3.6 1.9 2.3 2.7 Direct to Processor/large buyer 0.7 - 1.1 0.8 NGO 0.4 2.7 1.1 0.8 Total 275 264 262 259 Multiple response possible 3.2.14 Access to Finance This section covers access to finance (A2F) issues relating to producers. A2F information is drawn from KIIs with MSIKA staff and the MFI. It was not covered in the FE HH survey. As at June 2020, MSIKA reported that it had trained 744 VSL groups, each with 15-20 members. This equated to 10,000-13,000 members.21 In addition, MSIKA reports that 5,444 individuals had received at least one loan through a MSIKA related VSL group. It was encouraging that two thirds of MSIKA producers were in a MSIKA VSL or another VSL group, as these have proven to be effective at enabling rural households to save, and as a source of small, short term loans. The advantage for members is that although interest is charged on the loans, the accumulated interest earned by the group is returned to members, often for buying inputs in October and November for use in the main farming season. In addition to improved access, producers also needed to understand financial issues. As reported earlier (section 3.1.3.4), financial training reached 62-64% of FBOs. This topic is also covered in the sections on agricultural lending through MSIKA’s relationship with the MFI in section 3.3.3 below and in section 3.1.3 on FBOs. 3.2.15 Investment This section covers investment made by producers. Section 3.2.9 sets out the volume of building and/or refurbishing storage producers invested in. These respondents were asked how much they spent on refurbishment/building from August 2019 to July 2020. The amount spent on building new storage across the FE sample (979) was the calculated at $7.02 per farmer, equating to US $217,275 across the 30,922 FE population. For refurbishing storage, the amount spent was $3.90/farmer, equating to US $120,632 across the FE population. The total reported spent on equipment and land from August 2019 to July 2020 was a mean US $23.3 and $25.2 respectively per farmer, equating to US $ 668,563 and $ 778,377 across the 30,922 FE population. 21 MSIKA says there may be duplicates in this total, so could not give a precise figure. MSIKA Final Evaluation Page 60 kadale@africa-online.net 3.2.16 Employment This section covers employment at farm level based on the FE HH survey. Indicator 22 is the “number of jobs attributable to USDA”. Producers contribute to the total, along with processors, FBOs and other MSIKA partners. Respondents were asked if they hired anyone for four weeks or more (converted to full time equivalent) in the period August 2019 – July 2020, with 15% saying they did hire. The baseline reported 0.09 full time jobs equivalent (FTE) jobs created per participant household in the previous 12 months, against 0.07 FTE jobs calculated at the FE. Comparisons are difficult to make for two reasons. Firstly, the HHs in the baseline had larger land areas than the FE HHs, and demand for labour increases with land size as the land area and production exceeds what the HH can manage from its own resources. Higher land areas in the baseline would tend to require more labour. Secondly, the FTE figure is difficult to measure with certainty, as respondents are providing estimates of labour. Only those that were hired for more than four weeks count to the calculation, but the use of short term labour termed ‘ganyu’ is common for tasks like land preparation and weeding, which typically are for less than four weeks and may involve different workers over the three growing seasons.22 The number of people hired by the 15% of participants that hired someone for four weeks or more, was split by male and female, and converted to jobs per respondent across the sample (979). This was multiplied by the FE population (30,922) to calculate the total number of people hired for four weeks or more at 2,067 FTE workers between August 2019 – July 2020 3.2.17 COVID effects The KIIs with producers sought information on the effects of COVID on producers. The prevalence of COVID in rural areas at the time of the research was not clear. The main evidence of COVID was in the cities, but that could have been a function of the availability of testing and better appreciation of it. For the tomato respondents, of the 18 participants, 11 reported having not been affected by COVID-19 pandemic and that things were as before: "Everything is being done normally, just like the time before the Corona Virus." Tomato male farmer, Chileka EPA, Lilongwe "We continued with all our work, but followed the precautionary measures." Tomato male farmer, Chowe EPA, Mangochi For those that reported having been affected by the COVID, things changed especially in relation to preventing further spread. The changes included: a. Group gatherings were not possible for information sharing within the FBO or for marketing b. They were not able to travel to the city/town markets due to restrictions c. There were lower farmgate prices due to limited market opportunities as good buyers were rare d. There was a loss of production due to lower demand and higher supply e. Some FBOs were considering only producing one crop per member to reduce flooding the thin market with same crops 22 Note that these jobs may not necessarily be created, as the demand is seasonal and so people are hired each year. MSIKA Final Evaluation Page 61 kadale@africa-online.net "Vendors (in small numbers) coming to buy from us are buying at cheaper prices and as a result our tomatoes are getting spoiled due to shortage of buyers.” Tomato female farmer, Mkwinda EPA Lilongwe "(We are) failing to go to markets to sell tomatoes and as a result, vendors coming to us are buying at lower prices." Tomato male farmer, Ndemera EPA, Lilongwe. 3.3 Banks, MFIs and VSLs MSIKA aims to stimulate agricultural and small and medium enterprise (SME) lending. This section reviews MSIKA’s work with a Malawi Bank SME finance through a Special Purpose Vehicle (SPV) and with a Microfinance institution (MFI) for agricultural loans. The information for the Bank is drawn from the MTE, as this was not a focal area for Year 4. Information for the microfinance institution draws on the MTE, the KIIs with the MSIKA team and a KII with the the MFI staff. There is also information on VSL from MSIKA KIIs and reports. 3.3.1 Overall Performance The relevant indicators are 11 (# of individuals receiving financial services), 12 (# of loans disbursed) and 13 (value of loans disbursed). For indicator 11, MSIKA reports an additional 5,612 individuals receiving financial services in year 4, bringing the LoP total to 13,922 against a target of 9,984 individuals resulting in an achievement of 139%. Achievement was higher for female beneficiaries at 169% than for male at 115%. For indicator 12, MSIKA reports an additional 5,074 loans in Year 4 resulting in a LoP total of 10,377 against the LoP target of 9,561, an achievement of 109%. These were broken down as MFI loans - 4,909, VSL loans - 5,444 and those through the SPV facility/ Bank - 24. For indicator 13, MSIKA reports an additional US$ 413,100 in Year 4, resulting in a cumulative LoP total of US$ 699,879, a 55% achievement. The biggest contributor in terms of value has been MFI with an LoP total of US$ 431,943 which is 61.7% of the loan value. The SPV/Bank loans contributed a LoP total of US$ 193,343 (27.6% of all loan value) and VSLs contributed a LoP total of US$ 74,592 (10.7% of all loan value). Female beneficiaries received a relatively high proportion of loan value against their LoP target (72%) than male beneficiaries (41%). The key factor in not achieving this target has been the lower than anticipated numbers of loans and value of those loans under the SPV with the bank. MSIKA had reported delays in getting the facility off the ground (see next section), and eventually settled for working with the bank. In essence the loan process was slower than expected and the size of the loans limited by the size of the SMEs that applied and the limited collateral they had available which restricted the size of loans the bank was willing to lend. This is related to the relatively modest scale of operation of many of the SMEs operating in fruit and vegetable processing, other than a few larger firms, like Suncrest and Malawi Mangoes that have alternative means to secure their finance. While the bank might have been overly cautious, it was the small size of the SMEs and their limited capital, capacity and collateral that constrained the value of loans. In contrast, there were larger numbers of loans through the MFI and through VSLs, but the average value of these was relatively low and unable to make up the short fall from the SME lending. The expectation is that the MFI and the VSLs will continue to lend as sustainable savings and loans mechanism. It is also likely that the bank will continue to work with the SMEs and FBOs that it has built good relationships with, as long as they fulfil the loan terms. The bank may also progressively expand, but this is likely to be cautious, as is the nature of banks in Malawi. MSIKA Final Evaluation Page 62 kadale@africa-online.net 3.3.2 Finance for SMEs From the MTE findings and FE KIIs, MSIKA intended to have an SPV either as a fund managed by MSIKA or through a bank. MSIKA was advised to work through a bank or a Malawian entity to manage it. MSIKA went through a lengthy process of assessment before finalising a partnership with the selected bank. MSIKA provided an initial loan of $250,00023 in March 2019 as a fund for on-lending by the bank. In return, the bank agreed to ease its collateral requirements24, and to charge a lower interest rate than normal25, on the basis that MSIKA would assist SMEs to prepare screened proposals that are then put to the bank, which is described as the ‘pipeline’. To support the process, and importantly so from the bank’s perspective, MSIKA identified agri-processors and agro-dealers with potential to buy from, or supply, MSIKA producers. MSIKA provided due diligence information on these SMEs and assisted them to prepare and improve their loan proposals through group sessions. The bank conducted its own due diligence and proposal assessment, involving visits and gathering evidence of sales and cashflow. Not all proposals were accepted. In the MTE KII, the bank’s staff reported that the main issue for the bank is the low turnover of money through any bank account, compared to the higher business revenues claimed by the SMEs in their proposals and relative to the requested loan sizes. The bank decided to offer relatively small first loans compared to what is being asked, and making this first loan primarily for working capital, not for fixed asset purchase. 26 The aim was to see how the SME manages the initial loan typically for six months. If the SME showed good cash turnover through the account and the ability to service the loan without problems, then the bank would look to disburse larger second loans that could be for fixed asset purchase. From the perspective of the SMEs, a reduced loan size was a problem for some SMEs who perceived this to be too much effort to get a small loan. For two of the more established SMEs, the owners stated that they understood this and so they focused on servicing the loan to get the bigger second loan they were originally seeking. The President of Malawi announced in April 2020 prior to the election, that because of COVID, there would be a moratorium on collection/enforcement loans to SMEs. Four of the twelve SMEs requested a moratorium. Prior to this, one SME was facing problems with repayment. The others felt they could repay to the schedule.27 For Year 4, MSIKA expanded the facility with the bank for lending to selected FBOs, which on-lend to their members against business plans. A factor behind this approach was that microfinance loans (see next section) are more expensive than the bank loans at 36% compared to 23% respectively for a six-month period. More loans to FBOs were sought by MSIKA, but the bank was more cautious. Loans need to be tied into the timing of growing seasons. At this point it is too early to say if this will expand as the first loans were given in May 2020. MSIKA reports that the agricultural lending team has been responsive and faster at progressing applications than the SME team, though the bank in general is slower than that MFI. MSIKA continued to try to interest other banks, however with the closing of the project, there was not enough time for these to be established and for loans to be repaid before project end, so they did not progress for this and other more bank specific reasons. 23 At 2% of the forex value. 24 This is based on cash collateral of 20-25%, rather than land/vehicles or other ‘hard’ collateral. 25 The bank’s rate was 21% plus $30 processing fee versus other banks at 25% plus 2% processing fee. 26 Most applications have been for fixed asset investment. 27 Interest would still accrue during a moratorium, so there is an incentive for SMEs to repay on time. MSIKA Final Evaluation Page 63 kadale@africa-online.net Overall, MSIKA reports that the bank had on-lent US $93,000 to seven processors and agro-dealers at mid-term, which increased to US $193,343 by FE. The bank had on￾lent to 12 processors, and 12 FBOs. 28 Two SMEs have had second loans, but it appears that the bank has substantially tightened loan conditions requiring an unrealistic 75% deposit of cash against a loan on which interest is charged on 100% of the loan. This appears to be designed to deter SMEs and may be linked to the government announcement of a moratorium on SME interest due to COVID thereby deterring banks from lending to SMEs for now. The bank has also told MSIKA that they will not add more loans, as those loans would not be cleared before the project end. 3.3.3 Finance for Producers MSIKA worked with an MFI to finance loans for producers, particularly those with access to irrigation. MSIKA provided technical assistance to the MFI, made links with FBOs, organised producers, and trained the producers in financial literacy29. The MFI lent from its own funds, which when repaid and recycled have resulted in cumulative loans totalling US$ 431,943. The MFI’s requirement is that producers have to be members of a VSL with a track record of saving and repaying loans. Also, they must have sales records to show that they generate sufficient income to repay the loans. For the first year of the scheme (ending June 2019), repayment was 98%. For the 2020 loans for the rainfed season, there were arrears of around 8% of the loan amount, which may still be collected. Overall, the MFI has been very satisfied with the performance of the loans and the programme. “Results have been great.” MFI staff, at mid-term. “The combination with their technical was very valuable”. MFI staff at FE. The latter comment is interesting because it points to the value of MSIKA helping to screen FBOs as to which might be most appropriate and the training for members that has taken place. The staff highlighted that there is a reduced risk for groups where there was irrigation: “Irrigation makes it safer from drought, which is the most critical problem.” MFI Staff The MFI mentioned two ways in which working with MSIKA had changed their views. The first was that they did not know much about lending into fruit and vegetable value￾chains particularly to farmers. They now see that this can be a viable group of famers to lend to. Secondly, they had previously struggled with loans to farmers in the drier parts of Mangochi, with problems of default. Working with fruit and vegetable farmers who have access to irrigation/water sources or are willing to invest in relatively low cost pumps (around US$ 100) has enabled them to develop a new and better loan portfolio, which enables them to sustain their operation in Mangochi. One point of difference between the MFI and MSIKA was that the MFI says it was trying to collect interest during the tenure of the loan as a way to reduce the liability of the farmers and risk. Because the loan was repayable in one payment, MSIKA’s view was that the farmers should not be asked for repayment. As the funds at risk are the MFI’s and they are experienced in loan management, it would have been better to discuss this rather than the MFI and MSIKA before giving farmers contrary information. 28 This has included one default, which FDH thinks was a person who intended to default from the start; the remainder of the portfolio is operating at an acceptable level. 29 Budgeting, saving and record keeping. MSIKA Final Evaluation Page 64 kadale@africa-online.net Overall, the partnership with the MFI has operated very well, meeting the needs of both parties. The MFI says it will continue to work with the farmers that they have relationships with and that are meeting the loan conditions, so the results will likely be sustained. Over time, this may lead them to work with other FBOs, which they now see as “medium risk”. However, they do value having a link with a project to reduce risk and spread some of the preparatory costs, such as training in financial management that helps with loan repayments. The MFI also noted that they have limited funds, so some type of loan facility would have been helpful to expanding their reach. 3.4 Processors The findings in this section are drawn from the KIIs with processors and MSIKA staff, and MSIKA data gathered in the FE. MSIKA supported 18 processors based on their needs, including training in good manufacturing practices (GMP), certification by Malawi Bureau of Standards (MBS), linking them to financial institutions to invest in their business, and linking them with FBOs to source produce.” The access to finance work is covered in the previous sections on MSIKA’s work with banks (section 3.3.2). MSIKA provided monitoring details of 18 processors that it had worked, along with an outline of activities and details of sales. There were two other processors that MSIKA had been in the process of building relationships, but these had either not progressed sufficiently before COVID intervened, meaning there was no time left to proceed. 12 processors that were identified for interviewing, were purposively selected to ensure that there was a sufficient spread of products, size and activities. The consultant interviewed 11 out of the 12 processors. The larger processors were more difficult to schedule interviews with and reluctant to provide sales data to the evaluation team. MSIKA requires that the processors or agro-dealers have some link to the FBOs and beneficiary producers. From the interviews with processors, there were limited links with MSIKA’s FBOs and only relatively small quantities of target crops being purchased. Some of the purchases were for chillies, which was not a focal crop for Year 4. MSIKA had high hopes for the two additional processors to buy crop from FBOs, but there was insufficient time to enable this to happen. 3.4.1 Training and Application of Improved Techniques and Technologies A key part of MSIKA’s approach has been to improve quality of processing and the standards and products. Initial research in the early phase of MSIKA identified that processors often had poor levels of hygiene along with poor manufacturing practices and that their products were not certified, so as to be able to sell them, particularly through retailers. Technoserve (TNS) was commissioned to undertake training in implementing and complying with good manufacturing practices (GMP) and standards, such as the Malawi Bureau of Standards (MBS) standard ‘MBS21’ on hygiene practices, as well as marketing. The training included aspects of innovation, to bring variety to processed products, many of which are the same, such as mango achars and chili sauces. Indicator 32 is the number of processor staff trained in quality standards. The LoP target is 187, which was increased following over-achievement of the target at mid-term. MSIKA reports achievement of 187 (100%), with both male and female targets met. The processors interviewed at the FE confirmed that they had been trained in GMP, hygiene and related topics. MSIKA also engaged with the processors to assess their needs and develop a tailored plan to address hygiene improvements needed for MBS certification process, with MSIKA Final Evaluation Page 65 kadale@africa-online.net steps for measures like upgrading plant/equipment/buildings, undertaking product development, improving marketing and related financial planning. At an early stage, MSIKA decided to split these relationships and the related work for efficiency reasons, with some managed by TNS and the rest by the V37 team. As part of its work, TNS grouped enterprises for a series of group sessions over six months, after which it ended its involvement. This approach is efficient for delivery and has the benefit of processors being exposed to what their peers (and sometimes competitors) do which can bring new thinking. However, this time-limited engagement process did not fully respond to the needs of the processors, as the degree and depth of change required more time input and to take place over a longer period. The processors interviewed were very positive about the way the training was done, and that there had been individual follow ups. Indicator 19 the number of private enterprises, produce organisations, water user associations, women’s groups, trade and business associations, and CBOs that applied improved techniques and technologies as a result of USDA assistance. After the MTE, it was clarified that these related to the application of their operations compared to the original wording that referred to the application of agricultural techniques and technologies. At mid-term, the processors interviewed were able to give multiple examples of changes they had made to their businesses relating to GMP, hygiene and process improvements. The changes identified from the processor KIIs at FE were greater, with a wide range of changes adopted on good manufacturing practices, such as improved, clear and accurate labelling, record keeping, factory improvements (improved toilets), improved cleaning regimes, established standard operating procedures, conducting food analysis and testing, improved storage conditions, accurate records on use of ingredients, taste testing, and acidity testing. Only one processor said that they had not implemented any GMP changes, as the program had not met their expectations (for access to finance); however when interviewed at mid-term, this processor stated they had made multiple changes. "The whole system what we follow now has come from the trainings with LOL. When we receive new consignments of agricultural products, we record the amount, check on the quality and store them well in a bay before production." The processors at FE also stated a range of marketing practices that they had adopted following the training. Examples included undertaking customer research and analysis, improved presentation of the products particularly the quality of labels, setting up a website, using Facebook/Instagram/social media and advertising to promote their products, attending trade fairs, revised branding, changes to pricing and better stock management for distribution. "At the trade fairs we met many customers some who still buy our products. We had a chance to practise some of the marketing skills we learnt in the trainings such as how to talk to a customer." Processor KII Based on the GMP and marketing training, all nine processors that responded on these questions gave examples of new business practices they had adopted that can be attributed to MSIKA. One other business that said they had not adopted practices, had done so at mid-term, but this firm seemed to want to emphasise their disappointment at not getting finance from or facilitated by MSIKA. From MSIKA’s reports on the two processors that did not provide data for the interviews30, it is clear that these two (larger) processors did adopt several practices, notably on GMP. The consultant is 30 One was partially interviewed and the other kept changing the appointment date. MSIKA Final Evaluation Page 66 kadale@africa-online.net satisfied that there was adoption across all processors interviewed and multiple practices were adopted in most cases. 3.4.2 MBS Certification Data is drawn from KIIs and MSIKA data. Indicator 25 is the percentage of registered processing firms in target sectors that obtain certification with MBS related to product quality. The LoP target is 30%, with MSIKA reporting achievement of 44.0%, which is an achievement of 147%. MSIKA commissioned Michigan State University (MSU) to undertake a needs assessment and gap analysis on standards and certification. The work with MBS led to a process for defining and establishing standards. Food processors are required to follow MBS standards, notably on food hygiene. They are also required to meet the standards set for particular food products, where those standards exist. Many of the processors were operating without certification or only with pre-certification status, which potentially creates hazardous situations. There was also an absence of MBS standards for several fruit and vegetable-based food products. Together this meant that processors could not sell their products through formal outlets, thereby reducing their sales. “For example, if we found a shop to carry our products for us they would be thrown out by the MBS. It has happened before. We found markets yes, but we couldn’t display because of the certification.” Cooperative (Mid-term). At FE stage, three of the six processors that did not already have their products certified achieved full certification, with three others achieving pre-certification. These firms explicitly credited MSIKA with helping them achieve this status. Three other firms stated they had maintained accreditation. 3.4.3 Other Support to Processors MSIKA and TNS also offered other support to processors tailored to each business, based on a needs’ assessment. All the processors were offered help with loan facilitation with the Bank as covered in section 3.3.2. Three of the 11 processors interviewed took Bank loans, while the others either self-financed their investments, obtained grants or took loans from other banks unrelated to the MSIKA program. Two firms said they were offered second loans by the bank, but declined them because the conditions had become too stringent: "The agreement required I deposit three quarters of the amount I wanted. For instance, if the loan I wanted was K1 million, I needed to have K0.75 million in my account. I thought this was not a fair agreement. It wasn't helpful." Processor KII. Based on the information in the above comment, it would not make sense for firms to proceed with those loans, not least because interest is charged on the whole loan amount (not netting off the amount deposited), making the effective interest rate much higher for the incremental 25% of the loan value the processor would get over what they already had. While the work with the bank was helpful in the earlier phase, it looks like these conditions were there to deter borrowers, partly linked to the ending of the facility and the bank wanting to protect its position. Training on GMP/hygiene and support towards reaching MBS standards are discussed in the above sections 3.4.1 and 3.4.2. At MTE and FE, there was a range of additional support/activities provided. Book-keeping and financial management, as well as marketing training were cited by most of the firms. Help with operating procedures, planning the factory and identification of new equipment needed was specifically cited by five firms. Six processors stated they had help with product quality testing. MSIKA Final Evaluation Page 67 kadale@africa-online.net There were some planned activities that did not all come to fruition. There were attempts and some agreements to link the processors to FBOs who could supply product. There were two cases where there were purchases, typically for chilis, but these mostly did not result in purchases by processors. There were also efforts to link with buyers. While the link was made, it did not result in new business. "I found a very important supplier for the fruits I use to make my products through one of the trade fairs organised by LOL. They also linked us to Malawi Mangoes, but we were unable to complete the agreement because there was communication breakdown." Processor KII, FE From the KIIs, the processors had high expectations particularly for new (retail) customers/markets and finance. The highest expectation was on finance, some of which was satisfied with the initial round of loans via the bank, but the later rounds seem to have disappointed as the terms were onerous. One processor was very disappointed not to get finance through MSIKA, as that appeared to be their over-riding focus and disengaged when they could not qualify for the loan. There is always going to be a challenge in any development program over unrealistic expectations by some parties with perceptions that this is a source of easy/free money. 3.4.4 Performance data Although MSIKA identified 71 processors in the early stages of the program, MSIKA provided information on 20 processors that it had developed relations with, of which two did not have substantive activities resulting in a net 18 processors worked over the course of the program. Three of these 18 can be described as large enterprises in Malawi’s terms. Two are classed as medium-sized enterprises, with the rest would be termed small (5-20 employees) or micro (0-4 employees). It has been a challenge for MSIKA to find processors with any scale or potential scale. From the FE data, five of the 12 processors31 interviewed had 20 employees or more, while six of the 12 had 10 or fewer full-time/permanent employees. Work with processors contribute to several indicators. The findings on Indicators 19 and 25 are reported in the previous section. For indicator 22, the number of jobs created, the LoP target was reduced following the MTE to 1,657, with many of these expected to come from producers (see section 3.2.16). The contribution of processors was not measured in the mid-term as the number of KIIs was too limited, instead relying on MSIKA reported data. At FE, the processors were asked to give information on employment and nine did so. The additional jobs created for this group of nine processors since the start of MSIKA was reported as 26; however, it is worth noting that the two biggest firms did not provide information and at least two of the processors had been reducing employment due to operational challenges. It was also noted at mid-term and again at FE that firms found it difficult to make a direct attribution of jobs created to MSIKA’s assistance as opposed to a range of factors influencing the business. For indicator 24 – the value of new public or private investment leveraged by USDA assistance – at mid-term, three of the four processors interviewed provided evidence of investment, although the amounts were modest at that point at up to $20,000 for one processor. At mid-term, reported cumulative public and private investment was $550,597 against the overall target of $1,150,000. All of this has been private investment, including the loans taken, against a target of for private investment of $150,000, thereby exceeding the target. At FE, MSIKA reports US $ 3,320,736 in 31 These are all of the large and medium in the listing. MSIKA Final Evaluation Page 68 kadale@africa-online.net investment, against the revised target of US $ 1,967, 799 thereby exceeding the target (169%). This is a combination of investment by producers and by processors. From the FE KIIs, six processors gave information on additional investments in the form of loans received, additional capital from the owners and some grants from other sources (not MSIKA). These ranged from US $ 4,000 up to US $ 20,000 per processor. Other processors did not provide information, so there may have been additional investments that occurred. The investments were wide ranging in scope including equipment, materials, transport, the factory, and sales kiosks for direct selling. It is clear that there has been investment by processors, and much of that is linked to improvements that MSIKA has stimulated them to make. Indicator 27 is an increase in installed storage capacity (dry or cold). It was not possible to get details of increases in capacity in the KIIs due to time pressure and prioritizing other information, such as employment and investment. It is noted that at mid-term, there were examples of processors increasing the quantity, and also the quality of their storage. Indicator 29 relates to numbers of sales agreements with FBOs. Although this indicator focuses specifically on FBOs, there were some expectations of links between FBOs and the supported processors. At mid-term, there was little evidence of sales agreements between the four processors interviewed and FBOs. At FE, two processors stated that they had sales agreements with FBOs for chillies. Three other processors indicated that they had some engagement with MSIKA FBOs, but these did not come to fruition in terms of actual supply of produce. At the mid-term, the consultants interviewed a processor, who had around 20 sales agreements with FBOs, for around 140 mT of paprika and 70 mT of chili in the 2019 recent season. Due partly to its own financial challenges resulting in buying late and to cases of FBO members side selling even though they received seed from the processor, these sales agreements were only partially fulfilled. The challenge for MSIKA is that even if it facilitates agreements between processors and FBOs, these may not be fulfilled due to problems with either or both parties. Other inputs on sales by FBOs to other buyers, such as retailers and hotels/lodges is covered in section 3.1.5 above. 3.5 Government MSIKA has worked with Government of Malawi (GoM) on policy, training of extension staff, research in horticultural crop production and PHH, research into improved practices (Bunda College and MSU), and MBS to develop product standards. 3.5.1 Policy Level MSIKA worked with the Ministry of Agriculture, Irrigation and Water Development (the ‘Ministry’) to develop a new Horticulture Policy. The policy process was initiated by the Ministry and horticulture professionals who wanted to see more focus on horticulture and development of a strategy that would increase its importance for farmers and the country. Initially, the Ministry group was focused on a strategy, but following MSIKA’s interest in a horticulture policy the decision was to look at the policy and strategy concurrently. As reported at mid-term, MSIKA funded a consultant to support the process and a series of planning meetings. MSIKA also supported an online survey of 60 horticulture stakeholders to gather insights for the revised policy. Stakeholder meetings were co￾funded by MSIKA and the Ministry. At the start, the process moved quickly, leading to a draft Horticulture Policy that the Ministry’s Department of Planning needed to align with overarching policies. MSIKA Final Evaluation Page 69 kadale@africa-online.net At the mid-term point, the Ministry said that it did not budget for this alignment stage, so the process was stalled by the first quarter of 2019. A key factor in this was the Presidential and Parliamentary election in May 2019 which created an uncertain political situation and divided attention on side of the Ministry. The outcome of that election was disputed by the opposition, which embarked on a prolonged series of street protests and a judicial process from mid-2019 to early 2020. This uncertainty and regular stopping of activities due to protests, made it difficult to make progress on the remaining stages of the policy process. The courts eventually declared that the election must be re-run. The election was run in late June 2020 and the previous party of government was replaced with the former opposition. A changeover of parties in this manner has an impact on the Ministries, as there is a need to appoint new Ministers and senior staff in the Ministries. Following this changeover, MSIKA has reported that there is still a strong motivation in the Ministry to move the process forward, and that a further stage of the internal consultation with Departmental Directors, has been progressed by the Ministry without MSIKA needing to support this. There are now two remaining stages, including the final wider consultation with regional stakeholders, which MSIKA reports has been scheduled. Once all the Ministry stages have been completed, the Policy is put to Cabinet for approval.32 The process is closer to fruition than at mid-term and there do not seem to be any fundamental hurdles to its ultimate completion, though, as was always the case, the final stages are out of the hands of MSIKA. Beyond finalising the Policy, the Ministry will also need to work on its strategy. 3.5.2 Training District Extension Staff At the outset of the MSIKA project, it was identified that the provision of extension services by GoM staff at district level was a major weakness. To address this, MSIKA worked on training for GoM extension staff in horticultural GAP through a master trainer approach to promote tested and proven farming and PHH practices. Over the course of MSIKA, 267 GoM extension staff were trained in horticulture and PHH practices. Some also received training in marketing as well. For the FE, KIIs were conducted with Agricultural Extension Development Coordinators (AEDCs) and/or Agricultural Extension Development Officers (AEDOs), mainly the latter. In the early phases of the project, MSIKA gave a sub-award to MSU, to conduct a training needs assessment in collaboration with the Department of Agricultural Research Services (DARS) and the Department of Agricultural Extension Services (DAES). From this assessment, MSU developed and adapted training materials that would be practical for extension staff and lead farmers to use for onward training of farmers through a training of trainers (ToT) program. MSIKA also produced extensive materials on growing these crops in both English and Chichewa. The mid-term reported on the agricultural practices ToTs that were first conducted in November 2017. A key learning point for MSIKA was that mixing extension staff with lead farmers did not work well, because the level of knowledge is different. This was changed from the third ToT onwards. With this adaptation, MSIKA’s assessment is that the ToTs were effective in passing on horticulture knowledge and skills. An additional challenge was getting some of the GoM extension staff to conduct their own training of producers. Some GoM extension staff did go ahead and use the materials. However, others wanted additional resources to conduct the training, such 32 A Policy can be developed, but ultimately requires adoption by the Cabinet. Once the technical process is complete, a Policy should not face rejection or significant change at Cabinet level, though it can be returned with instructions for changes. MSIKA Final Evaluation Page 70 kadale@africa-online.net as allowances for themselves and for trainees. MSIKA was neither willing nor able, to provide financial support. Because of this mixed response by GoM extension staff, MSIKA states that it focused on those extension staff that are committed. MSIKA works through lead farmers, which is discussed in more detail in section 3.1.4. MSIKA notes that there is also a mixed response from lead farmers, as with extension staff. It has therefore focused on those lead farmers that show sufficient commitment. From the KIIs with AEDCs/AEDO, there was confirmation that the training of GoM extension staff had taken place and that the training had been used for onward training. All had attended the agricultural and the PHH training. "I attended a Training of Trainers then afterwards together with the Extension Workers from LOL we went to train the farmers in the FBOs." KII GoM Extn Worker, Ntcheu. '"The training was done in my section and I was acting as a supervisor in those trainings and offering technical advice." KII, GoM Extn. Worker, Lilongwe Some of the extension workers highlighted the joint nature of the training of farmers. There were some extension workers who saw their role as more passive: '"Together with the staff from LOL, we were training the farmers. Whenever the staff from the District office were present, I was just there to attend the training without any active role.” KII, GoM Extn Worker, Mangochi. The feedback on the training was generally positive, with a few issues, notably two who felt the training was rushed and one who felt the examples given by trainers from outside Malawi were not relevant enough. In terms of implementation of the training, three mentioned that they were not so involved in the onward training or knowing what training was being conducted. One commented about the insufficiency of MSIKA staff for delivering the onward training: "The content was good, just that perhaps they employed fewer staff and kept changing them and that affected smooth continuity of activities and farther outreach.” KII, GoM Extn. Worker, Mchinji On the PHH training, the general response was that it was good and the timing was good, however in some cases the officers said that the timing was not good, so that the ‘theory’ was covered, but it was too early for practical demonstrations of harvesting. There were also a few comments about not all the EPA was covered with the training. "Content was good. There were some differences in what the Government and LOL Extension Workers know about PHH but after discussions a consensus was reached" KII, GoM Extn. Worker, Mangochi. "It was 100% what was supposed to be taught." KII, GoM Extn. Worker, Lilongwe. " For example, tomato PHH should really be demonstrated on tomato at that stage, in real time. That way, farmers most of whom are illiterate can learn better by observing and practicing." KII, Govt Extn Worker, Mchinji There were some improvements suggested, such as better communication and co￾ordination, clarity on roles for the GoM staff, unfulfilled promise on demo plots and related seedling supply, and producer groups being too big to be effective: "They should communicate beforehand what role I am going to take, say a facilitator. That way everyone can thoroughly prepare and work better as a team." KII GoM Extn. Worker, Mchinji. “The major problem was attendance. In a single session, attendees would be 50 or more and to me, this affected some farmers especially those that take time to grab things. In future trainings, the number of sessions should be increased and the number MSIKA Final Evaluation Page 71 kadale@africa-online.net of those attending a single training session should be reduced so that everyone is able to understand." GoM Extn. Worker, Mangochi "We were surprised that after the training, they did not involve me or any extension worker in any of their activities. We were only hearing everything from the farmers and we would see some of the activities from afar." KII, GoM Extn. Worker, Lilongwe. One reflection from MSIKA was that the younger GoM extension staff were more responsive and more likely to implement and support the trainings. There was also work by MSIKA FBCs and lead farmers on demonstration plots. The feedback from most of the GoM extension staff was that the content of these was relevant to the farmers, though there were several comments about issue with organisation and timeliness. Overall, the comments partly reflect perceptions of roles and a view from the GoM staff side that they were supposed to be in control. Current GoM policy on extension is for plurality, meaning any provider is able to provide it, though with effort to co-ordinate. Sometimes there is a desire by GoM staff to control the program and resources that may not be matched with their own delivery. MSIKA, like other projects has had to get the right balance, though co-ordination, collaboration and good timing are all helpful, if difficult to achieve all of the time. 3.5.3 Malawi Bureau of Standards A further aspect to the MSIKA project was its work with the Malawi Bureau of Standards (MBS) on the development of product standards. MBS has several standards relating to food processing, such as MBS21 concerning food hygiene. However, there were gaps for several products that can be made from MSIKA’s target crops. As a result, nine products were identified that needed standards, as well as one standard that MBS wanted to include. There is a well-defined process for introducing a new/adapted standard that involves initial research to see if there is a standard that can be adopted or adapted. If there is no international standard, then there is a defined process to develop a national standard. The process involves examining other standards that might provide some base information or that can be drawn upon for the new standard. This research/ formulation is followed by a formal process of consultation through a technical committee consisting of processors, research institutions, the Consumer Association of Malawi, the Ministry of Health, the City Councils (who may need to enact by-laws) and academic. After there is agreement on the draft standard, MBS circulates the draft standard to the World Trade Organisation and on to the international community. The major challenge for MSIKA in this process has been the election and its re-run, as until there is a Government in place, the Board of parastatals like MBS cannot be appointed and this delayed ratification of standards. However, MSIKA reports that approval has now been given by the new Board of MBS, and this will be followed in the near future by formal Gazetting33, which is in the hands of the Ministry of Justice. In summary, the work with MBS moved quickly, was held up by the elections, but potentially now can reach fruition. 3.5.4 Bunda/MSU This section draws mainly on the findings from the MTE. MSIKA supported work by MSU and Bunda College on soil fertility through mounting soil health trials using different combinations of organic/inorganic fertilisers/composts for MSIKA’s target crops. The aim was to determine what physical, chemical and 33 Gazetting is the process where by regulations are officially published and so become operative. MSIKA Final Evaluation Page 72 kadale@africa-online.net biological changes these combinations result in, and the overall impact on plant health and yields. The results were to be determined through field trials in the five districts for the seven crops. Alongside the research are demonstration plots for farmer field schools, at which extension staff and producers can see the results and learn about the research. This was a collaboration between MSU, as the contractor to MSIKA, and Bunda College (‘Bunda’) with two academics engaged in this work. The academics at Bunda College set up the trials following a methodology agreed with MSU, and monitored the trials, collecting the necessary data for analysis, and sending data and samples to MSU for further analysis. The trials focused on testing different compost/fertilizer combinations, as well as use of drip irrigation The Bunda academics reported at mid-term that the work is progressing, and the relationship with MSU was working well, but that access to transport has restricted their work. In terms of results, the academics’ conclusions are that combining organic with inorganic compost/fertilizer is positive for the biological measurement of the soil, with an increase nematode numbers and good crop yields. They have made recommendations on soil health that will be incorporated into MSIKA’s final training materials. Getting the most out of this investment in research requires the findings to be incorporated into the training and demonstration work. With the early curtailment of MSIKA, the timing of this is too late to be cascaded out to producers. 4 Conclusions, Lessons and Recommendations This section sets out the conclusions of the final evaluation and recommendations that USDA and V37 should consider for future similar projects. 4.1 Key Conclusions and Lessons Conclusions and lessons are set out relating to the main groups of stakeholders covered in the evaluation: FBOs, producers, financial institutions, processors and government. Prior to these, there is a short section on the methodology conclusions and lessons relating to using a phone survey for producers. 4.1.1 Methodology The use of a phone survey for producers and other KIIs was necessitated by the advent of COVID. This change, part way through the design, required the reformulation of the instruments to reduce the content to be covered in a maximum 45-minute call. Importantly, this change in method also required MSIKA to obtain phone numbers for a randomly drawn sample of 8,000 producers. In the end it was only possible to get phone numbers for 2,973 of those producers (37%), all of whom the consultants attempted to contact. From this list, the consultant was able to reach 979 producers (33%), some of whom were interviewed on two crops in order to reach the required levels of confidence (90%) and margin of error (5%) for each crop. The approach and scale of this phone exercise was new to both MSIKA and the consultant, so there was inevitably a great deal to learn. A key lesson is that low phone ownership means that a sufficient sample is needed to get sufficient respondents (979 respondents out of a list of 8,000 names), with a call success of around one third due to wrong numbers, phones not available, no signal and respondent battery running out mid interview. Sending a modest cash incentive via airtime to the respondent when the interview was complete helped ensure collaboration, though there were some challenges making sure all respondents MSIKA Final Evaluation Page 73 kadale@africa-online.net received their airtime and some questions around how it should be shared if a respondent used another person’s phone. Overall, the exercise showed that it was feasible to undertake such a survey. Another lesson is that there are some questions that are difficult to verify over the phone. For example, the findings on the area of new and refurbished storage for producers were high, and it is likely that if an enumerator had been present, the enumerator could have seen or paced out the floor area and indicated the height so as to reduce errors in estimation. This only became apparent when the overall calculations were done and were much higher than expected. A further lesson is that using the phone may introduce a male bias, as ownership and/or control of the phone in Malawi may favor male participation. There may also be a bias in that there was a relatively high proportion of lead farmers in the sample, possibly because their numbers were known in the FBOs, so they would appear on most FBO lists, perhaps disproportionately. The male and lead farmer biases in the sample are not easy to resolve in future exercises, but the consultant did undertake some interviews with respondents using the phones of other respondents which partly offsets these potential biases. 4.1.2 Overall Progress Across its many indicators, the MSIKA program achieved or made considerable progress towards many of its LoP targets (see summary results, in Annex 1). Several factors have impacted on the program in the last two years, in particular: 1. The life of the program was shortened from five to four years and several planned activities were dropped due to a shortfall in the expected resources. The shortening of a program was disruptive for two main reasons. First of all, there is one year less of implementation of activities, some of which are part of a sequence and package that might result in the project not getting the most out of each of the components of its work. An example of this would be fully using the results of the Bunda College research trials to inform training and dissemination of better soil management practices. The second main effect is that activities that are designed to build up over time, have not had sufficient time to realize their full results. For example, MSIKA invested heavily in the training of extension staff and lead farmers and the onward training of farmers alongside project staff. It takes time for farmers to adopt the full set of practices, as they are likely to adopt the easier, lower costs ones first, see the results and progressively build their confidence to apply the more difficult and costlier ones. MSIKA has been increasing the numbers of farmers trained and the range of practices they have been trained in, but the full increase in impact will not be seen within the life of the project. An extra year would be expected to result in higher performance by producer beneficiaries. 2. A second major factor for MSIKA’s implementation was COVID, which began to affect Malawi towards the end of March 2020 and into the next two quarters (Apr￾Sep 2020). While the impact of COVID has not (at least not yet) seen widespread loss of life in Malawi, the GoM reacted to the pandemic by imposing a series of lockdown measures that restricted movement, restricted markets and closed tourism and hospitality venues and institutions that are major users of fruit and vegetables. This impacted on producer production and sales, and it restricted the operation of FBOs, processors and the MSIKA team. This reduced planned activities and contributed to a shortfall in several targets. The effects of the COVID restrictions have also hampered MSIKA’s closing out activities and its ability to implement its sustainability strategy. MSIKA Final Evaluation Page 74 kadale@africa-online.net 3. Malawi had a Presidential and Parliamentary election in May 2019, which was heavily disputed. There was considerable disruption in the cities and a risk of violence, that limited activities by MSIKA and other partners/stakeholders, notably GoM Ministries, MBS and DADOs. The eventual rerun of the election led to a change of Government which in turn resulted in changes at senior levels in Ministries and governmental bodies. The extended uncertainty in government bodies disrupted some of the policy-related work and field work. 4. In March 2019, Cyclone Idai hit Mozambique, with the effects reaching Malawi. These occurred on top of a period of prolonged heavy rains, resulting in widespread flooding and crop losses. The effects of Idai were seen in the MTE, where several results were impacted, notably yields. While in some ways the effects are short term in that farmers can plant again for the next season, the loss of crop(s) undermines farmers’ resources and reduces what they have to invest in the next planting. It can also have damaging effects on the land quality, if soils are washed away, and on buildings for storage, if these are damaged and the farmer cannot afford to repair. The above factors are not intended to provide reasons for shortfalls in performance, which overall was good; rather they are a reminder that implementation of projects can be disrupted by multiple factors beyond the control of the project team. In the case of MSIKA, there have been multiple factors, some of them concurrently, such that it has affected implementation. The restrictions due to COVID and the disruption from the election and its aftermath did take a toll on program activities, restricting field operations that are central to its operations. The lesson for USDA and V37 is that while it is not possible to predict the specific nature and timing of disruptions, it is predictable that there may be one, consecutive or concurrent disruptions of some kind in a volatile operating environment like Malawi. This should be factored in when planning what is realistic to deliver and achieve in a project of this nature in a country like Malawi where uncertainty is the norm. 4.1.3 Farmer Based Organisations MSIKA has worked with 217 FBOs across the five target districts, which is above the LoP target of 210. Of these, 167 (77%) were formed by MSIKA. FBOs provide a useful mechanism for organising farmers for engaging with projects, receiving project inputs (e.g. training), which is much more efficient than engaging directly with farmers. They may also have potential as sustainable organizations for collective activities by farmers e.g. for purchasing inputs and sales of produce for mutual benefit. MSIKA assessed that 27% of its FBOs are high performing, 39% are medium performing and 34% are low performing. For Year 4 of MSIKA, following the need to restructure based on fewer resources and the recommendation in the MTE to focus on better performing FBOs, MSIKA planned to focus its work on 50 better performing FBOs. This plan was disrupted by COVID, though the MSIKA team was able to partially implement this narrower focus. From the performance data and the KIIs across the spectrum of FBOs, the consultant’s conclusion is that it is difficult to bring low performing FBOs to be medium or even high performing and that working with these dilutes the effort of teams that are under pressure to deliver many activities. The lesson for USDA and V37 is that MSIKA and similar projects are likely to make more progress and achieve better returns by moving early to focus on better performing FBOs. Determining performance can only be done following some engagement; however, V37 has considerable experience in working with FBOs and has tools for this type of assessment, such that early decisions could lead to dropping low performing MSIKA Final Evaluation Page 75 kadale@africa-online.net FBOs at the earliest opportunity. That does not mean that some low performing FBOs could not improve with time and effort; rather it recognizes that many do not improve much, as they often have weak leadership that is looking for projects to do everything for them and to provide resources. MSIKA’s FBCs were stretched and this would have enabled them to be better focused. As stated above, FBOs are very useful mechanisms for engaging with farmers and some may ultimately become sustainable organizations. However, the consultant’s conclusion is that it is unrealistic to expect that most have the potential to become sustainable entities. It would have been helpful for MSIKA to set out its expectations for FBOs more clearly at the outset. The lesson related to this is that it is important to be realistic about the potential to turn most FBOs into well-functioning, viable and self￾sustaining registered entities, such as Associations or Co-operatives. This is unlikely in most cases, at least based on the MSIKA FBOs. The FBOs that MSIKA formed are based on multiple small clubs of 10-15 members, brought together into a newly created larger entity of up to 300 members. While it is not feasible to have very large FBOs at least early on, the MSIKA FBOs are too small and too weak to become viable entities in their own right. With leaders elected each year, and all the challenges associated with self-serving models of leadership in Malawi, it is highly unlikely that many can move beyond being mechanisms for organising farmers to engage with projects, like MSIKA. The progress made on governance capacity building (section 3.1.3.5) and overall in capacity development from the PMM assessment, suggests that it would take many years and considerable investment to develop them, with a very uncertain success rate. MSIKA’s own estimate at mid-term was that up to 10% of FBOs have potential to become co-operatives. The consultant’s conclusion from the interviews with FBOs and MSIKA’s FBO data is that this is an optimistic assessment, and that fewer than 5% might have some potential, perhaps as low as 1-2%. For projects like MSIKA, FBOs may best be seen as temporal mechanisms to engage efficiently and effectively with farmers, than to build into a project design plans and resources to try to turn them into sustainable entities or expect them to continue operation beyond a project life where there is no ongoing support. If there are a few FBOs that have potential, such as if there are pre-existing FBOs that have been able to progress and have larger memberships, then a more focused support package for the few would be a more realistic approach. FBOs and lead farmers have been a key mechanism for delivering a range of training modules including agricultural/horticultural practices, PHH practices, marketing and finance. The delivery of some of the training has been disrupted by the wider factors that impacted on the project and that the MSIKA technical team and FBCs were stretched. This relates back to the point about the level of ambition for what can be covered, the focus on fewer FBOs (be more selective earlier) and acceptance of FBOs as temporal entities (in most cases). Related to the this, it is not realistic that these FBOs would be able to invest or be able to operate storage and processing facilities in the foreseeable future. As noted in the evaluation, MSIKA has been very successful at increasing individual producer yields, and has made progress on PHH losses. Individual producers have generally increased their sales. However, progress with collective selling via the FBOs has been limited. While there have been sales agreements made, these have not always been fulfilled, partly because aggregation capacity is weak and there is still a strong preference by producers to sell individually. There is potential interest in links between producers and HAI and potentially Universal Industries, but this started late in the program and was disrupted by COVID, so did not reach fruition. Linking producers to markets is a strong theme for MSIKA and something that many projects MSIKA Final Evaluation Page 76 kadale@africa-online.net that work with producers want to achieve to convert success in increasing production volumes (and quality) into sales and incomes. This am is important to V37 and USDA. The lesson from MSIKA is that linking producers to more formal markets with expectations on quality, volumes, timeliness and ‘packaging/presentation’ is very difficult and may depend heavily on the ongoing facilitation efforts by a project and private sector businesses that are willing to persevere. In the KIIs, some FBOs said they had sales agreements, but they felt that they were not able to fulfil the requirements, so they pulled out of those agreements. It is noteworthy that although COVID disrupted the supply to formal markets such as lodges/hotels, conference venues, and bigger retailers, the producers reported that they still sold what they had produced, primarily to local markets and probably at poorer prices than the suspended formal markets. It is a major challenge to facilitate sustainable links between newly formed FBOs (with members who prefer to sell individually) and processors or other buyers with varying and relatively unpredictable demand, and who may switch to other suppliers/producers when there is a glut of supply and prices can be pushed down. FBO and member expectations were that MSIKA would help the FBOs and their members to find attractive formal markets with better prices. MSIKA played a part in raising the expectations as that was part of the project’s goals; however, these expectations look unlikely to be met given all the challenges in accessing such markets. Although this may be disappointing, it appears that the producers have still found markets for their produce even under COVID, mainly because they have to sell and there are retail and wholesale markets they can currently reach, even if these are sub￾optimal in demand and prices. With substantially increased production, these producers are still likely to be better off even with lower prices from these informal markets. Over time, markets may become more structured with the better organised FBOs finding ways to aggregate to supply more formal buyers. That process can be facilitated by a project, but it takes time and expectations of supply to formal buyers, such as processors and bigger retailers/hospitality outlets should be kept low. 4.1.4 Producer Households The FE found that MSIKA succeeded in reaching large numbers of producers (39,744) with training in improved agricultural and PHH practices that is valued, efficiently delivered and that is effective. The adoption rate of practices has been very high, with 99.5% of producers applying at least one new practice. With a slightly higher population of producers trained in agricultural practices, MSIKA would have hit its target for indicator 7. As noted, COVID had an effect on the overall numbers and the target would have likely been over-achieved, if it had not occurred. The extent of application of practices was impressive, with a mean 15.8 practices per producer per crop. This widespread application is likely to lead to higher yields. One note of caution is that there was a higher proportion of lead farmers in the FE sample compared to the FE population and mid-term – lead farmers would likely be expected to apply more practices than other producers. As noted at mid-term, the baseline for crop yields needed some revision to account for outliers. It was noted that progress on crop yields at mid-term was affected by Cyclone Idai, yet still showed reasonable progress by MSIKA. With better weather conditions at FE, another year of training, and opportunity to apply practices under the guidance from lead farmers, and with the narrower focus on the four crops, the results on yield at FE have been very good with a 54.4% aggregate weighted yield improvement against a revised target of 26.0%. MSIKA Final Evaluation Page 77 kadale@africa-online.net In terms of PHH, the MTE found relatively low beneficiary unprompted knowledge, yet high application rates for improved PHH practices for field crops. At FE, the applications rates have improved further, with tomato scoring over 90% application on most practices. Mango, like all the tree crops in the MTE still lags the field crops, reflecting that tree crops are not seen by most producers as crops to actively manage and invest in. The overall effect was that PHH losses reduced from 14.8% at baseline to 13.1% by FE. Losses are very difficult to measure and so caution is needed on any PHH loss data. Somewhat surprisingly given the disruption of COVID on markets, sales volumes increased substantially compared to the baseline, with the highest increase in mean sales volume for tomato at +70.0%, followed by potato (+28.8%), onion (+21.1%) and mango (+6.1%). The low increase in mango is not surprising, but the big increase in tomato seems to be a function of the increase in volume of the crop, with an estimated 68.8% increase. The sales value generated interesting results in comparison with the adjusted baseline. The sales of the four crops fall within a similar range with tomato at $1,229/ha followed by onion at $1,226/ha potato at $1,195/ha and mango at $996/ha. However, comparing these to the baseline there were big increases in sales value for mango (+99.7%), followed by potato (+92.7%), onion (+63.3%) and tomato (+44.1%). Part of the effect is down to higher MK prices (tomato being the exception at minus 8.0% and a relatively stable exchange rate of the Malawi Kwacha to the US dollar which overstates the value of changes in MK prices over the project period. However, the consultants still found relatively large sales increases even after adjusting for MK price changes. In summary, there has been good progress since the baseline and the mid-term in application of agricultural and PHH practices, with large increases in yields, in sales volumes and in sales value. There has been modest progress on PHH losses, noting that there are challenges in measuring losses. 4.1.5 MFIs/Banks/VSLs MSIKA continued with its SME lending mechanism with the bank. This came relatively late in the project due to finding and negotiating the terms of the facility. The initial results at mid-term were pointing in the right direction, though the bank was unsurprisingly a bit cautious and a bit slow in assessing and disbursing loans. While there was hope for bigger and more flexible second loans, with the winding up of MSIKA and a Presidential declaration on a moratorium on SME loan repayments, the bank made the terms much less attractive and deterred several SMEs that had been approved. In the end, the total number and value of loans disbursed through the facility was much lower than originally intended, highlighting the difficulties of establishing a successful SME loan facility. The lesson is that facilities for SME lending are not attractive to all banks/lenders and that the banks have their own requirements that may override what the project is seeking, such as loan values, collateral and payment terms. Banks are cautious, but they are also regulated, and caution can reflect regulatory concern over what is seen as relatively high risk lending to SMEs. MSIKA’s work with an MFI to increase agricultural lending has been very positive with both parties regarding the initiative as a success. The MFI has been the biggest source of lending to MSIKA beneficiaries ($413,100) compared to banks ($193,343) and VSLs ($74,592). The approach is low cost for MSIKA and proving to be very effective, as the MFI is using its own capital. The MFI has been encouraged by the high repayment rates from the well-organised and trained MSIKA beneficiary producers. It has seen that there is lower risk in horticulture due to the relatively short growing periods meaning there can be up to three cycles a year if there is access to irrigation. The MSIKA Final Evaluation Page 78 kadale@africa-online.net challenge for the MFI has been that it has limited capital to on-lend, but it is very reluctant to borrow at commercial rates, not least because monthly interest rates are now capped by the Reserve Bank, limiting what is viable for the MFI. The MFI is a well-established MFI and will continue to offer these loans to these and potentially other MSIKA beneficiary producers after the end of MSIKA. This means the result is sustainable for the immediate future. The lesson for projects like MSIKA is that, while there can be value in exploring relationships with banks and sensitising them to the needs of agri-business SMEs, MFIs may be a more fruitful partnership than banks. The positive effect of MSIKA’s work to establish VSLs for FBO members was highlighted at the mid-term. MSIKA has now established 751 VSL associations, improving access to finance for over 10,000 beneficiary producers. VSLs are self￾sustaining community managed mechanisms with a high degree of continuity. Although the value of loans through VSL is small compared to the MFI and even the bank, the number of loans is higher than these partners indicating that many producers (5,444) are accessing small loans in their localities and are able to get a share of the profits on the lending when the VSLs distribute their funds. The lesson is that projects like MSIKA should continue to establish VSLs and train members in VSL. 4.1.6 Processors MSIKA has established relationships with 18 processors. This has involved application of GMPs/hygiene standards, financial management, storage, processing, access to finance and access to markets. There is high uptake of GMPs and hygiene improvements, increases in storage, certification to MBS standards, improvements in processing with indications that further progress will be made, adopting of marketing practices and financial management practices. However, as was the case at mid-term, the outcomes of this work are relatively limited at FE, particularly for purchases from FBOs and additional employment. MSIKA believes that with a fifth year that it might have been able to increase the purchases from FBOs, and this might happen based on the relationships with two processors. In conclusion, development of processing in these value-chains is beneficial overall, however, there is not yet a strong connection between the processors and the producers and their FBOs. Five of the 18 processors are larger or medium enterprises, but most are micro, with less than 10 employees and with limited potential to increase investment, purchases, storage/warehousing and employment. As noted at the mid-term, MSIKA’s efficiency and effectiveness was limited from investing time and resource in these micro￾enterprises. At the outset, MSIKA identified 71 processors as potential partners, but most were micro-enterprises and the pool of larger and medium processors was limited to less than ten. It has proven difficult to make much headway with processors due to their relatively small size. The work has led to better understanding and change in the smaller processors, and may ultimately lead to wider changes over time. The lesson is that MSIKA was better focusing on the five processors that have shown willingness and capability to make progress, while recognising that there is still limited potential with processors in Malawi at this point in time. 4.1.7 Government As noted at mid-term, MSIKA engaged GoM on policy, training of extension staff, work with MBS and research on crop practices/farmer field schools. Updating the Ministry of Agriculture’s Horticulture Policy, moved quickly in the early stages due to the support from MSIKA, however the preparations for fresh Presidential election and its aftermath, as well as the major effects on government due to COVID, disrupted and slowed down progress. MSIKA has moved the policy process along and there is a good chance that the policy will be adopted in the future, but the lesson is MSIKA Final Evaluation Page 79 kadale@africa-online.net that policy change of this nature is often very slow, even with strong commitment within government and prone to delays, such that success is out of the hands of projects like MSIKA. The ToTs for GoM’s district-based extension officers have relevant content based on a sound participatory process of development. At mid-term, it was recognised as necessary to train GoM district extension staff, as there can be difficulties operating in the districts if they are excluded or do not feel involved. At FE, the KIIs found that Govt staff want to be involved in the co-ordination of activities, but some see this more as a matter of being in control, than necessarily to be engaged in delivery. Some AEDOs/AEDCs have been involved and assisting on delivery, but the GoM extension staff are not likely to be a key driver in onward training. Rather, it is the lead farmers that are the main mechanism for onward delivery. The lesson is that projects like MSIKA have to include GoM staff, but should focus on training and supporting lead farmers as the most sustainable, effective and efficient mechanism to reach producers. MSIKA worked with MBS to develop 10 new product standards. This was also held up due to the hiatus of the election period and its aftermath and eventual re-running. It is likely that the 10 new standards will be adopted. The lesson is that working on regulations or supporting measures may yield more results than working on national policies and strategies, and may be of more immediate relevance to the private sector. MSIKA’s engagement with MSU/Bunda on research was linked to the farmer field schools at the demonstration sites. Getting the most out of this investment in research requires maximising the use of the demonstration sites for the farmers field schools. There are challenges working with academia including resourcing (such as transport) and the speed of delivering results. It is unclear at this point whether MSIKA will be able to leverage this research to bring forward practices that can be readily adopted by many producers. The lesson is that research may not be able to bring results in the project period and it is probably better to focus on demonstrating under-utilized knowledge on proven practices. 4.2 Recommendations The following recommendations are made: #1 That to undertake a phone survey of producers requires access to a sufficient database of phone numbers, as representative enough of the population as possible, and may require additional efforts to get female producers included. Considerable thought should be given to question inclusion, such as those that may rely heavily on face to face interaction with respondents. #2 That in the design of projects of this type, USDA should consider in its expectations and ambitions that there is a high likelihood of consecutive or concurrent disruptive events. While these can be listed as risks and assumptions, the flexible response shown by USDA on MSIKA can avoid projects scrambling to make up ground and hit pre-determined numbers. #3 That projects like MSIKA should aim to identify and select out low performing groups and FBOs at an early stage using appropriate tools and drawing on experience. #4 That projects like MSIKA should not aim to turn FBOs into sustainable entities in most cases, but be highly selective in deciding which might have potential, and to see most FBOs as temporal entities for efficient and effective project delivery that may or may not continue post project. #5 That projects like MSIKA should seek to facilitate sales between better organised FBOs only where there is a realistic capacity to supply and members are committed to collective selling and that there are accessible markets. Where that is not possible, MSIKA Final Evaluation Page 80 kadale@africa-online.net there is still merit in working to increase production for sale by individual producers into less formal markets. #6 That because Banks are regulated and relatively cautious, projects like MSIKA need to consider other more flexible and responsive lenders, such as well-established MFIs. #7 That VSLs should continue to be a focus for projects like MSIKA as they provide ready access to finance and over time, the VSL builds trust and cohesion between producers. #8 That projects like MSIKA should find an appropriate balance between working with enough processors, but of sufficient size and capacity to invest. Working with many micro- and small-enterprises is resource intensive with limited potential to deliver results. Work with smaller enterprises should be limited to training and general information rather than more intensive support relationships. #9 That projects like MSIKA embarking on support for developing government policies and strategies need sufficient time and resource to see such processes through to the end and that it should be accepted at the outset that there is a risk that these processes may not yield results within the project life or at all. #10 That projects like MSIKA should focus on lead farmers as a more sustainable, effective and efficient mechanism for delivering services, particularly knowledge, than government extension staff. However, government structures should be engaged and government staff included in training and activities where they can support of lead farmers. Kadale Consultants Ltd., Plot 244, Bwaila Road, Area 15, Lilongwe, MALAWI. kadale@africa-online.net Tel ++ 265 (0)1 770000 MSIKA Final Evaluation Page 81 kadale@africa-online.net Annex 1: Table of Program Indicators Performance Indicator Impact Baseline Value Year 4 target (NEW) Y4 TOTAL LOP TARGET (NEW) LOP ACTUAL % Achievement (Y1-Y4) COMMENTS 1 Percentage change in yield of beneficiary producers as a result of USDA assistance Baseline Excluding Outliers (kg/ha) Tomato 2,617 Onion 3,575 Potato 3,329 Mango 4,347 Guava 3,796 Citrus 5,517 Chilli 467 26.0% 54.4% 26.0% 54.4% 209% The indicator values are from the final evaluation survey, conducted by Kadale, with 1,074 responses (individual crops) drawn from a sample frame of 30,922 trained producers that at least received basic agricultural production training before 31st May 2020. The overall yield for Yr4 (now end of project - EOP) improved by 54.4% compared to the baseline, exceeding the Yr4 target (26%). Tomato improved the most (68.8%) followed by mango (62.8%), potato (48.1%) & onion (36.9%). Actual Yr4 yields are: Tomato: 4,418 Kg/Ha Onion 4,896 Kg/Ha Potato 4,931 Kg/Ha Mango 7,078 Kg/Ha Weather conditions for 2019-20 were much better than 2018-19, where the impacts of Cyclone Idai suppressed yield gains. The extra period has also enabled more farmers to implement practices and to see the benefits of these. MSIKA's other activities on inputs, access to finance, etc., have also contributed to the higher results. Kadale set outlier values at the FAO maximum yields from applying Good Agricultural Practices (GAP) for these crops. Percentage change in yield of beneficiary producers as a result of USDA assistance-Male 28.9% 54.7% 29.0% 54.7% 189% Percentage change in yield of beneficiary producers as a result of USDA assistance-Female 23.6% 61.8% 24.0% 61.8% 258% Percentage change in yield of beneficiary producers as a result of USDA assistance- Tomato 25.0% 68.8% 25.0% 68.8% 275% Percentage change in yield of beneficiary producers as a result of USDA assistance- Potato 29.0% 48.1% 29.0% 48.1% 166% Percentage change in yield of beneficiary producers as a result of USDA assistance- Onion 30.0% 36.9% 30.0% 36.9% 123% Percentage change in yield of beneficiary producers as a result of USDA assistance- Mango 21.0% 62.8% 21.0% 62.8% 299% 2 Number of individuals benefiting directly as a result of USDA assistance 0 8,000 11,433 36,000 41,185 114% MSIKA reached 2,319 individuals amid the COVID-19 pandemic (April to Sept 2020) mainly through activities that sustain impact. The project still reached 4,265 (new) individuals in the final year as FBOs continued to grow their membership. Throughout the 4 years, the project reached 41,185 participants, which is 114% of the target. Farmers, government staff, processor staff & civil society have benefited from MSIKA interventions. Farmers are the biggest beneficiary group (95%). Number of individuals benefiting directly as a result of USDA assistance- Male 4,480 4,890 20,160 18,433 91% Number of individuals benefiting directly as a result of USDA assistance- Female 3,520 6,543 15,840 22,752 144% Number of individuals benefiting directly as a result of USDA assistance- New - 4,265 36,000 41,185 114% Number of individuals benefiting directly as a result of USDA assistance- Continuing 8,000 7,168 - - MSIKA Final Evaluation Page 82 kadale@africa-online.net Performance Indicator Impact Baseline Value Year 4 target (NEW) Y4 TOTAL LOP TARGET (NEW) LOP ACTUAL % Achievement (Y1-Y4) COMMENTS 3 Number of individuals benefiting indirectly as a result of USDA assistance 0 36,800 52,592 165,600 189,451 114% 10,667 indirect individuals have benefited from MSIKA interventions in this reporting period, with a total of 52,592 in year 4 against a target of 36,800. The indicator is calculated by multiplying the number of direct beneficiaries by 4.6, which is the average number of individuals per household minus 1 (the direct beneficiary). 4 Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance 0 8,000 10,669 36,000 39,744 110% MSIKA continued fostering agricultural sector productivity in the April - Sept period with 1,427 farmers trained in ag￾production, marketing & financial literacy, bringing the Yr4 total to 10,669 thereby over-achieving the Yr4 target (8,000). Cumulatively, MSIKA has trained 39,744 individuals against a target of 36,000. 98% of the individuals trained are farmers. The indicator counts individuals that have received training in ag-production, marketing, financial literacy, and post-harvest handling. Participants include farmers, government staff, people in civil society and people in firms. Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance- Male 4,480 4,619 20,160 17,847 89% Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance- Female 3,520 6,050 15,840 21,877 138% Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance- New - 4,256 - 39,744 - Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance- Continuing 8,000 6,413 - - - Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance- Producers 8,000 10,507 36,000 39,332 109% Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance- People in Firms - 2 - 97 - MSIKA Final Evaluation Page 83 kadale@africa-online.net Performance Indicator Impact Baseline Value Year 4 target (NEW) Y4 TOTAL LOP TARGET (NEW) LOP ACTUAL % Achievement (Y1-Y4) COMMENTS Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance- People in Government - 150 - 267 - Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance- People in Civil Society 0 10 0 48 - 5 Percentage of producers utilizing improved soil fertility management practices 81% N/A NA N/A N N Indicator was removed following modification of PMELP Percentage of producers utilizing improved soil fertility management practices - Female 76% N/A NA N/A N N Percentage of producers utilizing improved soil fertility management practices - Male 86% N/A NA N/A N N 6 Number of hectares of land under improved techniques or technologies as a result of USDA assistance 0 6,581 7,909 10,801 12,129 112% Based on the 1,074 sample responses in the final evaluation (30,922 agric trained farmer sample frame), a calculated 7,909 Ha of land was under improved techniques and technologies (T&T) in Yr4, surpassing the Yr 4 target by 121%. Overall, MSIKA achieved 112% of the revised LOP target. High adoption rates contributed to the increased hectarage under improved techniques & technologies. In terms of the disaggregates, after V37 consultation with USDA, we decided to exclude questions specific to measuring these disaggregates to shorten the interviewing time because the expected in-person interviews were converted to phone interviews due to COVID-19. Number of hectares of land under improved techniques or technologies as a result of USDA assistance - New 5,515 NA 9,052 NA - Number of hectares of land under improved techniques or technologies as a result of USDA assistance - Continuing 1,029 NA 1,689 NA - Number of hectares of land under improved techniques or technologies as a result of USDA assistance - Crop Genetics 4,910 NA 8,059 NA - Number of hectares of land under improved techniques or technologies as a result of USDA assistance - Pest Management 5,536 NA 9,086 NA - MSIKA Final Evaluation Page 84 kadale@africa-online.net Performance Indicator Impact Baseline Value Year 4 target (NEW) Y4 TOTAL LOP TARGET (NEW) LOP ACTUAL % Achievement (Y1-Y4) COMMENTS Number of hectares of land under improved techniques or technologies as a result of USDA assistance - Disease management 5,662 NA 9,293 NA - Number of hectares of land under improved techniques or technologies as a result of USDA assistance - Soil-related fertility and conservation 6,392 NA 10,491 NA - Number of hectares of land under improved techniques or technologies as a result of USDA assistance – Irrigation 1,837 NA 3,015 NA - Number of hectares of land under improved techniques or technologies as a result of USDA assistance - Water management (non-irrigation based) 1,993 NA 3,271 NA - Number of hectares of land under improved techniques or technologies as a result of USDA assistance -Climate mitigation or adaptation 1,354 NA 2,222 NA - Number of hectares of land under improved techniques or technologies as a result of USDA assistance – Other 915 NA 1,502 NA - Number of hectares of land under improved techniques or technologies as a result of USDA assistance -Total w/one or more improved techniques or technologies 6,581 NA 10,801 NA - MSIKA Final Evaluation Page 85 kadale@africa-online.net Performance Indicator Impact Baseline Value Year 4 target (NEW) Y4 TOTAL LOP TARGET (NEW) LOP ACTUAL % Achievement (Y1-Y4) COMMENTS 7 Number of individuals who have applied new techniques or technologies as result of USDA assistance 0 31,680 30,778 31,680 30,778 97% The number of individuals applying new T&T was calculated based on a population of 30,922 farming beneficiaries who have been trained in ag T&T as of 31st May 2020. A very high 99.5% of the sample was applying the practices. Therefore, if the sample frame had included all 39,332 farmers trained it would have resulted in hitting the LOP target. Further training was disrupted by COVID, but subsequent trainings to the finalizing of the sample frame strongly suggests the LOP target would have been exceeded. Number of individuals who have applied new techniques or technologies as result of USDA assistance- Male 12,643 13,542 12,643 13,542 107% Number of individuals who have applied new techniques or technologies as result of USDA assistance- Female 19,037 17,236 19,037 17,236 91% Number of individuals who have applied new techniques or technologies as result of USDA assistance- New 25,188 23,638 25,188 30,778 122% Number of individuals who have applied new techniques or technologies as result of USDA assistance- Continuing 6,492 7,140 6,492 0 0% Number of individuals who have applied new techniques or technologies as result of USDA assistance - Crop Genetics 30,990 24,703 30,990 24,703 80% Number of individuals who have applied new techniques or technologies as result of USDA assistance - Pest Management 28,727 30,173 28,727 30,173 105% Number of individuals who have applied new techniques or technologies as result of USDA assistance - Disease management 31,335 30,404 31,335 30,404 97% Number of individuals who have applied new techniques or technologies as result of USDA assistance - Soil-related fertility and conservation 31,680 29,886 31,680 29,886 94% Number of individuals who have applied new techniques or technologies as result of USDA assistance - Irrigation 31,384 25,279 31,384 25,279 81% MSIKA Final Evaluation Page 86 kadale@africa-online.net Performance Indicator Impact Baseline Value Year 4 target (NEW) Y4 TOTAL LOP TARGET (NEW) LOP ACTUAL % Achievement (Y1-Y4) COMMENTS Number of individuals who have applied new techniques or technologies as result of USDA assistance - Water management (non-irrigation based) 31,384 27,726 31,384 27,726 88% Number of individuals who have applied new techniques or technologies as result of USDA assistance -Climate mitigation or adaptation 31,038 21,277 31,038 21,277 69% Number of individuals who have applied new techniques or technologies as result of USDA assistance - Other 31,481 25,999 31,481 25,999 83% Number of individuals who have applied new techniques or technologies as result of USDA assistance -Total w/one or more improved techniques or technologies 31,680 30,778 31,680 30,778 97% 8 Number of individuals who have applied improved farm management practices (i.e. governance, administration, or financial management) as a result of USDA assistance 0 26,320 30,196 26,320 30,196 115% The number of individuals who applied improved farm management practices is calculated from the sample of 979 individuals (sample frame of 30,922 farmers). Overall, a very high 97.7% of individuals applied improved farm management practices. The number of individual applying is 30,196, thereby exceeding the target (114.7%). Number of individuals who have applied improved farm management practices (i.e. governance, administration, or financial management) as a result of USDA assistance- Male 14,213 13,286 14,213 13,286 93% Number of individuals who have applied improved farm management practices (i.e. governance, administration, or financial management) as a result of USDA assistance- Female 12,107 16,910 12,107 16,910 140% MSIKA Final Evaluation Page 87 kadale@africa-online.net Performance Indicator Impact Baseline Value Year 4 target (NEW) Y4 TOTAL LOP TARGET (NEW) LOP ACTUAL % Achievement (Y1-Y4) COMMENTS Number of individuals who have applied improved farm management practices (i.e. governance, administration, or financial management) as a result of USDA assistance- Producers 26,320 30,196 26,320 30,196 115% Number of individuals who have applied improved farm management practices (i.e. governance, administration, or financial management) as a result of USDA assistance- People in Firms 0 0 0 0 - Number of individuals who have applied improved farm management practices (i.e. governance, administration, or financial management) as a result of USDA assistance- People in Government 0 0 0 0 - Number of individuals who have applied improved farm management practices (i.e. governance, administration, or financial management) as a result of USDA assistance- People in Civil Society 0 0 0 0 - 9 Number of agro-dealers and village based input agents operating in target districts as a result of USDA assistance 0 77 80 77 80 104% MSIKA has worked with 80 input suppliers. 3 additional input suppliers were added to 77 existing input suppliers in this reporting period. Input suppliers have played a pivotal role by increasing access of inputs in the MSIKA districts. The indicator counts input suppliers that have been supported with MSIKA and continue operating in the MSIKA districts. Number of agro-dealers and village based input agents operating in target districts as a result of USDA assistance- Male 67 70 67 70 104% Number of agro-dealers and village based input agents operating in target districts as a result of USDA assistance- Female 10 10 10 10 100% MSIKA Final Evaluation Page 88 kadale@africa-online.net Performance Indicator Impact Baseline Value Year 4 target (NEW) Y4 TOTAL LOP TARGET (NEW) LOP ACTUAL % Achievement (Y1-Y4) COMMENTS 10 Number of irrigation schemes facilitated 0 0 0 45 45 100% No additional irrigation schemes were supported or facilitated in this reporting period. MSIKA facilitated or supported 45 irrigation schemes over the LOP, this is in line with the LOP target of 45. 11 Number of individuals receiving financial services as a result of USDA assistance 0 5,000 5,612 9,984 13,922 139% 2,876 individuals received financial services in the April-Sept reporting period, bringing the year 4 total to 5,612, exceeding the target. Over the 4 years of the project, 13,922 unique individuals received financial services against a target of 9,984. Most of these individuals attended training and mentorship on financial readiness that lead them to access financing. Overall, 10,453 loans have been administered to the MSIKA beneficiaries, through MFI, VSL, and SPV. Number of individuals receiving financial services as a result of USDA assistance- Male 2,700 2,520 5,391 6,175 115% Number of individuals receiving financial services as a result of USDA assistance- Female 2,300 3,092 4,593 7,747 169% 12 Number of loans disbursed as a result of USDA assistance 0 4,108 5,074 9,561 10,377 109% 2,375 loans have been facilitated through MSIKA access to finance activities in this semi-annual reporting period, contributing to a total of 5,074 loans, above a target of 4,108 in year 4, representing 125% achievement. Cumulatively, MSIKA has disbursed 10,377 loans in the LOP exceeding the LOP target (109%). MSIKA beneficiaries accessed the loans from VSL (5,444), MFI (4,909) and SPV (24). Adjustments - 50 loans were double counted in the previous semi-annual periods, thus subtracted from what was reported in the current period. 13 Value of loans provided as a result of USDA assistance 0 $ 333,593 $ 413,100 $ 1,263,431 $ 699,879 55% $137,170 value of loans have been disbursed in the period April to Sept 2020. The loans are from MFI ($129,495) and VSL ($7,675. The April-Sept loans contributes to the total value $413,100 in Yr 4, exceeding the Yr 4 target ($333,593) by 128.8%. A cumulative total value of $699,879 in loans have been disbursed in the LOP, representing 56% achievement against the $1,263,431 target. The loans are contributed by MFI ($431,943), SPV ($193,343) and VSL ($74,592). The loans have benefited processors and individual farmers. The targets that in the previous reporting periods anticipated that MSIKA beneficiaries would be getting bigger loans from the SPV. SPV loans are pegged at the collateral that the beneficiaries have and most processors and farmers that Value of loans provided as a result of USDA assistance- Female $ 118,462 $ 154,639 $ 379,029 $ 273,504 72% Value of loans provided as a result of USDA assistance- Male $ 177,693 $ 175,667 $ 568,544 $ 233,032 41% MSIKA Final Evaluation Page 89 kadale@africa-online.net Performance Indicator Impact Baseline Value Year 4 target (NEW) Y4 TOTAL LOP TARGET (NEW) LOP ACTUAL % Achievement (Y1-Y4) COMMENTS Value of loans provided as a result of USDA assistance- Joint $ 98,718 - $ 315,858 $ 98,785 31% applied have no/limited high value assets, hence low achievement in the LOP. There have been more loans disbursed through MFI and VSL, but these are usually in smaller amounts. $94,558 includes loans not qualifying as disaggregates for Male, Female and Joint, as these are loans distributed to processors and FBOs, as well as including adjusting amounts. Therefore, these have bene put under not applicable disaggregates. The value reported this period reflect the adjustments made in the removal of the 50 duplicate loans described in Indicator 12. Value of loans provided as a result of USDA assistance- Not Applicable 82,794 $ 94,558 - 14 Percentage of producers who can recite five or more improved agricultural techniques and technologies 81% 66% NA 66% NA - After V37 consultation with USDA, it was decided to exclude questions specific to measuring this indicator to shorten interview time. However, it is worth noting that based on the very high achievement on the application of new practices in indicator 7, the level of knowledge of at least one practice would be at least be this level of 97.2%. Ultimately, the aim is application, with knowledge as a stepping stone towards application. Also, the achievement by Yr3 (midterm) was 56.7% overall, which suggests that with another year's training that the LOP target 66% would likely have been met if measured. Percent of producers who can cite five or more improved agricultural techniques and technologies￾Female 76% 64% NA 64% NA - Percent of producers who can cite five or more improved agricultural techniques and technologies- Male 85% 68% NA 68% NA - 15 Percent of agricultural producers in target region who can identify key characteristics of a well-managed farm 65% 90% NA 90% NA - After V37 consultation with USDA, it was agreed to exclude questions specific to measuring this indicator to shorten interview time. However, based on the very high level of application in indicator 8 of farm management practices at 97.7%, the level of knowledge of farm management practices is at least this high. Ultimately, the aim is application, with knowledge as a stepping stone towards application. It is also noted that the mid-term, the achievement level was 100%, so it is very likely that the LOP target of 90% would have been achieved if measured. Percent of agricultural producers in target region who can identify key characteristics of a well-managed farm- Female 63% 88% NA 88% NA - Percent of agricultural producers in target region who can identify key characteristics of a well-managed farm- Male 67% 92% NA 92% NA - MSIKA Final Evaluation Page 90 kadale@africa-online.net Performance Indicator Impact Baseline Value Year 4 target (NEW) Y4 TOTAL LOP TARGET (NEW) LOP ACTUAL % Achievement (Y1-Y4) COMMENTS 16 Number of on-going agricultural research initiatives by DARTS supported by USDA assistance 0 NA 0 - Removed from list of indicators in modification 4, signed September 17, 2020. 17 Number of policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance 0 2 2 2 2 100% MSIKA has facilitated development of two policies, the horticultural policy and product standards for fruits and vegetables. The policies are near completion. MSIKA is currently facilitating final review meetings of the Draft Horticultural Policy. Final Regional validation workshops are expected to be conducted in three regions of Malawi to ensure buy-in to the policy by the wider stakeholder community, before it is passed to the Office of President and Cabinet for approval. The 10 fruit & Vegetable Standards with MSIKA assisted in developing have been approved by the Malawi Bureau of Standards (MBS) Board. The list of Standards have been submitted to the Ministry of Industry, Trade and Tourism who will prepare "minutes" requesting the Ministry of Justice to approve the Gazetting of these 10 Standards. Number of policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance- Stage 1 0 2 1 2 200% Number of policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance- Stage 2 0 2 2 2 100% Number of policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance- Stage 3 0 2 2 2 100% Number of policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance- Stage 4 2 0 2 0 0% Number of policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance- Stage 5 0 0 0 0 - 18 Percent of producers who have access to current market information through their cooperatives or producer associations. 13% 48% 27.6% 48% 28% 57% The indicator is calculated based on the sample of 979 farmers (sample frame: 30,922). MSIKA has improved progressively from the impact baseline (13%) through to Yr4 (28%). MSIKA Final Evaluation Page 91 kadale@africa-online.net Performance Indicator Impact Baseline Value Year 4 target (NEW) Y4 TOTAL LOP TARGET (NEW) LOP ACTUAL % Achievement (Y1-Y4) COMMENTS Percent of producers who have access to current market information through their cooperatives or producer associations- Female 11% 46% 27.4% 46% 27% 60% MSIKA undertook activities to accelerate access to marketing information through the FBOs e.g. intensified marketing activities, continued running of radio programs where farmers share marketing information, built aggregation centers, and facilitated sales agreements. The pandemic resulted in government policies that restricted FBO meetings where FBOs share marketing information, and also restricted access to markets. Percent of producers who have access to current market information through their cooperatives or producer associations- Male 15% 50% 27.7% 50% 28% 55% 19 Number of private enterprises, producer organizations, water users associations, women’s groups, trade and business associations, and community-based organizations (CBOs) that applied improved techniques and technologies as result of USDA assistance 0 129 226 129 226 175% MSIKA assessed performance of FBOs and processors in this reporting period. A sample of 49 FBOs was randomly drawn from 217 total number of FBOs. 10 processors were purposively selected from the total processors reached. The results reveal that 100% of the sampled FBOs and 9/10 processors are applying one or more techniques and technologies. The total number comes to at least 226 in this reporting period, surpassing Yr4 and LOP targets of 129. Number of private enterprises, producer organizations, water users associations, women’s groups, trade and business associations, and community-based organizations (CBOs) that applied improved techniques and technologies as result of USDA assistance- New 7 107 129 226 175% Number of private enterprises, producer organizations, water users associations, women’s groups, trade and business associations, and community-based organizations (CBOs) that applied improved techniques and technologies as result of USDA assistance- Continuing 122 119 NA - MSIKA Final Evaluation Page 92 kadale@africa-online.net Performance Indicator Impact Baseline Value Year 4 target (NEW) Y4 TOTAL LOP TARGET (NEW) LOP ACTUAL % Achievement (Y1-Y4) COMMENTS Number of private enterprises, producer organizations, water users associations, women’s groups, trade and business associations, and community-based organizations (CBOs) that applied improved techniques and technologies as result of USDA assistance- Private enterprise 7 9 7 9 129% Number of private enterprises, producer organizations, water users associations, women’s groups, trade and business associations, and community-based organizations (CBOs) that applied improved techniques and technologies as result of USDA assistance- Producer organization 122 217 122 217 178% Number of private enterprises, producer organizations, water users associations, women’s groups, trade and business associations, and community-based organizations (CBOs) that applied improved techniques and technologies as result of USDA assistance- Women’s group 0 NA 0 NA - Number of private enterprises, producer organizations, water users associations, women’s groups, trade and business associations, and community-based organizations (CBOs) that applied improved techniques and technologies as result of USDA assistance- Trade and business association 0 NA 0 NA - MSIKA Final Evaluation Page 93 kadale@africa-online.net Performance Indicator Impact Baseline Value Year 4 target (NEW) Y4 TOTAL LOP TARGET (NEW) LOP ACTUAL % Achievement (Y1-Y4) COMMENTS Number of private enterprises, producer organizations, water users associations, women’s groups, trade and business associations, and community-based organizations (CBOs) that applied improved techniques and technologies as result of USDA assistance- Water users association 0 NA 0 NA - Number of private enterprises, producer organizations, water users associations, women’s groups, trade and business associations, and community-based organizations (CBOs) that applied improved techniques and technologies as result of USDA assistance- Community-based organization 0 NA 0 NA - Number of private enterprises, producer organizations, water users associations, women’s groups, trade and business associations, and community-based organizations (CBOs) that applied improved techniques and technologies as result of USDA assistance- Other 0 NA 0 NA - 20 Number of public-private partnerships formed as a result of USDA assistance 0 0 8 181 192 106% MSIKA formed 3 partnerships in the period April to Sept 2020, Bayer Crop Science, Newton Investments and Organic Fertilizer Production Company. These partnerships have been made to enhance showcasing techniques and technologies in MSIKA farmer field schools and Yankho plots, hence fostering agricultural productivity of MSIKA farmers. Cumulatively, the project has made 192 partnerships above the LOP target of 181. More partnerships entail more investment in agriculture which ultimately contributes to agricultural sector growth. Number of public-private partnerships formed as a result of USDA assistance- Agricultural Production 0 5 12 17 142% Number of public-private partnerships formed as a result of USDA assistance- Agricultural post￾harvest transformation 0 2 9 11 122% MSIKA Final Evaluation Page 94 kadale@africa-online.net Performance Indicator Impact Baseline Value Year 4 target (NEW) Y4 TOTAL LOP TARGET (NEW) LOP ACTUAL % Achievement (Y1-Y4) COMMENTS Number of public-private partnerships formed as a result of USDA assistance- Nutrition 0 0 0 - - Number of public-private partnerships formed as a result of USDA assistance- Multi-focus 0 0 155 158 102% Number of public-private partnerships formed as a result of USDA assistance- Other 0 1 5 6 120% 21 Volume of commodities (metric tons) sold by project beneficiaries 7.3 per beneficiary 24,628 1,261 86,330 62,963 73% Sales were significantly down in the latest reporting period. FBOs sold 332 MT in April-Sept 2020, so the total sales volume in Yr 4 comes to 1,261 MT, representing 5% of Yr4 target. Cumulative sales were 62,963 MT against the LOP target of 86,330 MT (73%). Sales data for processors was not available in the reporting period, as project staff could not travel to conduct monitoring activities. Processor sales are reported in the KIIs to have also been affected by the pandemic restrictions. The government enforced COVID-19 measures, which led to closure of schools, hotels, and other private and government bodies. They also restricted meetings and conferences nationwide resulting in lost sales for fruit and vegetable farmers. Farmers had to rely on local markets. 22 Number of jobs attributed to USDA assistance 0 1,657 2,067 1,657 2,067 125% The Yr 4 and the LOP targets were surpassed other than for females. The result was calculated in the Final evaluation using data from the beneficiary producers based on the sample of 979 farmers (sample frame of 30,922 farmers trained in basic ag production as of 31st May 2020). The total for Yr4 does not include jobs created by processors and by FBOs, and other businesses e.g. input suppliers, as it has not been possible to collect this data from all these parties due to the COVID restrictions. Number of jobs attributed to USDA assistance- Female 762 507 762 507 67% Number of jobs attributed to USDA assistance- Male 895 1,560 895 1,560 174% 23 Value of sales by project beneficiaries $2122.36 per beneficiary $ 4,900,310 $ 1,367,776 $ 21,972,909 $ 18,440,375.71 84% FBO sales in April-Sept period were $134,740 to the total value of sales in year 4, which is 28% of the target ($4,900,310). As noted on volumes, sales were particularly affected by COVID restrictions. No data was available for processors in this period due to limitations on collecting data. MSIKA Final Evaluation Page 95 kadale@africa-online.net Performance Indicator Impact Baseline Value Year 4 target (NEW) Y4 TOTAL LOP TARGET (NEW) LOP ACTUAL % Achievement (Y1-Y4) COMMENTS 24 Value of new public or private investment leveraged by USDA assistance $0.00 $ 333,593 $ 1,686,530 $ 1,967,799 $ 3,320,736.32 169% MSIKA beneficiaries leveraged the value of $1,413,173 in investments in the April-Sept period. This was mainly contributed to by loans ($413,100) and Processors ($1,000,070). The total value for year 4 is $1,686,530, representing 506% achievement against a target of $333,593. The value of $3,320,736 in investments has been made over MSIKA LOP against the LOP target of $1,967,799, representing 169% achievement. There has been an increase in investments in this reporting period particularly farmers as they have intensified irrigation farming in the MSIKA value chains. Value of new public or private investment leveraged by USDA assistance- Public $ - $ - $ - $0.00 - Value of new public or private investment leveraged by USDA assistance- Private $ 333,593 $ 1,686,530 $ 1,967,799 $3,320,735.93 169% 25 Percent of registered processing firms in target sectors that obtain certification with Malawi Bureau of Standards related to product quality 0 30% 44% 30% 44% 147% No processing firm obtained certification in the April-Sept reporting period. Only essential government services have been operating in full due to COVID 19. It's assumed most processors have not been visited in this period. Despite this, the LOP target had already been achieved (147%). 26 Percent post-harvest losses for beneficiary producers 12% 14% 13.8% 14% 13.8% 101% MSIKA has surpassed the LOP target overall and across the disaggregates. The IMPACT Baseline (IB) values have been revised to account for just the four crops (Tomato, Onion, Potato and Mango) that were of focus in year 4, not the original 7 crops. Thus, the revised IB for this indicator is now 14.77%. In the final evaluation, the losses reported were 13.8%, meaning that PH losses have reduced marginally (from 14.77% to 13.8%) by EOP. Losses are very difficult to calculate, as farmers rarely weigh what they throw away. There are also losses at various stages (at the farm, transport to market and at the market), so it can be difficult to estimate/calculate what losses have occurred. At best, the figures from farmers are rough estimates. This implies that the baseline and measurement data should be taken with caution. Also, the target for this indicator (14%) was set at the beginning of the project when the focus was on 7 crops and not revised following the revisions to the IB or the change to 4 focus crops. Hence it is irrelevant given current estimates of the IB and the reported losses in the in the FE. Percent post-harvest losses for beneficiary producers - female 13% 16% 15.4% 16% 15.4% 104% Percent post-harvest losses for beneficiary producers - male 12% 13% 11.8% 13% 11.8% 110% 27 Total increase in installed storage capacity (dry or cold storage) as a result of USDA assistance 0 - 438,788 45,770 484,558 1059% MSIKA has very significantly surpassed the LOP target for this indicator across all the disaggregates. The volume in storage which was built or refurbished has been calculated based on a sample of 979 out of a population of 30,922 individuals. MSIKA Final Evaluation Page 96 kadale@africa-online.net Performance Indicator Impact Baseline Value Year 4 target (NEW) Y4 TOTAL LOP TARGET (NEW) LOP ACTUAL % Achievement (Y1-Y4) COMMENTS Total increase in installed storage capacity (dry or cold storage) as a result of USDA assistance- Dry - 438,788 45,770 484,558 1059% 23.3% farmers built new storage, while 13.78% refurbished their storage. The data for the calculation has been reviewed several times, as the amounts appear very high in comparison with previous data. The consultants applied some realistic maxima, but the data still appeared high. The consultant attributes the high data to farmers estimating over the phone, but the enumerators being unable to verify the numbers by getting farmer to pace out the size of the storage or indicate its height visually. While the data is probably overstating the change, the consultant's view is that the target is very likely to have been achieved even if there had been verification of the farmers estimated measurement of the storage. Total increase in installed storage capacity (dry or cold storage) as a result of USDA assistance- Cold - - - - - Total increase in installed storage capacity (dry or cold storage) as a result of USDA assistance￾Refurbished - 169,109 33,941 203,050 598% Total increase in installed storage capacity (dry or cold storage) as a result of USDA assistance- New 8,500 269,680 11,829 281,508 2380% 28 Number of national export promotion advertising campaigns as a result of USDA assistance 0 0 0 6 6 100% As planned, there was no national export promotion advertising in the period April to Sept 2020. Cumulatively the project has achieved 100% of 6 LOP target. 29 Number of sales agreements with FBOs 0 10 209 62 261 421% No sales agreements were reported in this April - Sept period, as COVID 19 forced closure of many institutions such as schools, hotels and lodges that are major buyers of fruit and vegetables, so farmers were relying on sales in local markets not under sales agreements. 30 Number of FBOs using improved financial management practices and systems as a result of USDA assistance 0 - Indicator was removed following modification of PMELP 31 Percentage of FBOs who are aware of international production and handling standards as a result of USDA assistance 0 71% NA 71% 78% 109.58% No data for this indicator was captured during the final evaluation, as it was not a priority in the limited interview time due to COVID-19. Based on the data available prior to that, the LOP target has been surpassed (110%). 32 Number of processor staff trained in quality standards 0 0 0 187 187 100% Trainings for processor staff were not planned in this April￾Sept period. The LOP actual is at 187, which is 100% the LOP target. Number of processor staff trained in quality standards- Male 0 0 97 97 100% Number of processor staff trained in quality standards- Female 0 0 90 90 100% MSIKA Final Evaluation Page 97 kadale@africa-online.net Performance Indicator Impact Baseline Value Year 4 target (NEW) Y4 TOTAL LOP TARGET (NEW) LOP ACTUAL % Achievement (Y1-Y4) COMMENTS 33 Number of FBOs trained in improved financial and organizational management 0 50 48 135 183 136% 38 FBOs have been trained in financial and organizational management in the April to Sept period, bringing the total for year 4 to 48 unique FBOs trained, representing 96% of year 4 target (50). A total of 183 FBOs have trained over the LOP (136% achievement). MSIKA Final Evaluation Page 98 kadale@africa-online.net Annex 2: MSIKA Results Frameworks MSIKA Final Evaluation Page 99 kadale@africa-online.net Annex 3: Terms of Reference/Scope of Work Malawi Strengthening inclusive Markets for Agriculture (MSIKA) Final Evaluation Modification: August 27, 2020 1. INTRODUCTION AND JUSTIFICATION This contract is to hire an external evaluation firm to conduct the final evaluation (FE) of the four-year MSIKA Food for Progress project, funded by United States Department of Agriculture (USKA) and implemented by Land O’Lakes Venture37 (Venture37). The project is implemented in Mchinji, Dedza, Ntcheu, Lilongwe and Mangochi Districts of Malawi from October 1, 2018 – December 31, 2020. The FE will examine the results of the project, and provide recommendations for the implementation of similar projects in the future. An externally conducted FE is required by the donor, USDA, in the project contract. This document describes the background of the project, the scope of work that the evaluation contractor will implement, and the timeline for conducting the scope of work. 2. BACKGROUND MSIKA is a four-year value chain development project that will reach 36,000 smallholder farmers, 217 farmer-based organizations (FBOs), and 18 processors in south central Malawi in the fruit and vegetable value chains, specifically tomato, potato, onion, chili, mango, citrus and guava. MSIKA is catalyzing increased value addition and income for value chain actors by facilitating improved processing, increased crop productivity, improved post-harvest handling (PHH) and storage, expanded market linkages between farmers and processors, more efficient domestic trade, and increased potential exports of processed products in the long term. MSIKA interventions and market linkages will target generating a total of $21,972,909 in sales by project participants across the four years and leveraging $1,967,799 in new public or private investment by the end of the project. MSIKA focuses on achieving the following objectives: • Increase agricultural productivity in the fruit and vegetable sector by increasing the availability of improved inputs, improving infrastructure to support on-farm production, facilitating access to finance, and training farmers on improved agricultural techniques and technologies, as well as farm management. • Expand trade of agricultural products in the fruit and vegetable sector by improving quality of postproduction agricultural products, training producers and processors on improved post￾production processes, facilitating improved linkages between buyers and sellers, improving market and trade infrastructure, and facilitating improved management of buyer/seller groups. Due to monetization issues, MSIKA had to shorten the project duration from five year to four years. During the fourth and final year, the project will focus on a subset of the participants (122 FBOs, 8000 farmers and 7 processors) and the tomato, potato, onion and mango value chains to maximize sustainability within the most promising values chains for commercialization. 2.1 Project Implementation This section provides a more detailed description of MSIKA activities. Training: Improved agricultural production techniques Venture37will provide tailored training, technicalresources, and cutting-edge research to fruit and vegetable farmersto improve technical and businessskills. Venture37 will build capacity in tandem with, and transfer knowledge to, community leaderssuch aslead farmers and extension agents fromtheMinistryofAgriculture, IrrigationandWaterDevelopment(MoAIWD). Venture37 will also build capacity through Farmer Field Business Schools(FFBS) and demonstration plots(Yankho PlotsTM) on topicssuch as orchard management, budding and grafting, nursery establishment, MSIKA Final Evaluation Page 100 kadale@africa-online.net pruning, and weeding. Venture37 will also ensure that FFBS and Yankho PlotsTM build core climate smart agronomic and business skills of target producers. In addition,Venture37 will engage private sectorpartnersto ensure increased utilization of irrigation technologies and increased accessto improved inputs. Venture37 will engage private sector partners to co-invest in Yankho PlotsTM and to establish for-profit nursery enterprises. Infrastructure: Post-harvest handling and storage Venture37will improve post-harvest handling and storage infrastructure by providing training and technical assistance,facilitatingmarketlinkages, andsupporting accessto finance to improve on-farm and off-farm post-harvest infrastructure and to increase the use of improved post-production processing and handling practices. Venture37 willfacilitate increased adoption of established standards, such as Global Good Agricultural Practice (Global GAP), by market actors, such as traders, processors, and producers. Venture37willfacilitate Farmer-BasedOrganizations(FBO) andprivate enterprisesto develop wholesale markets, and aggregation centers, and to utilize technology and practicessuch as optimalpacking technology, improved storage practices, and sorting, grading, and washing technology. Venture37 willprovide technical assistance to supportthese initiatives. Training: Post-harvest processing Venture37willimprovetheefficiencyandprofitabilityofvalue-addedpost-harvest manufacturing and promote the utilization of value-preserving practices and technologiesthroughoutthe fruit and vegetable value chains. Venture37 will provide targeted technical assistance to small, medium- and large-scale fruit and vegetable processors. Technical assistance will focus on processing line operational efficiency,improved packagingandlabelling,modernizationofphysicalequipment and plantfacilities, and improved business and financialmanagement practices. Venture37 willtrain fruit and vegetable processors on improved product quality standards and facilitate Malawi Bureau of Standards (MBS) certification. Venture37 will facilitate FBOaccessto improved post-harvest techniquessuch as use of reusable crates, sorting and grading, and washing. Venture37 will also facilitate improved phytosanitary practices by agricultural enterprises and FBOs engaged in small￾scale processing of fruits and vegetables into productssuch as jams, juices, sauces and paste making. Capacity Building: Producer groups and cooperatives Venture37 will apply cooperative developmenttools, including Venture37’s AgPrO products, to improve farm management and to increase the capacity of FBOsin agricultural production, post￾harvest handling, and processing. Venture37 will link FBOsto business development service (BDS) providers and finance providers, andwill collaboratewith national-levelproducer associations and district-level FBOs(such asthe Mchinji Horticulture Association) to strengthen the delivery of member services and facilitate the transformation of FBOsinto agricultural enterprises. Market Access: Facilitate buyer-seller relationships Venture37will improvemarket accessby facilitatingbuyer-sellerlinkages, increasing access to marketinformation, and strengthening the capacity of key organizationsin the trade sector. Specifically,Venture37willfacilitatebuyer-sellernetworkingevents,linkagestotransporters the establishment and strengthening of equitable outgrowerschemes between producers and processors (e.g., Malawi Mangoes, RAB Processors), stronger linkages to retail supermarkets forstandards and specification trainings and procurement, promotion of village-based input sales agents, and targeted support for women- and youth-owned small and medium enterprises (SMEs) engaged in food processing. Financial Services: Facilitate agricultural lending Venture37 will build the capacity of a local financial institution to conduct risk assessments and develop financial productsfor value addition in the fruit and vegetable sectors,such as working capital and trade finance for processing, aggregation, irrigation, export, and transportation. MSIKA Final Evaluation Page 101 kadale@africa-online.net Venture37 willfacilitateincreasedlendingforclimatesmartirrigation;workwithfinancialinstitutions todevelop loan products appropriateforhorticulturevalue-adding activities;facilitate the developmentofinput credit relationships between processors and producers or FBOs; improve the financial literacy and credit- worthiness of FBOs; and facilitate the development of capital investment plans by fruit and vegetable processorsfor plant modernization and expansion, including improvement of business and financial management systems. Financial Services: Provide SME finance Venture37will stimulateincreasedagriculturallendingbyestablishing a Special Purpose Vehicle (SPV). The SPVwillmake loans to financial institutions that will on-lend those fundsto enterprises, farmers and farmer organizations. The SPVmay catalyze further investmentin value-adding activities and trade,such asloansfromtheOverseas Private Investment Corporation (OPIC). Venture37 will ensure that loans originating from the SPV, priorto the award’s date of completion, will be used for purposes such as increasing access to working capital for farmers and SMEsfor input purchases, transport forspace arbitrage of fruits and vegetables, market and trade infrastructure, processing equipment, and equipment upgrades. After the award’s date of completion, Venture37will ensure thatloans originating from the SPV will be used to expand agricultural production and trade in African Least Developed Countries, as defined by the World Bank. Forthe life of the SPV, Venture37 will use principal and interest repaid to the SPV for continued lendingandinvestment. Government Capacity Building: Improve Enabling Environment Venture37will buildthe capacityof key governmentresearchand extensionagencies, including MoAIWDand theDepartment ofAgricultural ExtensionServices(DAES). Venture37 will work closely with the Lilongwe University of Agriculture and Natural Resources (LUANAR) to conduct research on soil fertility and other value chain specific trials. Venture37 will build the capacity of processors, traders, national associations, and FBOs to advocate for improved regulations and policies. Venture37 willwork with the Malawi Bureau of Standards(MBS)to jointly establish and disseminate international quality standards. Venture37 will conduct a gap analysis and recommend concrete improvementsto bring MBS’s standards, certification processes, and quality audit processes up to international levels. Venture37 will support multi-stakeholder and public- private dialogues on issues of key importance to the agriculture production, processing, and trade sectors. 3. EVALUATION DESIGN Objectives of the Final Evaluation The objectives of the final evaluation are as follows: 8. Assess the relevance of the project strategy and approach to project participants; 9. Measure progress the project has made toward key results, including effectiveness and efficiency of interventions in achieving established targets (see annex 2 for key outcome indicators); 10. Identify enablers and constraints to progress (both internal and external factors) that have supported or limited success of the project; 11. Assess the sustainability of project outcomes, and the effectiveness of sustainability efforts that were undertaken by the project; 12. Assess operational aspects of the project, such as project management and monitoring and evaluation; 13. Document lessons learned, challenges and unanticipated effects; 14. Provide recommendations for best practices that were realized or areas of improvement for future USAID and Venture37 program in similar areas and value chains. Evaluation Questions MSIKA Final Evaluation Page 102 kadale@africa-online.net The final evaluation will seek to answer the following key questions within the standard evaluation criteria: Criteria Questions Relevance • Did the results framework, assumptions, program design and project activities meet the needs of the participants and local conditions in the five target districts of Malawi? • How aligned was the program strategy and activities with Government of Malawi (GoM) strategies and with USDA and USG development goals, objectives and strategies? Effectiveness • What internal and external factors have influenced the ability of the project to meet expected results and revised targets? • To what extent have the revised program targets and outcomes been achieved by the end of the project? • How have the project participants and their businesses been impacted by the COVID-19 pandemic? Impact • What impacts are the project activities having on the program participants, both positive and negative, especially in relation to the expected results and strategic objectives? o How has project training and access to finance affected the uptake of improved agricultural techniques, farm management practices, PHH, and use of improved infrastructure for farmers? o How has the use of improved agricultural techniques and technologies and market linkages affected crop yield and farmer sales? o How has improved PHH practices affected post-production losses and improved the quality of the product? o How have producer group trainings affected the capacity of the groups? o How has the project training and access to finance affected uptake of improved processing techniques, quality standards, and improved infrastructure for processors? • What key successes should be replicated, or key improvements should be made in future programming? Efficiency • Were the resources and activities provided by the program carried out in a timely manner and with effective use of resources? • How well has the project been managed and M&E data used to make programmatic decisions? Sustainability • Which project activities and benefits are likely to be sustained or not, why? How? • What evidence is there that the interventions are likely to scale up beyond the project life Scope of Work MSIKA Final Evaluation Page 103 kadale@africa-online.net The contractor will conduct a mixed methods evaluation with a non-experimental design, incorporating both primary quantitative and qualitative data collection, supplemented by project monitoring data. Due to COVID-19 pandemic, the final evaluation will collect information through phone surveys of registered participants, including farmers, FBOs and processors, to compare key metrics to the values at the impact baseline. The below text details how data will be collected from the key program participants and stakeholders to answer the key evaluation questions. A summary of the data collection methods can be found in the table below. The current sample frame is 217 FBOs, 18 processors, and about 36,000 farmers, whereas the participantsthat will receive focus (continued capacity building and support) in the fourth year are 50 FBOs, 8,000 farmers and 7 processors. The final sample frame will be determined in collaboration with the contractor. Stakeholder Data Collection Method Participant Farmers Household Survey Key Informant Interview Farmer-based Organizations Financial Data Key Informant Interview Processors Financial Data Key Informant Interview Other Key Stakeholders Key Informant Interview Farmers: The contractor will collect quantitative data from participant farmers in each of the four targeted value chains (tomato, potato, onion, mango) to assess their agricultural and post-production practices, crop yields, post-harvest losses, crop sales and use of finance. For each target value chain, the final evaluation will collect information from a statistically relevant number of households that participate in that value chain at the 90 percent34 confidence level, as possible. Participant respondents will be selected randomly from the farmer participant list, the sample size proportionally spread across the target districts. With the change to phone interviews, each farmer will only be asked about one crop that they grow. This crop will be selected randomly until a quota is reached and then that crop will be taken out of the random selection for future farmers. The contractor and project will agree at inception how to ensure representation of both the farmers that did and did not receive support in year 4. The contractor will also collect qualitative data from farmers in the targeted value chains through key informant interviews with farmers, in each of the districts. The KIIs will provide context to the quantitative data to describe why farmers are or are not changing their agricultural practices, successes and challenges in growing and selling their crops, participation in FBOs and feedback on how the project can be improved. Farmer-based Organizations: The contractor will utilize monitoring data collected from FBOs. This data will include group agreements with input providers, processors and retailers, financial information on group sales and use of financing. The contractor will also collect quantitative and qualitative information from a purposive sample of project-supported FBOs through remote structured key informant interviews to verify the monitoring data, understand their relationships with input providers, processors, and retailers; the functioning of the group and value they provide to their members; their successes and challenges of working as a group, and solicit feedback on how the project could improve its activities. The evaluation will sample FBOs across the different districts and 34 The overall sample size will exceed the 95% confidence level for all horticulture, but meet the 90% confidence level for each individual value chain. MSIKA Final Evaluation Page 104 kadale@africa-online.net value chains that are 1) under performing; 2) of average performance; and 3) performing well, according to their monitoring data. Samples will be taken both from those that are reached in the fourth year and those that have not been reached in the fourth year. Processors: The contractor will utilize monitoring data collected from processors, including quantity of production and value of sales. The contractor will also collect quantitative and qualitative information through remote structured key informant interviews with 12 processors, or those willing to participate within the project timeframe, to understand changes they have made in their processing practices, ways they are engaging with the FBOs, use of financial resources, successes and challenges in their current functioning, and suggestions for project improvement. Other Key Stakeholders: The evaluation will conduct remote key informant interviews with other key project stakeholders, including input supply distributers, trader/wholesalers, participating government staff, local leaders and program staff to understand how they have participated in the project, challenges and successes, and suggestions for improvement. The specific tasks that the contractor will carry out are detailed below: Meeting with USDA: Conduct an introductory phone call with USDA prior to conducting fieldwork for the FE. If requested by USDA, conduct a phone call with USDA after the draft report is completed to share key findings. Review of Documents: Undertake review of the MSIKA program documents and other relevant documents that are available at the time, including, but not limited to, the following: • Project agreement with USDA, including the MSIKA scope of work and all amendments • MSIKA Performance Management Plan • MSIKA MEL Plan • Semi-Annual farmer performance survey data • Monitoring data, including list of farmers, FBOs, processors, and other key stakeholders • Semi-annual reports submitted by Venture37 to USDA; • Initial and impact baseline and midterm report, data collection tools and data; • Any other program documents which will enable the evaluator to get acquainted with the project progress including value chain and financial studies; • Relevant Government of Malawi reports and documents for background information and establishing the socio-economic and political context in which MSIKA occurred. Refinement of methodology and data collection tools: The evaluator, in close collaboration with the Venture37 Global Monitoring, Evaluation, and Learning team, will do the following: • Develop a finalized methodology, including a sampling frame, sampling technique and sample sizes for both quantitative and qualitative surveys. The sampling frame will use a minimum confidence level of 90 percent for each value chain, as possible. Surveys should be comparable to the impact baseline to ensure comparability of data over time, but additional questions could be added, or tools created at final evaluation to ensure all evaluation questions, described above, are answered. • Based upon a reading of the program documents, propose any additional topics or issues for analysis prior to conducting the final evaluation. • Further refine the methodology and tools based on the COVID-19 pandemic to transition interviews to remote phone-interviews. Field Data Collection • Plan and coordinate the necessary logistics to collect the data in accordance with the selected methodology. • Pre-test, edit, translate, finalize and reproduce the survey instruments. • Train and orient enumerators and data collection team. MSIKA Final Evaluation Page 105 kadale@africa-online.net • Carry out the phone interviews using own office space, including: o Household survey: Approximately 1,092 phone surveys with farmers; o Key informant interviews with farmers: Approximately 12 KIIs o Interviews with other Key Informants (farmer groups, input services providers, district agriculture officers and Venture37 program staff): Approximately 52 KIIs. Data entry, analysis and reporting • Enter, clean, synthesize, analyze, and interpret data from both the quantitative surveys and the qualitative protocols using approved statistical packages. The quantitative data from farmers will be analyzed using pre-post comparison of key indicators between the impact baseline and the final evaluation. The quantitative data from FBOs and processors will be compared over time. Qualitative data will be transcribed and analyzed by themes. • Prepare a draft evaluation report addressing the objectives and questions of this evaluation outlined in this TOR and recommendations on the overall Venture37/FFPr MSIKA project for potential similar future project for review by Venture37 staff and stakeholders. • Develop a PowerPoint presentation of evaluation findings, present and submit to Venture37 and stakeholders. • Prepare a final evaluation report that includes revisions based on feedback on the draft report and presentation. 4. TIMEFRAME The contractor will carry out the work according to the below timeline, starting on June 15th, 2020. Final Evaluation Activity Responsibility Due Date Review of relevant documents to prepare for inception meeting Evaluator June 15th – 19th, 2019 Inception meeting with Venture37 to discuss protocol, methodology, sampling, tools and timeline Evaluator and Venture37 June 22nd, 2020 Develop an inception report and update data collection tools Evaluator June 23rd – July 8th, 2020 Inception report and tools due to Venture37 Evaluator July 10th, 2020 Venture37 reviews report and tools and provides feedback, comments and suggestions to evaluator Venture37 July 13th – 17th, 2020 Prepare for field work, finalize tools based on Venture37 feedback, test, refine and code instruments Evaluator July 13th – 17th, 2020 Adjust methodology and tools for remote data collection due to COVID-19 Evaluator July 18th – August 21st Enumerator training and data collection Evaluator August 24th – September 11th, 2020 Data cleaning, analysis and report writing Evaluator – September 14th – October 20th , 2020 Draft Outline of final evaluation report for review Evaluator September 25th, 2020 Draft final evaluation report submitted to Venture37 Evaluator October 20th, 2020 Venture37 reviews draft final report and provides evaluator with comments and suggestions for revisions Venture37 October 21st – 27th, 2020 Presentation of evaluation findings to Venture37 Evaluator Week of October 26th MSIKA Final Evaluation Page 106 kadale@africa-online.net Final Evaluation Activity Responsibility Due Date Finalize report based on Venture37 feedback and prepare all deliverables Evaluator October 28th – November 3rd, 2020 All Final Deliverables Due (Final evaluation report, clean data, and PPT presentation) Evaluator November 3rd, 2020 5. REQUIRED DELIVERABLES The deliverables under this assignment are listed in the table below. Deliverable Due Date Description Inception Report July 10th, 2020 Report should describe the following: i- Understanding of the project based on project documents and literature review i- Finalized methodology including detailed sampling plan and field procedures i- Quality control measures v- Communication protocol v- Finalized timeline (activities, responsible party, outputs, and timing) i- Draft Data collection tools Outline of evaluation report September 25th , 2020 Submit a high-level outline of the report, including proposed sections for results and findings. Draft final evaluation report October 20th, 2020 The report should be submitted in English addressing all the evaluation objectives and questions listed in the scope of work PowerPoint Presentation November 3rd , 2020 Presentation should include an abbreviated list of evaluation findings that can be presented to relevant internal and external stakeholders Non-technical summary of evaluation findings November 3rd 2020 A short (2-4 pages) non-technical summary of the evaluation findings, easily understood by non-experts, and include graphics such as infographics, charts and/or graphs. Can be the same as the executive summary Final version of the final evaluation report November 3rd , 2020 Electronic copy of the final evaluation report should be submitted in English in both Microsoft- Word and PDF version. Report should include the following sections: a. Acknowledgements b. List of Acronyms and abbreviations c. Table of Contents d. Executive Summary (no longer than four pages) e. Background (Program description and purpose of baseline) f. Methodology and Implementation g. Results and Findings h. Recommendations i. Annex: Table of key program indicators with updated values in comparison to baseline values for all required disaggregates in the PMP j. Annex: Scope of Work for the evaluation k. Annex: Inception Report for the evaluation l. Annex: Survey Instruments: questionnaire(s), survey(s), interview protocol(s), focus group discussion protocol(s) MSIKA Final Evaluation Page 107 kadale@africa-online.net Deliverable Due Date Description Final Data Collection Tools November 3rd , 2020 Electronic copies of all clean and final English-version of data collection tools Final Cleaned Data November 3rd , 2020 Clean and final English versions of: - quantitative data sets in Microsoft-Excel and any other utilized format (SPSS, STATA, etc) - qualitative summaries, field and interview notes, complete list of key informant interviews and FGDs in Microsoft-Word document MSIKA Final Evaluation Page 108 kadale@africa-online.net Annex 4: Instruments Household Survey Malawi Strengthening Inclusive Markets for Agriculture (MSIKA) Final Evaluation 2020 Kadale Consultants Ltd MSIKA Final Evaluation Page 109 kadale@africa-online.net 2. ID Module Q# Question Response Logic* ID1 Enumerator Name ID2 Date of Interview ID4 District Dedza Lilongwe Mangochi Mchinji Ntcheu ID5 EPA ID8 Village ID6 TA ID7 GVH 3. Introduction & Consent Read out and seek consent: “Hello, my name is [NAME] and I work for Kadale Consultants. As I already explained previously, Kadale is conducting a final evaluation for Land O’Lakes’ MSIKA program, which includes interviewing MSIKA beneficiary households in your community through the phone. Your household has been randomly selected to participate in this survey, which will ask questions about your household’s farming activities specifically in fruits and vegetable production and sales over the phone. The aim is to gather useful information that will be used to improve fruit and vegetable farming in this and other communities. The issues discussed in this interview will remain confidential and what you say will not be shared with anyone outside the research team. The survey will take about 45 minutes. Taking part in this research is voluntary. Are you willing to participate in this survey?” Consent is given by the person (continue to screening questions) Does not consent (Terminate interview and record the name and why they did not consent) 4. Screening Questions 1. What is your first name and family name? (check that it is the selected person, not a person sent on their behalf, such as a spouse or other relative) Yes, it is the selected person (continue) No, it is not the selected person (ask if the selected person is available and where to find them/how to contact them – then terminate interview and try to find the selected person, If not possible, inform the Supervisor). 2. Which of the following crops did you yourself or jointly with other family members plant and harvest, in the past 12 months between August 2019 to July 2020? a. Tomato – yes / no b. Onion – yes / no c. Potato – yes / no d. At least four mango trees e. Did not grow any of these If Yes to any one or more of the above (continue) If option e, (If they did not grow any of these crops in the last year (1st August 2019 to 31st July 2020), then they may not be the person that is registered. Check to see if it is the person. If so, find out if they ever grew these and note in what period they grew them. If not in the last 12 months end the interview and tell the Field Supervisor). 2b. Out of these crops, please say which is the most important crop to you in order, starting with the most important. (1 being the most important, 4 the least – ODK will only now give the ones they stated in 2 above) a. Tomato b. Onion c. Potato MSIKA Final Evaluation Page 110 kadale@africa-online.net d. Mango 3. Have you yourself received any training from the Land O’Lakes MSIKA project, including: (Read out list) • Training by a Land O’Lakes /MSIKA staff member or Land O’Lakes /MSIKA lead farmer(s) • or at a Land O’Lakes /MSIKA field day • or at a Land O’Lakes /MSIKA Yankho plot • or at a Land O’Lakes /MSIKA farmer field school? Yes (continue to A. General Information - If you continue and it becomes clear that they have not been truthful on the screening, check with the Supervisor) No (If they have not received any training or other services from Land O’Lakes /MSIKA, end interview and tell the Field Supervisor) As already communicated, we will send to you airtime worth MK 500 after the interview is completed. If the call is cut off, we will call you back. Does your batter have enough charge? If not, when is a good time to call you when it is charged? Enumerator instruction - Using the random crop selection method, make the selection from the crops they grow and tell them that we will be asking them about . ODK will link to the selected crop. The crop selected is: a. Tomato b. Onion c. Potato d. Mango 5. A. General Information Q# Question Response Logic A2 [Enumerator to note sex of respondent] Male Female A3 What is the highest level of education you reached? [Prompt if necessary] Adult literacy Standard 1-8 Form 1-4 Further Education None A4 What is your marital Status? [Prompt if necessary] Married Widowed Divorced Separated Never Marred A5 What is your age? [Estimate age of person if don’t know] [If less than 17 terminate interview] ___ years A6 Are you the head of the household (HHH)? Yes No A7 What is your relationship to the HHH? Head of HH Spouse of Head of HH Daughter/Son of HHH Brother/Sister of HHH Father/Mother of HHH Other relation of HHH Friend of HHH A8 BLANK [IF NOT HHH] A9 What is the sex of the HHH? Male Female [IF A6=No] A13 How many family members does this household have? A14 – A23 BLANK MSIKA Final Evaluation Page 111 kadale@africa-online.net Q# Question Response Logic State: Now I want to talk to you about your participation in the MSIKA project A25 Are you a Lead Farmer trained by Land O’Lakes /MSIKA? Yes No A25a As a Lead Farmer, have you done any of the following? Yes No If A25=Yes (multiple response possible) a. Conducted trainings of other farmers b. Submitted training forms to Land O’Lakes /MSIKA about the training c. Established a demonstration plot d. Other (please specify) A26 BLANK A27 Which of the following activities from Land O’Lakes /MSIKA have you participated in? [Read out list] a. Training in improved horticultural farming practices by Land O’Lakes /MSIKA staff or Lead Farmer Yes No b. Training in horticultural post-harvest handling practices by Land O’Lakes /MSIKA staff or Lead Farmer c. Training in marketing d. Training in financial literacy e. Training in gender equality f. Training or membership of a village savings and loan (VSL) group g. Been linked to an MFI, h. Been linked to markets to sell horticulture products i. Participated in an international or a district trade fair j. Any other? (specify) 6. S. Crops and Land Area Q# Question Response Logic Note I will now ask you questions related to [CROP] As selected in intro section S1a From how many mango trees did your household harvest mango fruit from for the first full harvest in the past 12 months (August 2019 to July 2020)? [If mango stated as one of their crops] Cross check that must be =>4 S1b How many times did you plant and harvest in the last 12 months (August 2019 to July 2020)? [This can only be for field crops not tree crops] One time Two times Three times [If one of the field crops is selected] S2a How many acres of did your household plant and harvest for the first full harvest in the past 12 months (August 2019 to July 2020)? [If S1 = yes] first planting (acres) S2b How many acres of did your household plant and harvest a second time in the past 12 months (August 2019 to July 2020)? second planting (acres) [If S1b is two or three times] S2b How many acres of these crops did your household plant and harvest a third time in the past 12 months (August 2019 to July 2020)? third planting (acres) [If S1b is three times] MSIKA Final Evaluation Page 112 kadale@africa-online.net C. Growing Practices [base crop relevance on question S2] Tomatoes (A) Q# Question Response Logic C1_1 BLANK C1_2 Which of the following improved practices did you use for your TOMATOES in the past 12 months (August 2019 to July 2020)? [Read out list] a. Composting (use of rotted down plant and other organic matter) Yes No b. Manuring (use of animal urine or excrement) c. Ridging (banking up the soil to form a ridge for planting on) d. Mulching (cover soil with dead plant/ compost to keep moisture/ suppress weed growth and improve soil fertility) e. Crop rotation (grow different crops on same plot in successive seasons) to control pests and diseases and improve soil fertility f. Minimum tillage (making planting holes only rather than tilling all the soil) g. Blank h. Planting seeds in a nursery before planting out i. Staking (adding stakes to enable plants to stay upright) j. Succession planting (plant part of plot one week, then other parts in following weeks to spread harvesting) k. Blank l. Using irrigation (application of water to crops using motorized or solar pumps, treadle pumps, watering cans or buckets, river diversion or canalization) m. Soil and water conservation (terracing, vetiver grass) n. Draining excess water o. Testing soil acidity p. Adding lime or ash to soil before planting to reduce acidity q. Choosing the variety (choosing different varieties for the growing characteristics and market that aiming for) r. De-suckering or removing unwanted side-shoots s. Spraying for pests and disease t. Using recommended plant and ridge spacing u. Using recommended fertilizer and application rates v. Sterilizing nursery beds before planting (burning crop residues to kill weed seeds, disease spores and pest eggs) w. Scouting for pests and diseases x. Uprooting infected plants and burning (to avoid spread of disease) y. Sowing seed in row/groove nursery z. Using fish soup / sugar solution as bait for insects aa. Planting in sunken beds bb. Hardening off before planting out (reduction of water to nursery seedlings) cc. Selecting the best seedlings for planting out dd. Intercropping - growing more than one type of crops (combining legumes with non-legume crops) on the same piece of land at the same time to improve soil fertility through nitrogen fixation, keep moisture, control soil erosion and suppress pest populations C1_4 On how many acres of TOMATOES have you applied any one or more of these improved practices in the past 12 months (August 2019 to July 2020)? [Acres] [Cross check to S2] The first planting and harvesting time The second planting and harvesting time The third planting and harvesting time MSIKA Final Evaluation Page 113 kadale@africa-online.net C1_5 Have you used this improved practice for the first time in the last 12 months? (August 2019 to July 2020) [List from C1_2] Yes No C1_7 Overall, what are the reasons why you did not use the other improved TOMATO practices? [Do not read responses] [enumerator classifies] [multiple response possible] Did not know it Expensive to use it Time consuming Inefficient Not confident it will work Not confident I can use it properly Too difficult for farmers to do it Did not understand it Inadequate resources Need did not arise Other (specify__________) If C1_2= no Onions (B) Q# Question Response Logic C2_1 BLANK C2_2 Which of the following improved practices did you use for your ONIONS in the past 12 months (August 2019 to July 2020)? [Read out list] a. Composting (use of rotted down plant and other organic matter) Yes No b. Manuring (use of animal urine or excrement) c. Ridging (banking up the soil to form a ridge for planting on) d. Earthing up (adding soil on growing plants to stimulate tuber development) e. Mulching (cover soil with dead plant/ compost to keep moisture/ suppress weed growth and improve soil fertility f. Crop rotation (grow different crops on same plot in successive seasons) to control pests and diseases and improve soil fertility g. Minimum tillage (making planting holes only rather than tilling all the soil,) h. Blank i. Planting seeds in a nursery before planting out j. Blank k. Using irrigation (application of water to crops using motorized or solar pumps, treadle pumps, watering cans or buckets, river diversion or canalization) l. Soil and water conservation (terracing, plant vetiver grass) m. Draining excess water n. Testing soil acidity o. Adding lime or ash to soil before planting to reduce acidity p. Choosing the variety (choosing different varieties for the growing characteristics and market that aiming for) q. Spraying for pests and disease r. Clipping plant ends to allow seedlings make fresh sprouts s. Sterilizing nursery beds before planting (burning crop residues to kill weed seeds, disease spores and pest eggs) t. Scouting for pests and diseases u. Using recommended plant and ridge/row spacing v. Using recommended fertilizer and application rates w. Uprooting infected plants and burning them (to avoid spread of disease) x. Sowing seed in row/groove nursery MSIKA Final Evaluation Page 114 kadale@africa-online.net Q# Question Response Logic y. Using raised beds (rainy season) z. Using fish soup / sugar solution as bait for insects aa. Planting in sunken beds (dry season) bb. Hardening off before planting out (reduction of water to nursery seedlings) cc. Selecting the best seedlings for planting out dd. Intercropping - growing more than one type of crops (combining legumes with non-legume crops) on the same piece of land at the same time to improve soil fertility through nitrogen fixation, keep moisture, control soil erosion and suppress pest populations C2_5 Have you used this improved practice for the first time in the last 12 months? (August 2019- July 2020) [List from C2_2] Yes No C2_7 Overall, what are the reasons why you did not use the other improved ONION practices? [Do not read responses] [enumerator classifies] [multiple response possible] Did not know it Expensive to use it Time consuming Inefficient Not confident it will work Not confident I can use it properly Too difficult for farmers to do it Did not understand it Inadequate resources Need did not arise Other If C2_2=no Irish Potatoes (C) Q# Question Response Logic C3_1 BLANK C3_2 Which of the following improved practices did you use for your IRISH POTATOES in the past 12 months (August 2019 to July 2020)? [Read out list] a. Composting (use of rotted down plant and other organic matter) Yes No b. Manuring (use of animal urine or excrement) c. Ridging (banking up the soil to form a ridge for planting on) d. Earthing up (adding soil on growing plants to stimulate tuber development) e. Mulching (cover soil with dead plant/ compost to keep moisture/ suppress weed growth and improve soil fertility f. Crop rotation (grow different crops on same plot in successive seasons) to control pests and diseases and improve soil fertility g. Blank h. Succession planting (plant part of plot one week, then other parts later) i. Blank j. Blank k. Using irrigation (application of water to crops using motorized or solar pumps, treadle pumps, watering cans or buckets, river diversion or canalization) l. Soil and water conservation (terracing, plant vetiver grass) m. Choosing the variety (choosing different varieties for the growing characteristics and market that aiming for) C2_4 On how many acres of ONIONS have you applied these improved practices in the past 12 months (August 2019 to July 2020)? [Acres] [Cross check to S2] The first planting time The second planting time The third planting time MSIKA Final Evaluation Page 115 kadale@africa-online.net Q# Question Response Logic n. Spraying for pests and disease o. Scouting for pests and diseases p. Using recommended plant and ridge spacing q. Using recommended fertilizer and application rates r. Uprooting infected plants and burning (to avoid spread of disease) s. Using fish soup / sugar solution as bait for insects t. Selection of seed potatoes u. Chitting (using diffused storage light to initiate sprouting of tubers) v. Intercropping - growing more than one type of crops (combining legumes with non-legume crops) on the same piece of land at the same time to improve soil fertility through nitrogen fixation, keep moisture, control soil erosion and suppress pest populations C3_5 Have you used this improved practice for the first time in the last 12 months? (August 2019- July 2020) [List from C3_2] Yes No C3_7 Overall, what are the reasons why you did not use the other IRISH POTATO improved practices? [Do not read responses – enumerator classifies] Did not know it Expensive to use it Time consuming Inefficient Not confident it will work Not confident I can use it properly Too difficult for farmers to do it Did not understand it Inadequate resources Need did not arise Other If C3_2= no Mangoes (D) Q# Question Response Logic C4_1 BLANK C4_2 Which of the following improved practices did you use for your MANGOES in the past 12 months (August 2019 to July 2020)? [Read out list] a. Composting (use of rotted down plant and other organic matter) Yes No b. Manuring (use of animal urine or excrement) c. Blank d. Pruning (cut back the canopy to let more light& air in; & to stop trees growing too tall to harvest from) e. Mulching (cover soil with dead plant/ compost to keep moisture/ suppress weed growth and improve soil fertility f. Spraying for pests and disease (use of crop protection chemicals) g. Blank h. Water capture (creating a basin around fruit trees) i. Using irrigation (application of water to crops using motorized or solar pumps, treadle pumps, watering cans or buckets, river diversion or canalization) j. Soil and water conservation (terracing, plant vetiver grass) k. Draining excess water l. Testing soil acidity m. Adding lime or ash to soil before planting to reduce acidity n. Blank C3_4 Excluding ridging, on how many acres of IRISH POTATOES have you applied these farming practices in the last 12 months? (August 2019 to July 2020) [Acres] [Cross check to S2] The first planting time The second planting time The third planting season MSIKA Final Evaluation Page 116 kadale@africa-online.net Q# Question Response Logic o. Blank p. Blank q. Scouting for pests and diseases r. Blank s. Uprooting infected plants and burning (to avoid spread of disease) t. Using fish soup / sugar solution as bait for insects u. Grafting (using stems of plants) v. Budding (using plant buds) w. Staking young trees x. Digging of planting holes prior to planting y. Deflowering of first flowers C4_7 Overall, what are the reasons why you did not use the other improved MANGO practices? [Do not read responses – enumerator classifies] Did not know it Expensive to use it Time consuming Inefficient Not confident it will work Not confident I can use it properly Too difficult for farmers to do it Did not understand it Inadequate resources Need did not arise Other If C4_2= no D. Harvest and Post-Harvest Handling Practices [base crop relevance on question S2] Tomatoes (A) Q# Question Response Logic D1_2 Which improved harvest & post-harvest practices did you use for handling TOMATOES the last 12 months (August 2019 to July 2020)? [Read list] a. Timely harvesting not too early so immature and not too late so over-ripe Yes No b. Handling carefully to avoid damage c. Storing in cool places out of the sun prior to sale d. Grading by color, size and shape j. Sorting by variety, size/maturity, damaged, ripeness e. Packing - Loading into appropriate containers for storage or transportation f. Not over-filling baskets & bags or using very big bags, as this damages product at the bottom g. BLANK h. Not stacking more than three containers high to avoid crushing tomatoes in lower containers. i. Adding soft dry grass as soft litter D1_2b Have you used this improved practice for the first time in the last 12 months? (August 2019 to July 2020) [List from D1_2] Yes No C4_4 On how many MANGO trees have you applied at least one of these improved practices in the past 12 months? (August 2019- July 2020) [Trees] [Cross check to S2] C4_5 Have you used this practice for the first time in the last 12 months? (August 2019- July 2020) [list from C4_2] Yes No MSIKA Final Evaluation Page 117 kadale@africa-online.net Onions (B) Q# Question Response Logic D2_2 Which improved harvest & post-harvest practices did you use for handling ONIONS in the last 12 months (August 2019 – July 2020)? [Read list] a. Harvesting & selling green/fresh onions with stalks on Yes No b. Lifting when at least 20% of tops have bent over & dried; leave until outer is dry & brown (cured) Drying (curing) onions in the sun or under a shade before storing to improve in keeping quality d. Trimming dried stems and roots before packing e. Grading by variety, size and appearance f. Packing - Loading commodities into appropriate bags/containers for storage or transportation g. Not over-filling baskets & bags or using very big bags, as this damages product at the bottom h. Sorting by variety, dryness, removing damaged etc D2_2b Have you used this improved practice for the first time in the last 12 months? (August 2019 to July 2020) [List from D2_2] Yes No Irish Potatoes (C) Q# Question Response Logic D3_2 Which improved harvest & post-harvest practices did you use for handling IRISH POTATOES the last 12 months (August 2019 to July 2020? [Read list] a. Harvesting when plant tops wilt and start to wither; slashing tops close to soil to cure in soil a day before harvesting Yes No b. Lifting potatoes with a fork/prong, not a hoe c. Rubbing off surface dirt after harvest; or after drying and curing d. Blank e. Keeping clean potatoes in a cool dry place to cure or heal surface damage, before packing, storage & selling f. Handling carefully to avoid damage – remove all damaged potatoes to avoid rotting g. Grading by color, size and shape k. Sorting by variety, maturity, removing damages, other materials etc h. Packing - Loading commodities into appropriate bags/containers for storage or transportation i. Using night vented sheds for short term storage for selling outside main season D3_2b Have you used this improved practice for the first time in the last 12 months? (August 2019 to July 2020) [list from D3_2] Yes No Mangoes (D) Q# Question Response Logic D4_2 Which improved harvest & post-harvest practices did you use for handling MANGOES the last 12 months (August 2019 to July 2020)? [Read list] a. Harvesting when ripening just started and color begins to change Yes No b. Harvesting by climbing the tree c. Harvesting by using a special harvesting pole d. Catching fruit before it falls using bags or spreading sheets e. Handling carefully to avoid damage MSIKA Final Evaluation Page 118 kadale@africa-online.net Q# Question Response Logic f. Storing in cool places and out of the sun prior to sale g. Grading by color, size, and shape l. Sorting by variety, size/maturity, ripeness, removing damaged h. Packing - Loading commodities into appropriate containers for storage or transportation i. Storing in crates to prevent damage in transporting j. Washing to improve appearance k. Not over-filling baskets & bags or using very big bags as this damages product at the bottom D4_2b Have you used this improved practice for the first time in the last 12 months? (August 2019 to July 2020) [List from D4_2] Yes No Q# Question Response Logic D8 Where do you store this ? [Read responses] [select all that apply] Inside my house In a separate building/store/shed In a Nkhokwe/outside store In baskets/bags outside or in a pile FBO Warehouse Other D8_b Did you build any storage building or storeroom or shed in the last 12 months? Yes No D8_bb Did you restore any storage building or storeroom or shed in the last 12 months? Yes No If Yes D9a What is new building/store/shed’s floor area? [length x width in square meters] [1 large step = 1 pace = 1 meter] If D8_b= Yes D9_b What is the new building/store/shed’s height? [Meters] [1 large step = 1 pace = 1 meter] If D8_b= Yes D9_c What was the total amount you spent on the new building/store/shed for storing crop? [MK] If D8_b= Yes D10_a What is the restored building/store/shed’s floor area? [length x width in square meters] [1 large step = 1 pace = 1 meter] If D8_bb= Yes D10_b What is the restored building/store/shed’s height? [Meters] [1 large step = 1 pace = 1 meter] If D8_bb= Yes D10_c What was the total amount you spent on restoring a building/store/shed for storing crop? [MK] If D8_bb= Yes F. Production & Sales [base V.C. relevance on question S2] Tomatoes (A) Q# Question Response Logic MSIKA Final Evaluation Page 119 kadale@africa-online.net F1_1a In the past 12 months (August 2019 to July 2020), what volume/Weight of TOMATOES did your household harvest in the first harvest? [20 litre pail has the same volume as ndowa] [other farmers use 40 litre or 60 litre volume pails for onions and tomatoes, convert to kgs based on 20 litre pail] # of 20lt Basins (dish) (weight=16kg) # of 40lt Pails (weight=32kg) # of 60lt Pails (Weight=48kg) # of kgs F1_2a From August 2019 to July 2020, what quantity did you sell of TOMATOES from your first harvest? [Ask for first harvest in the last 12 months] [<= production] # of 20lt Basins (dish) (weight=16kg) # of 40lt Pails (weight=32kg) # of 60lt Pails (Weight=48kg) # of kgs F1_3a From August 2019 to July 2020, what volume/weight of your first TOMATO harvest was spoiled [Ask for first harvest in the last 12 months] # of 20lt Basins (dish) (weight=16kg) # of 40lt Pails (weight=32kg) # of 60lt Pails (Weight=48kg) # of kgs F1_5a What was the common/typical SALES price for TOMATOES in the first harvest? [Ask for first harvest in the last 12 months] # of 20lt Basins (dish) (weight=16kg) # of 40lt Pails (weight=32kg) # of 60lt Pails (Weight=48kg) # of kgs F1_1b In the past 12 months (August 2019 to July 2020), what volume/Weight of TOMATOES did your household harvest in the second harvest? [20 litre pail has the same volume as ndowa] [other farmers use 40 litre or 60 litre volume pails for onions and tomatoes, convert to kgs based on 20 litre pail] # of 20lt Basins (dish) (weight=16kg) # of 40lt Pails (weight=32kg) # of 60lt Pails (Weight=48kg) # of kgs F1_2b From August 2019 to July 2020, what quantity did you sell of TOMATOES from your second harvest? [Ask for second harvest in the last 12 months] [<= production] # of 20lt Basins (dish) (weight=16kg) # of 40lt Pails (weight=32kg) # of 60lt Pails (Weight=48kg) # of kgs F1_3b From August 2019 to July 2020, what volume/weight of your second TOMATO harvest was spoiled? [Ask for second harvest in the last 12 months] # of 20lt Basins (dish) (weight=16kg) # of 40lt Pails (weight=32kg) # of 60lt Pails (Weight=48kg) # of kgs F1_5b What was the common/typical SALES price for TOMATOES in the second harvest? [Ask for second harvest in the last 12 months] # of 40Lt Basins (dish) (weight=32kg) # of 20lt Pails (weight=16kg) # of 60lt Pails (Weight=48kg) # of kgs MSIKA Final Evaluation Page 120 kadale@africa-online.net F1_1c In the past 12 months (August 2019 to July 2020), what volume/Weight of TOMATOES did your household harvest in the third harvest? [20 litre pail has the same volume as ndowa] [other farmers use 40 litre or 60 litre volume pails for onions and tomatoes, convert to kgs based on 20 litre pail] # of 20lt Basins (dish) (weight=16kg) # of 40lt Pails (weight=32kg) # of 60lt Pails (Weight=48kg) # of kgs F1_2c From August 2019 to July 2020, what quantity did you sell of TOMATOES from your third harvest? [Ask for third harvest in the last 12 months] [<= production] # of 20lt Basins (dish) (weight=16kg) # of 40lt Pails (weight=32kg) # of 60lt Pails (Weight=48kg) # of kgs F1_3c From August 2019 to July 2020 what volume/weight of your third TOMATO harvest was spoiled? [Ask for third harvest in the last 12 months] # of 20lt Basins (dish) (weight=16kg) # of 40lt Pails (weight=32kg) # of 60lt Pails (Weight=48kg) # of kgs F1_5c What was the common/typical SALES price for TOMATOES in the third harvest? [Ask for third harvest in the last 12 months] # of 20lt Basins (dish) (weight=16kg) # of 40lt Pails (weight=32kg) # of 60lt Pails (Weight=48kg) # of kgs F1_4 Why were the TOMATOES spoiled/lost? [Select all that apply] Damaged when harvesting Damaged at the farm in storage Damaged during transport to market Damaged at the market Could not be sold prior to spoiling DNK If F1_3 or F1_3c or F1_3c>0 F1_4b Why did you not sell any TOMATOES? No surplus to sell Could not get a buyer Prices were too poor Quality of my crop was too poor Other DNK If F1_2 = 0 F1_6 BLANK F1_7 Did you primarily sell TOMATOES: [Read responses] With farmers in my club/group With farmers not in my club/group On my own as an individual Other DNK Only if Sales > 0 F1_8 Did you sell TOMATOES at any of the following locations? [Read responses] [select all that apply] Neighbors Local market Traders who came to me Traders that I delivered to Distant market Direct to Processor/large buyer NGO My Farmer group Other Only if Sales > 0 F1_10 How was the weather for growing TOMATOES in the past 12 months (1st August 2019 to 31st July)? Favorable Moderate Bad DNK MSIKA Final Evaluation Page 121 kadale@africa-online.net Onions (B) Q# Question Response Logic F2_1a In the past 12 months (August 2019 to July 2020), what volume/Weight of ONIONS did your household harvest in the first harvest? [20 litre pail has the same volume as ndowa] [other farmers use 40 litre or 60 litre volume pails for onions and tomatoes, convert to kgs based on 20 litre pail] # of 20lt Pails (weight=22kg) # of 40lt Pails (weight=44kg) # of 50 kgs bags (Weight=55kg) # of oxcarts (Weight= 660kgs) # of kgs F2_2a From August 2019 to July 2020, what quantity did you sell of ONIONS from your first harvest? [Ask for first harvest in the last 12 months] [<= production] # of 20lt Pails (weight=22kg) # of 40lt Pails (weight=44kg) # of 50 kgs bags (Weight=55kg) # of oxcarts (Weight= 660kgs) # of kgs F2_3a From August 2019 to July 2020, what volume/weight of your first ONIONS harvest was spoiled [Ask for first harvest in the last 12 months] # of 20lt Pails (weight=22kg) # of 40lt Pails (weight=44kg) # of 50 kgs bags (Weight=55kg) # of oxcarts (Weight= 660kgs) # of kgs F2_5a What was the common/typical SALES price for ONIONS in the first harvest? [Ask for first harvest in the last 12 months] # of 20lt Pails (weight=22kg) # of 40lt Pails (weight=44kg) # of 50 kgs bags (Weight=55kg) # of oxcarts (Weight= 660kgs) # of kgs F2_1b In the past 12 months (August 2019 to July 2020), what volume/Weight of ONIONS did your household harvest in the second harvest? [20 litre pail has the same volume as ndowa] [other farmers use 40 litre or 60 litre volume pails for onions and tomatoes, convert to kgs based on 20 litre pail] # of 20lt Pails (weight=22kg) # of 40lt Pails (weight=44kg) # of 50 kgs bags (Weight=55kg) # of oxcarts (Weight= 660kgs) # of kgs F2_2b From August 2019 to July 2020, what quantity did you sell of ONIONS from your second harvest? [Ask for second harvest in the last 12 months] [<= production] # of 20lt Pails (weight=22kg) # of 40lt Pails (weight=44kg) # of 50 kgs bags (Weight=55kg) # of oxcarts (Weight= 660kgs) # of kgs F2_3b From August 2019 to July 2020, what volume/weight of your second ONIONS harvest was spoiled? [Ask for second harvest in the last 12 months] # of 20lt Pails (weight=22kg) MSIKA Final Evaluation Page 122 kadale@africa-online.net # of 40lt Pails (weight=44kg) # of 50 kgs bags (Weight=55kg) # of oxcarts (Weight= 660kgs) # of kgs F2_5b What was the common/typical SALES price for ONIONS in the second harvest? [Ask for second harvest in the last 12 months] # of 20lt Pails (weight=22kg) # of 40lt Pails (weight=44kg) # of 50 kgs bags (Weight=55kg) # of oxcarts (Weight= 660kgs) # of kgs F2_1c In the past 12 months (August 2019 to July 2020), what volume/Weight of ONIONS did your household harvest in the third harvest? [20 litre pail has the same volume as ndowa] [other farmers use 40 litre or 60 litre volume pails for onions and tomatoes, convert to kgs based on 20 litre pail] # of 20lt Pails (weight=22kg) # of 40lt Pails (weight=44kg) # of 50 kgs bags (Weight=55kg) # of oxcarts (Weight= 660kgs) # of kgs F2_2c From August 2019 to July 2020, what quantity did you sell of ONIONS from your third harvest? [Ask for third harvest in the last 12 months] [<= production] # of 20lt Pails (weight=22kg) # of 40lt Pails (weight=44kg) # of 50 kgs bags (Weight=55kg) # of oxcarts (Weight= 660kgs) # of kgs F2_3c From August 2019 to July 2020, what volume/weight of your third ONIONS harvest was spoiled? [Ask for third harvest in the last 12 months] # of 20lt Pails (weight=22kg) # of 40lt Pails (weight=44kg) # of 50 kgs bags (Weight=55kg) # of oxcarts (Weight= 660kgs) # of kgs F2_5c What was the common/typical SALES price for ONIONS in the third harvest? [Ask for third harvest in the last 12 months] # of 20lt Pails (weight=22kg) # of 40lt Pails (weight=44kg) # of 50 kgs bags (Weight=55kg) # of oxcarts (Weight= 660kgs) # of kgs F2_4 Why were the ONIONS spoiled/lost? [Select all that apply] Damaged when harvesting Damaged at the farm in storage Damaged during transport to market Damaged at the market Could not be sold prior to spoiling DNK F2_4b Why did you not sell any ONIONS? No surplus to sell Could not get a buyer Prices were too poor Quality of my crop was too poor Other DNK If F2_2 = 0 MSIKA Final Evaluation Page 123 kadale@africa-online.net F2_6 BLANK F2_7 Did you primarily sell ONIONS: [READ RESPONSES] With farmers in my club/group With farmers not in my club/group On my own as an individual Other DNK (do not read out) Only if Sales > 0 F2_8 Did you sell ONIONS at any of the following locations? [Read responses] [select all that apply] Neighbors Local market Traders who came to me Traders that I delivered to Distant market Direct to Processor/large buyer NGO Ny farmer group Other Only if Sales > 0 F2_10 How was the weather for growing ONIONS in the past 12 months (August 2019 to July 2020)? Favorable Moderate Bad DNK Irish Potatoes (C) Q# Question Response Logic F3_1a In the past 12 months (August 2019 to July 2020), what volume/Weight of IRISH POTATOES did your household harvest in the first harvest? # of 20lt Pails (weight=19kg) # of 40lt Pails (weight=38kg) # of 50 kgs bags (Weight=70kg) # of oxcarts (Weight= 840kgs) # of kgs F3_2a From August 2019 to July 2020, what quantity did you sell of IRISH POTATOES from your first harvest? [Ask for first harvest in the last 12 months] [<= production] # of 20lt Pails (weight=19kg) # of 40lt Pails (weight=38kg) # of 50 kgs bags (Weight=70kg) # of oxcarts (Weight= 840kgs) # of kgs F3_3a From August 2019 to July 2020, what volume/weight of your first IRISH POTATOES harvest was spoiled [Ask for first harvest in the last 12 months] # of 20lt Pails (weight=19kg) # of 40lt Pails (weight=38kg) # of 50 kgs bags (Weight=70kg) # of oxcarts (Weight= 840kgs) # of kgs F3_5a What was the common/typical SALES price for IRISH POTATOES in the first harvest? [Ask for first harvest in the last 12 months] # of 20lt Pails (weight=19kg) # of 40lt Pails (weight=38kg) # of 50 kgs bags (Weight=70kg) # of oxcarts (Weight= 840kgs) # of kgs F3_1b In the past 12 months (August 2019 to July 2020), what volume/Weight of IRISH POTATOES did your household harvest in the second harvest? # of 20lt Pails (weight=19kg) # of 40lt Pails (weight=38kg) # of 50 kgs bags (Weight=70kg) MSIKA Final Evaluation Page 124 kadale@africa-online.net # of oxcarts (Weight= 840kgs) # of kgs F3_2b From August 2019 to July 2020, what quantity did you sell of IRISH POTATOES from your second harvest? [Ask for second harvest in the last 12 months] [<= production] # of 20lt Pails (weight=19kg) # of 40lt Pails (weight=38kg) # of 50 kgs bags (Weight=70kg) # of oxcarts (Weight= 840kgs) # of kgs F3_3b From August 2019 to July 2020, what volume/weight of your second IRISH POTATOES harvest was spoiled? [Ask for second harvest in the last 12 months] # of 20lt Pails (weight=19kg) # of 40lt Pails (weight=38kg) # of 50 kgs bags (Weight=70kg) # of oxcarts (Weight= 840kgs) # of kgs F3_5b What was the common/typical SALES price for IRISH POTATOES in the second harvest? [Ask for second harvest in the last 12 months] # of 20lt Pails (weight=19kg) # of 40lt Pails (weight=38kg) # of 50 kgs bags (Weight=70kg) # of oxcarts (Weight= 840kgs) # of kgs F3_1c In the past 12 months (August 2019 to July 2020), what volume/Weight of IRISH POTATOES did your household harvest in the third harvest? # of 20lt Pails (weight=19kg) # of 40lt Pails (weight=38kg) # of 50 kgs bags (Weight=70kg) # of oxcarts (Weight= 840kgs) # of kgs F3_2c From August 2019 to July 2020, what quantity did you sell of IRISH POTATOES from your third harvest? [Ask for third harvest in the last 12 months] [<= production] # of 20lt Pails (weight=19kg) # of 40lt Pails (weight=38kg) # of 50 kgs bags (Weight=70kg) # of oxcarts (Weight= 840kgs) # of kgs F3_3c From August 2019 to July 2020, what volume/weight of your third IRISH POTATOES harvest was spoiled? [Ask for third harvest in the last 12 months] # of 20lt Pails (weight=19kg) # of 40lt Pails (weight=38kg) # of 50 kgs bags (Weight=70kg) # of oxcarts (Weight= 840kgs) # of kgs F3_5c What was the common/typical SALES price for IRISH POTATOES in the third harvest? [Ask for third harvest in the last 12 months] # of 20lt Pails (weight=19kg) # of 40lt Pails (weight=38kg) # of 50 kgs bags (Weight=70kg) # of oxcarts (Weight= 840kgs) # of kgs MSIKA Final Evaluation Page 125 kadale@africa-online.net F3_4 Why were the IRISH POTATOES spoiled/lost? [Select all that apply] Damaged when harvesting Damaged at the farm in storage Damaged during transport to market Damaged at the market Could not be sold prior to spoiling DNK F3_4b Why did you not sell any IRISH POTATOES? No surplus to sell Could not get a buyer Prices were too poor Quality of my crop was too poor Other DNK If F3_2 = 0 F3_6 BLANK F3_7 Did you primarily sell IRISH POTATOES: [Read responses] With farmers in my club/group With farmers not in my club/group On my own as an individual Other DNK (do not read out) Only if Sales > 0 F3_8 Did you sell IRISH POTATOES at any of the following locations? [Read responses] [select all that apply] Neighbors Local market Traders who came to me Traders that I delivered to Distant market Direct to Processor/large buyer NGO My Farmer group Other Only if Sales > 0 F3_10 How was the weather for growing IRISH POTATOES in the past 12 months (August 2019 to July 2020)? Favorable Moderate Bad DNK Mangoes (D) Q# Question Response Logic F4_1 What volume/Weight of MANGOES did your household harvest in the past 12 months (August 2019 to July 2020)? [Do not double count] # of 15lt Pails (weight=13.9kg) # of 20lt Pails (weight=16.3kg) # of 40lt Pails (weight=32.6kg) # of kgs F4_2 From August 2019 to July 2020, what quantity did you sell of MANGOES? [<= production] # of 15lt Pails (weight=13.9kg) # of 20lt Pails (weight=16.3kg) # of 40lt Pails (weight=32.6kg) # of kgs F4_3 From August 2019 to July 2020, what volume/weight of MANGOES was spoiled? # of 15lt Pails (weight=13.9kg) # of 20lt Pails (weight=16.3kg) # of 40lt Pails (weight=32.6kg) # of kgs MSIKA Final Evaluation Page 126 kadale@africa-online.net F4_5 From August 2019 to July 2020, what was the common/typical SALES price for MANGOES? # of 15lt Pails (weight=13.9kg) # of 20lt Pails (weight=16.3kg) # of 40lt Pails (weight=32.6kg) # of kgs F4_4 Why were the MANGOES spoiled/lost? [Select all that apply] Damaged when harvesting Damaged at the farm in storage Damaged during transport to market Damaged at the market Could not be sold prior to spoiling DNK F4_4b Why did you not sell any MANGOES? No surplus to sell Could not get a buyer Prices were too poor Quality of my crop was too poor Other DNK If F4_2 = 0 F4_6 BLANK F4_7 Did you primarily sell MANGOES with: [Read responses] Farmers in my club/group Farmers not in my club/group On my own as an individual Other DNK (do not read out) Only if Sales > 0 F4_8 Did you sell MANGOES at any of the following locations? [Read responses] [select all that apply] Neighbors Local market Traders who came to me Traders that I delivered to Distant market Direct to Processor/large buyer NGO My Farmer group Other Only if Sales > 0 F4_10 How was the weather for growing MANGOES in the past 12 months (August 2019 to July 2020)? Favorable Moderate Bad DNK H. Farm Management [base crop relevance on question S2] Q# Question Response Logic H1 BLANK H2 Have you used these management practices in the last 12 months? [READ PRACTICES] a. Keep good farm records Yes – In all ways Yes – In some ways No DNK b. (blank) c. Sell together with other farmers d. Do costings for growing crops e. Calculate profits after selling f. Plan production for upcoming season g. Plan production for a specific market h. Understand the specifications for specific buyers i. Separating commodities into the different grades j. Timely buying of inputs H3 BLANK H4 Did you employ or hire anyone for more than four weeks in the last 12 months (August 2019-July 2020)? Yes No H5 How many men did you employ or hire? If H4=Yes MSIKA Final Evaluation Page 127 kadale@africa-online.net H6 How many women did you employ or hire? If H4=Yes H7 How many weeks in the last 12 months (August 2019- July 2020) did you employ or hire men If H4=Yes H8 How many weeks in the last 12 months (August 2019- July 2020) did you employ or hire women If H4=Yes H9 – H13 BLANK H14 Did you buy any equipment in the last 12 months? Yes No H14b Did you buy any land in the last 12 months? Yes No H15 What was the total amount you spent on equipment in the last 12 months? If yes in H14 H15b What was the total amount you spent on land in the last 12 months? If yes in H14b I. Information [base crop relevance on question S2] Q# Question Responses Logic I2 Where do you get information about the following: [Read list] [Enumerator to classify responses] [Select all that apply] Neighbor/friend Govt ext’n officer Land O’Lakes/MSIKA extension worker Land O’Lakes/MSIKA lead farmer Private buyer ext’n officer NGO/Project ext’n officer Vendors/traders Radio SMS FBO Leadership Fellow FBO member Posters Agricultural shows Trade fairs Other sources Did not get this info a. Locations of markets to sell your crops b. Names of possible buyers c. Where to buy inputs d. Where to access to finance e. Buyers’ product requirements (volumes, grades, quality etc) f. Weather which can affect your crops g. Market prices for your crops J. COVID Information Tomatoes (A) J1_1a In what ways has COVID-19 affected your TOMATO farming and sales? [Unprompted] [Multiple response possible] [all responses have to be related to COVID, not just problems they faced which might be from other causes. Probe to check the response was due to COVID not some other factor – if it was from COVID and another factor, it should be recorded. Have never heard about Corona virus Not able to get inputs for the crop Not able to go to the fields to plant /look after the crop Not able to work because I was sick Not able to hire labour Not able to harvest Not able to go to the market Not able to sell at the market Other (Specify) J1_1b To what extent has COVID-19 affected your TOMATO farming? [Single response – Prompted] A lot Moderately Not a lot Not at all DNK/NR If J1_1a is not option a MSIKA Final Evaluation Page 128 kadale@africa-online.net Onions (B) J2_1a In what ways has COVID affected your ONION farming and sales? [Unprompted] [Multiple response possible] [all responses have to be related to COVID, not just problems they faced which might be from other causes. Probe to check the response was due to COVID not some other factor – if it was from COVID and another factor, it should be recorded. Have never heard about Corona virus Not able to get inputs for the crop Not able to go to the fields to plant /look after the crop Not able to work because I was sick Not able to hire labour Not able to harvest Not able to go to the market Not able to sell at the market Other (Specify) J2_1b To what extent has COVID affected your ONION farming? [Single response – Prompted] A lot Moderately Not a lot Not at all DNK/NR If J2_1a is not option a Irish Potatoes (C) J2_1a In what ways has COVID affected your IRISH POTATO farming and sales? [Unprompted] [Multiple response possible] [all responses have to be related to COVID, not just problems they faced which might be from other causes. Probe to check the response was due to COVID not some other factor – if it was from COVID and another factor, it should be recorded. Have never heard about Corona virus Not able to get inputs for the crop Not able to go to the fields to plant /look after the crop Not able to work because I was sick Not able to hire labour Not able to harvest Not able to go to the market Not able to sell at the market Other (Specify) J3_1b To what extent has COVID affected your IRISH POTATO farming? [Single response – Prompted] A lot Moderately Not a lot Not at all DNK/NR If J3_1a is not option a Mangoes (D) J4_1a In what ways has COVID affected your MANGO farming and sales? [Unprompted] [Multiple response possible] [all responses have to be related to COVID, not just problems they faced which might be from other causes. Probe to check the response was due to COVID not some other factor – if it was from COVID and another factor, it should be recorded. Have never heard about Corona virus Not able to get inputs for the crop Not able to go to the fields to plant /look after the crop Not able to work because I was sick Not able to hire labour Not able to harvest Not able to go to the market Not able to sell at the market Other (Specify) J4_1b To what extent has COVID affected your MANGO farming? [Single response – Prompted] A lot Moderately Not a lot Not at all DNK/NR If J4_1a is not option a MSIKA Final Evaluation Page 129 kadale@africa-online.net Thank you for your time. MSIKA Final Evaluation Page 130 kadale@africa-online.net Qualitative Interview Guide – Farmer Beneficiaries Name of Interviewer: __________________________ Date held:______________ 2020 Name of Farmer: _____________________________ FBO name: __________________________ District:___________________ GVH: __________________ EPA:_____________________________ Village: ______________________ Respondent is (circle one): Male 01 Female 02 Crop focus (circle one): Tomato Onion Irish potato Mango Other target crops grown (circle all that apply): Tomato Onion Irish potato Mango (Focus only on the main crop that the person is growing). (Check that the respondent is the named person that was selected not a spouse or a proxy, otherwise they cannot contribute). A. Introduction and Consent: “Hello, my name is and I am working with Kadale Consultants. Kadale has been asked to talk to fruit and vegetable farmers by MSIKA project, which is run by Land O’Lakes, a US organization that works with farmers. Your name has been given to us as someone who have been trained by MSIKA project. We would like to discuss your experience as a grower and about the training you received from Land O’ Lakes’ MSIKA project. The discussion will take about 45 minutes. What you tell us will not be attributed to any of you. Anything said by any of you should not be shared outside of the research team. Are you willing to take part in the discussion?” (If any say no, ask why and go to the next person) Dzina langa ndi [DZINA] ndimagwira ntchito ndi bungwe la Kadale Consultants, ku Lilongwe. Ife a Kadale Consultants tapemphedwa ndi a Land O’ Lakes kuti tilankhulane ndi alimi a mbeu za masamba komanso zipatso omwe bungweli limagwira nawo ntchito kudzera m’project yawo ya MSIKA. Atipatsa maina monga alimi omwe aphunzitsidwa ndi project ya MSIKA. Mukucheza kwathu, tikambirana nanu momwe ulimi wanu wa wakhala ukuyendera ndinso maphunziro omwe munalandira kuchokera ku project ya MSIKAyi. Kucheza kwathu kotenga ola limodzi ndi theka (1 hour 30 minutes). Zomwe tikambirane pano sidzidzazindikirika mwanjira ina iliyonse kuti mwanena ndiinuyo ndipo sidzidzamvedwa ndi ena omwe sali mbali ya kagulu komwe kakuyendetsa kafukufukuyu. Muli omasuka kutenga nao mbali pakafukufukuyu? (If any say no, ask why and go to the next person) Reason(s) for not giving consent: _________________________________________________________________________________ _________________________________________________________________________________ __________________ B. Background Information (to get the person talking) 1. How long have you been growing ? Were you growing this before being trained by LoL/MSIKA? Mwakhala mukulima mbeu imeneyi kwa nthawi yaitali bwanji? Munkalima mbeu imeneyi musanaphunzitsidwe ndi Land O’ Lakes/MSIKA? MSIKA Final Evaluation Page 131 kadale@africa-online.net C. Agricultural production training from MSIKA (nb. This can include training by MSIKA directly or by Lead Farmers trained by MSIKA. Post-Harvest Handling training is covered in the next section, so come back to that after discussing agricultural production training) 1. What agricultural production training did you receive from LOL /MSIKA, including by Lead Farmers trained by LOL/ MSIKA? What did you like about the training? How could it be improved? Kodi kuchokera ku Land O’ Lakes/MSIKA komanso ma Lead Farmer ake, munalandira maphunziro otani okhudza kasamalidwe ka mbeu? Ndi chiyani chomwe chinakusangalatsani pa maphunziro amenewo? Mukuona kuti zingasinthidwe bwanji kuti zikhale bwino kuposerapo? 2. Which of the agricultural production practices that you learnt from LOL/MSIKA have you been able to apply/use, starting with the most important ones? Ndinjira ziti za ulimi zomwe mwakwaniritsa kugwiritsa ntchito mutaphunzira kuchokera ku Land O’ Lakes/MSIKA? Potchula, tsogozani zomwe zili zofunika kuposa zina kwa inuyo Practices that have been applied 1. 2. 3. 4. 5. 6. Probe: Why apply these? Why not apply others? What stops you from using other practices that you learned about? This is a key question so probe extensively. 3. What difference has the agricultural production training by Land O’Lakes/MSIKA made to the amount of crop you produce and the quality of your crop, if any? If amount and quality have changed, how have they also changed the market pricing of the crops? Why do you think that is? Probe: This is trying to see what difference it has made to the amount and the quality of their production – ask them to compare with the seasons before they applied the practices, and ideally some quantification, even if it is “about half more than before.” Kodi maphunziro ochokera ku Land O’ Lakes/MSIKA a kasamalidwe ka mbeu asintha bwanji kakololedwe komanso quality ya mbeu yanu? Ngati quality ndi kakololedwe zasintha, kodi mitengonso yogulitsira mbeu yasintha? Ngati maphunzirowa asintha zimenezi, mukuganiza kuti zili choncho chifukwa chiyani? 4. Have you heard of corona virus or Covid-19? - if say yes, continue. If not, skip to 6. Kodi munamvapo za mulili wa matenda a COVID-19, kapena kuti Corona Virus? MSIKA Final Evaluation Page 132 kadale@africa-online.net 5. Did COVID-19 (Corona Virus) affect your crop’s production in any way? Kodi mulili wa matenda a COVID-19, kapena kuti Corona Virus, udakhudza ulimi wanu mu njira iliyonse? (Probe: If yes, how did it affect your crop production? What did you do to mitigate? Were the trainings that were offered by MSIKA useful in mitigating some of the impacts? Ask respondent to be specific about which trainings/techniques) 6. Which practices that you have learnt from MSIKA trainings are you likely to keep practicing beyond the end of the project? What will encourage you to continue practicing what you have learnt? What do you suggest could be done to have you, and other farmers continue with the practices you have learnt from MSIKA? What do you think could be done to have agro-dealers still work with farmers beyond the end of the project? Ndinjira ziti zaulimi zomwe mwaphunzira kuchokera ku MSIKA mukuona kuti mupitilirabe kugwiritsa ntchito ngakhale project itatha? Ndichiyani chomwe chikulimbikitseni kupitirizabe zomwe mwaphunzirazo? Mukuganiza kuti chofunika kuchitika ndichiyani kuti inu ndi alimi ena mupitirizebe kugwiritsa ntchito njira zomwe mwaphunzira ku MSIKAzi? Mukuganiza kuti chofunika kuchitika ndichiyani kuti ma agro-dealers (ogulitsa zipangizo zaulimi) apitirizebe kugwira ndi alimi ngakhale project itatha? (Probe: Ask respondents to be specific about which trainings/techniques? Why do you say that?) D. Post-harvest handling (If the respondent has not had training in PHH, then skip this section) 1. What training on post-harvest handling did you receive from Land O’Lakes/MSIKA including that from Lead Farmers trained by LOL/ MSIKA? What did you like about the approach? How can it be improved? Munalandira maphunziro otani kuchokera ku Land O’ Lakes/MSIKA kuphatikizapo ma Lead Farmer ake pa kasamalidwe ka zokolola zanu? Ndi chiyani chinakusangalatsani pa kaphunzitsidwe kawo? Mukuona kuti zingasinthidwe bwanji kuti zikhale bwino kuposerapo? (Note: this includes practices about handling the crop at 1. harvesting, 2. storage on the farm and 3. at the market, so ensure they consider practices for all three areas) 2. Which of the harvest and post-harvest handling practices that you learnt from Land O’Lakes/MSIKA (list from previous question) have you been able to apply/use, starting with the most important ones? Ndi maphunziro ati ochokera ku Land O’ Lakes/MSIKA okhudza kakololedwe ndi kasamalidwe ka zokolola omwe mwakwanitsa kugwiritsa ntchito? Potchula, tsogozani zomwe zili zofunika kuposa zina kwa inuyo? Practices that have been applied 1. 2. 3. 4. 5. MSIKA Final Evaluation Page 133 kadale@africa-online.net Practices that have been applied Probe: Why did they apply these? Why not apply others? What stops you from using other post-harvest and handling practices that you learned about? Chimakulepheretsani kukwanitsa kugwiritsa ntchito njira zina zomwe munaphunzirazo ndi chiyani? This is a key question so probe extensively. 3. Have you seen any change in losses at harvest, on the farm or at the market? Probe: If so, what has been the change? Mwaonako kusintha kulikonse pa zokolola zowonongeka, ku munda kapena ku msika? E. Markets 1. How do you sell your ? Mumagulitsa bwanji mbeu yanu ya ? Probe: Do you sell on your own? With others? Where do you sell your crops? Why do you sell it in this way? Mumagulitsa pa nokha? Pamodzi ndi anthu ena? Mumagulitsa kuti mbeu zanu? Mumagulitsa mu njira imeneyi chifukwa chani? 2. How has Land O’ Lakes/MSIKA better helped you to find markets for your crop? What works well and what does not work so well? How could MSIKA’s support to help farmers sell their crops be improved? Kodi a Land o’ Lakes akuthandizani bwanji pa nkhani yopeza misika? Ndi ziti zomwe zikuyenda bwino ndipo ndi ziti zomwe sizikuyenda bwino? Kodi chithandizo cha MSIKA chothandiza alimi kugulitsa mbeu zawo bwino chingapititsidwe bwanji patsogolo kuposerapo? Probe this to get different responses from group members and see how many get each type (e.g. market identification, market prices, market information). F. Other observations from the facilitator Thank you. MSIKA Final Evaluation Page 134 kadale@africa-online.net FBO KII Guide Prior to the FBO interview, MSIKA to provide the following info for the selected FBO: Name of FBO: Year Established: District: Target crops grown: EPA: GVH location: Village location: # of MSIKA farmers: Chairperson name Contact number: Secretary name Contact number: Treasurer name Contact number: How many sales agreements has the FBO made with the help of MSIKA? 2020 ___________ 2019 ___________ 2018 ___________ 2017 ___________ 2016 ___________ What value of FBO member sales are claimed by MSIKA? 2020 – MK _________ 2019 – MK _________ 2018 – MK _________ 2017 – MK _________ 2016 – MK _________ Has this FBO been trained in improved financial and organizational management? How many improved financial mgt practices and systems has MSIKA helped the FBO to implement? ___________ Has this FBO been trained in marketing? Has this FBO been trained in good governance? Prior to the FBO interview, MSIKA/LOL to provide the information for columns 1-3: MSIKA to provide Interviewer to complete Training module /name When held (month/yr) # of members trained Verified by interviewer Comments 1. Wholly/partially/ not verified 2. Wholly/partially /not verified 3. Wholly/partially /not verified 4. Wholly/partially /not verified 5. Wholly/partially /not verified 6. Wholly/partially /not verified MSIKA Final Evaluation Page 135 kadale@africa-online.net MSIKA to provide Interviewer to complete Training module /name When held (month/yr) # of members trained Verified by interviewer Comments 7. Wholly/partially /not verified 8. Wholly/partially /not verified Opening – interview with the committee member(s). My name is . We are conducting an independent review of Land O’Lakes’ MSIKA program which has been training farmer member organisations like yours, and training farmers in agricultural techniques and technologies. We want to ask you some questions that will help us to learn more about the work of Land O’Lakes/MSIKA. What you tell us will not be shared with Land O’Lakes/MSIKA, so they will not find out what you say about them. Are you happy to continue? (Yes/No) Dzina langa ndine . Ndimagwira ntchito ku bungwe la Kadale Consults, ku Lilongwe. Ife tikuchita kafukufuku, kuunika ntchito za MSIKA program yomwe, pansi pa bungwe la Land O’ Lakes, yakhala ikuphunzitsa ndikulimbikitsa pa alimi njira zamakono za ulimi wa mbeu za masamba ndi zipatso. Mayankho anu kumafunso athu atithandiza kumvetsetsa za ntchito za program ya MSIKAyi ndipo a MSIKA sadzadziwa zomwe inuyo mwatiyankha pakafukufukuyu. Ndinu okondweretsedwa kuti tipitilire? (Yes/No) 1. Please introduce the members of the committee: Mr/Mrs /Ms Name Position Phone # Chair person Secretary Treasurer 1. Tell me a bit about this FBO, such as when it was established, how many members it has and the crops your members grow. Mutandifotokozerako za gulu lanu/kalabu yanu, monga kuti inayamba liti komanso ili ndi mamembala angati, ngakhalenso mbeu zomwe mamembalawo akulima, mwa zina (This question aims to just start the conversation and make them comfortable – some existed before MSIKA, and some were formed by MSIKA) MSIKA Final Evaluation Page 136 kadale@africa-online.net When FBO started/ Inayamba liti Membership size/ Mamembala angati Crops grown/Mbeu zomwe mamembala akulima 2. What training have you received about running a farmer member-based organisation such as Mwalandirapo maphunziro anji pakayendetsedwe ka gulu la alimi longa lanuli la ? Training module /name When held (month/yr) # of members trained Verified by interviewer Comments Phunziro Liti? (Mwezi/ Chaka) Anaphunzitsa alimi angati? 1. Wholly/partially /not verified 2. Wholly/partially /not verified 3. Wholly/partially /not verified 4. Wholly/partially /not verified 5. Wholly/partially /not verified 6. Wholly/partially /not verified 7. Wholly/partially /not verified 8. Wholly/partially /not verified 3. What have been the most useful training the leadership of the FBO has received? Why do you say that? What has been the least useful? Why do you say that? In what ways could the training be improved? Ndimaphunziro ati a utsogoleri, omwe gulu lanu linalandira anali aphindu kwambiri? Mukutero chifukwa chiyani? Ndimaphunziro ati omwe ndiaphindu lochepa kwambiri? M’maganizo anu, chofunika kuchita ndichiyani kuti maphunzirowa akhale abwino kuposera pamenepa? Has this FBO been trained in improved financial and organizational management? Kodi gulu ili linaphunzitsidwapo luntha/luso lazachuma ndi How many improved financial mgt practices and systems has MSIKA helped the FBO to implement? Kodi ndi njira zingati zomwe Land O’ Lakes yathandiza gulu lanu kukhazikitsa ___________ MSIKA Final Evaluation Page 137 kadale@africa-online.net kayendetsedwe ka gulu? (probe to find out what training modules/content) ndikutsata pa luntha/luso lazachuma ndi kayendetsedwe ka gulu? 4. What improved crop practices have you seen many members now adopting? Why these ones and not others? What could help to increase the uptake of techniques and technologies by members? What have been the results on production volumes and quality from the training? Kodi ndinjira ziti za ulimi wamakono zomwe mwaonako alimi a gulu lanu akutsatira tsopano? Akutsatira zimenezi, kusiya zinazo chifukwa chiyani? Chingachitike ndichiyani kulimbikitsa kuti alimi ochuluka mugululi adzitsatira njirazi? Zotsatira zake za maphunzirowa ndizotani kumbali ya mulingo ndi kolite yazokolola? (push for numbers and/or examples of change). 5. What market information has the FBO received and what does it pass on to members? What training in accessing markets has the FBO and its members received? How have these trainings helped the FBO and its members to access markets? Can you give some examples of these new markets? Kodi gulu lanu lalandirapo mauthenga ati okhudza misika, ndipo ndi ati omwe lapereka kwa alimi ake? Gululi ndi alimi ake mwalandirapo maphunziro ati azamisika? Maphunziro amenewa athandiza gululi ndi alimi ake motani pakapezedwe kamisika? Mungandipatseko zitsanzo za misika yatsopanoyi? 6. How does the FBO help its members to sell their crops? What records do you keep of members sales? What was the volume of sales for your members in 2020, 2019, 2018, 2017, 2016? Kodi gululi limathandizira alimi ake motani kugulitsa mbeu? Mumalembera ndikusunga chiyani pa zomwe alimi anu akugulitsa? Alimi anu agulitsa zochuluka bwanji mu chaka cha 2020, 2019, 2018, 2017 ndi 2016? How many sales agreements has the FBO made with the help of MSIKA? Gululi lapeza migwirizano ingati yogulitsa mbeu mothandizidwa ndi Land O’ Lakes/MSIKA? 2020 ___________ 2019 ___________ 2018 ___________ 2017 ___________ 2016 ___________ What value of FBO member sales are claimed by MSIKA? Malonda omwe alimi anu anagulitsa mbeu mothandizidwa ndi Land O’ Lakes/MSIKA ndiokwana ndalama zingati? 2020- MK___________ 2019 -MK__________ 2018 - MK _________ 2017 - MK _________ 2016 - MK _________ MSIKA Final Evaluation Page 138 kadale@africa-online.net 7. Have you heard of covid-19 or corona virus? Has it affected the FBO activities in any way? What were these? What measures were taken to mitigate the effects? Are these effects continuing? Kodi munamvapo zamuliri wa matenda a COVID-19, kapena kuti Corrona Virus? Muliriwu wakhudza ntchito za committee yanu mu njira iliyonse? Munjira ziti? Panachitika chiyani kuti muthane ndi zovutazi? Zovutazi zikupitilirabe? Any other comments you want to share about working with MSIKA/Land O’ Lakes. Muli ndi ndemanga iliyonse pa momwe mwagwilira ntchito ndi a land O’ lakes/MSIKA? Thanks MSIKA Final Evaluation Page 139 kadale@africa-online.net KII Topic Guide AEDO or Horticulture Specialist – MSIKA Introduction: “My name is , and I am working for Kadale Consultants. Kadale has been asked by Land O’Lakes’ MSIKA project to talk to key stakeholders, such as Government Officers, who know about the implementation of its programme. We would like to discuss your experience of working with Land O Lakes’ MSIKA project. The discussion will take about an hour and a half. What you tell us will not be directly attributed to you, so Land O’Lakes’ MSIKA team will not know what you have said about them. Are you willing to be interviewed?” Consent given – Yes / No 1. When did you first come into contact with Land O’Lakes MSIKA team? What did they tell you about the program? What expectations did you have at the start? 2. What are the activities of the MSIKA program that you have seen being implemented in your District? Please tell me more about each of these Probe: Get the list of activities first, before getting the detail - Ask so that you get them all (“any other elements?)” – agricultural production training, post-harvest handling training, demonstration plots, formation and training of crop-based Farmer Business Organisations (FBOs), working with any processors or agro-dealers, etc. Probe to get details of each component per the table. We are looking for process issues – e.g. how did the co-ordination with govt go; how involved are govt in the identifying areas, making links etc.; how well organised were the activities, etc. We are also looking for content issues: how appropriate was the training content for these types of farmers, were these useful activities, etc. Probe for suggestions on ways to improve it and why they think that. While not seeking criticisms, explore any areas where there is some weakness to address, what contributes to this, how has it been addressed since being identified, etc. District: Name of interviewee: Contact # or email: Position of interviewee: Conducted by: Date conducted: MSIKA Final Evaluation Page 140 kadale@africa-online.net Activity Interviewee confirms What involvement have you had in the trainings and other activities run by LOL/MSIKA? What are your comments on how the activities were run (timely, how organised, AEDOs/AEDCs/HCS involved, etc.)? What are your observations on the ‘content’ of the activities (relevant, useful, something smallholders can implement, etc.)? Any improvements or suggestions? Agricultural Production Training Yes/No PHH Training Yes/No Demo plot establishment Yes/No Formation of, and training of Farmer Based Organisations Yes/No MSIKA Final Evaluation Page 141 kadale@africa-online.net 3. What changes have you seen as a result of the MSIKA program, such as changes in farmer knowledge and understanding, adopting agricultural practices, lower harvest losses, more sales, etc. Activity Include if mentioned in Q2 above What changes have you seen knowledge and use of practices? Have you seen any changes for farmers, such as higher production, higher quality, lower losses, more sales? Have you seen any changes in the operations of the FBOs? Any improvements or suggestions? Agricultural Production Training Yes/No PHH Training Yes/No Demo plot establishment Yes/No Formation of, and training of Farmer Based Organisations Yes/No MSIKA Final Evaluation Page 142 kadale@africa-online.net 4. How has MSIKA improved your and/or other government staff, such as other extn officers, knowledge and skills about horticultural crops and fruit trees? Probe: In what ways, who benefited, how great was the change in knowledge, skill, etc. 5. What has been the impact of COVID/Corona Virus on horticulture crop and fruit tree production and harvesting? Probe for specifics of in what ways (people getting sick, access to inputs, etc.), to what extent/how severe, affecting how many farmers in these value-chains (few, some, many, etc), etc. 6. What has been the impact of COVID/Corona Virus on horticulture crops and fruit tree markets? Probe for specifics of in what ways (markets closed, no transport, etc.), to what extent/how severe, affecting how many farmers in these value-chains (few, some, many, etc), etc. 7. What could be done to help the farmers continue practicing what they have been trained by MSIKA after the project’s end? Thank you. MSIKA Final Evaluation Page 143 kadale@africa-online.net MSIKA Final Evaluation Page 144 kadale@africa-online.net KII Topic Guide Processors – MSIKA Final Evaluation Prior to each processor interview, obtain key information from the LOL/MSIKA team/key documents on the background to the processor and the work that has been done with them. Check if MSIKA worked directly or via Technoserve. Brief background of the processor (what they process, products, markets, scale, plans, etc.) What LOL/MSIKA has done with them: Training conducted (specify trainings e.g. quality standards, good manufacturing practices, food safety, product marketing, Standards regulations – MBS, ISO, HACCP): Technical support/help facilitated (specify who by and the focus of the support) Facilitated loans (with which bank/MFI and how much, for what purpose): Facilitated MBS certification: Results claimed: Investment leveraged by MSIKA’s support (specify MK or USD, what was investment by firm from own funds versus what from bank loans facilitated by MSIKA): MBS Certification achieved with MSIKA’s help (pre-certify or full certified) Change in sales (specify if MK or USD) last year and over time worked with MSIKA: 2020: 2019: 2018: 2017: Sales agreements facilitated – number, value and with whom: New markets accessed and new products launched: Investment in current or new warehousing: Obtain the following data from MSIKA prior to interview. Type of employee # reported at Baseline # reported latest MSIKA data Full time – male Full time – female Part time – male Part time – female Temporary – male Temporary – female Introduction: “I am working for Kadale Consultants. Kadale has been asked by Land O’Lakes’s MSIKA project to talk to fruit and vegetable processors. Your business name and contacts have been given to us as an organization that MSIKA (nb. if relevant “and Technoserve) has been working with. We would like to discuss your experience of working with MSIKA. The discussion will take about 45 minutes . What you tell us will not be directly attributed to you, so Land O’Lakes will not know what you have said about them. Are you willing to take part in the discussion?” Consent given – Yes / No If no, ask if there is a particular reason for not giving consent (as this would be surprising). Name of organization: Name of interviewee: Contact # or email: Position of interviewee: Conducted by: Date conducted: MSIKA Final Evaluation Page 145 kadale@africa-online.net 1. “Tell me a little bit about your business, for example, when and how it started/you started it?” (This is a gentle entry to make them comfortable to talk, but move on reasonably quickly to) 2. “What have been the areas of support from, and collaboration with, LOL/MSIKA(/Technoserve)? “ (Probe: Check for any areas stated by LOL/MSIKA, but not mentioned by the processor. The respondent may not separate the support given, such as help with MBS certification, with the result of that work – record here the nature of the support/work done and the result on the next page) Support from MSIKA Responses: what done, how useful, challenges/things that worked well? Training conducted (specify all the trainings e.g. quality standards, good manufacturing practices, food safety, product marketing, Standards regulations – MBS, ISO, HACCP etc.): Technical support help direct from MSIKA or Technoserve of facilitated by these (specify who by and the nature of the support – what did they help with/work on?) Facilitated loans (what was the loan for, with which bank/MFI and how much loan obtained/): Facilitated MBS certification: Other? 3. Adoption of new business practices/techniques This is a key section for measuring indicator 19, so must be collected. The focus is on what was implemented/adopted, not what was learned/understood. The focus is on change, not continuation. Do not prompt on specific practices, but encourage them to keep giving as many changes as they can think of. New business practices implemented/adopted Based on the training and technical support from MSIKA, what business practices have you changed/adopted compared to what you were doing before working with MSIKA? (Keep asking – any others/any more, so as to prompt them to keep giving specific things that have changed) 1. Quality Assurance and Good Manufacturing Practices: • Implement Malawi specific product standards • Implement product packing standards • Develop new products based on new Mwi standards for tomato powder, chili pickles, potato starch, potato crisps, onion flakes, fruit leather, • Develop labelling for new products based on standard MS 19 • Steps to acquire and maintain certification/standards • Conduct analysis of foods • Take corrective action to bring foods up to standards/maintain standards • Personal hygiene practices when handling food – install improve washing facilities/toilets, staff improving their practices, etc. MSIKA Final Evaluation Page 146 kadale@africa-online.net New business practices implemented/adopted Based on the training and technical support from MSIKA, what business practices have you changed/adopted compared to what you were doing before working with MSIKA? (Keep asking – any others/any more, so as to prompt them to keep giving specific things that have changed) • Staff health – steps taken when staff are ill (i.e. not to work) and for well-being/health of staff, changes to personnel policies, safety rules implemented, clean clothes, protective equipment used • Food safety – changes to improve these, preventing contamination • HACCP – implementing the principles of HACCP • Factory layout and equipment to improve standards, equipment sanitising/cleaning • Waste disposal • Record keeping in support of quality/standards 2. Product Marketing Mission and vision • Customer analysis • Analysing customer attributes • Differentiating your products from competitors • Changes to way they are selling • Changes to the way they price their products • Changes to how they are promoting and communicating with customers • Assessing the competition • Changes to how they brand (name) their products • Inventory management 3. Gender equality Changes in business practices around treatment of women in the workplace by mgt and by staff Development of plans Implementation of gender plans. MSIKA Final Evaluation Page 147 kadale@africa-online.net 4. How has the support/collaboration from MSIKA assisted the business? (Ask on each result that LOL/MSIKA has indicated as counted for this processor – see page 1 for details): Results claimed: Details New investment made by the business (e.g. new processing equipment, bigger premises, more warehousing, more stock, etc. – ask what they did…) Have you increased your warehousing/storage capacity (dry or cold) with LOL/MSIKA help? Probe for details – when, by how much, in what way, why, etc.? Did the business access a loan(s) with MSIKA’s support? Loans amount (MK/USD): Loan accessed from (bank): Additional investment by the firm: (MK/USD) (did they put in their own funds or funds other than from a loan? How much? Specify where funds came from, if possible) MBS Certification achieved with MSIKA’s help (clarify if pre-certify or full certified and what their status was before MSIKA) When did they get this MBS Certification? If they have pre-certification, when do they hope to get full certification? Employment change and attribution to MSIKA (see table below) Change in sales (specify if MK or USD) last year and over time worked with MSIKA: 2020: 2019: 2018: 2017: Have there been any sales agreements facilitated by MSIKA with farmers that MSIKA has supported? How many, which FBOs? New markets accessed and new products launched (get specifics – e.g. new mango achar, selling to supermarkets following getting MBS certification): Other results of note? How many employees do you currently have? How has that changed in the last year and also since you started working with MSIKA? Type of employee Currently employed Change since last year (e.g. +4, -2, 0, against each category) Change since working with MSIKA (e.g. +4, - 2, 0, against each category) Full time – male Full time – female Part time – male Part time – female Temporary – male Temporary – female MSIKA Final Evaluation Page 148 kadale@africa-online.net 5. How has COVID affected your business operations and plans for investing? Probe: Any sickness/absence of staff (cautious probe if any employees have died), effect on suppliers particularly farmers/FBOs, any problems with getting produce to markets, any issues with falls in demand, etc. Do you have any questions for me? Thank you. MSIKA Final Evaluation Page 149 kadale@africa-online.net Annex 5: Comparison of Practices by Crop Practice Used Tomato Onion Potato Mango Baseline EoP Diff Baseline EoP Diff Baseline EoP Diff Baseline EoP Diff 1 Compositing 45.3 78.3 33.0 59.2 82.2 23.0 29.8 65.4 35.6 17.9 61.6 43.7 2 Manuring 64.9 80.1 15.2 73.5 79.9 6.4 42.9 64.3 21.4 14.5 57.0 42.5 3 Ridging 51.4 77.5 26.1 36.7 74.2 37.5 84.1 78.6 -5.5 n/a n/a n/a 4 Mulching 66.7 86.2 19.5 63.3 83.0 19.7 19.0 47.0 28.0 6.0 45.6 39.6 5 Crop rotation 45.6 78.3 32.7 50.0 85.2 35.2 53.6 81.2 27.6 n/a n/a n/a 6 Minimum tillage 14.1 50.0 35.9 7.1 47.3 40.2 5.5 n/a n/a n/a n/a n/a 7 Planting seeds in a nursery before planting out 86.2 96.4 10.2 85.7 95.5 9.8 13.8 n/a n/a n/a n/a n/a 8 Staking 80.8 97.5 16.7 n/a n/a n/a n/a n/a n/a n/a n/a n/a 9 Succession planting 29.1 59.1 30.0 n/a n/a n/a 29.8 43.2 13.4 n/a n/a n/a 10 Using irrigation 88.9 91.3 2.4 81.6 96.2 14.6 68.5 84.6 16.1 17.0 55.9 38.9 11 Soil and water conservation 24.0 71.0 47.0 25.5 67.4 41.9 n/a 71.8 n/a 3.4 60.5 57.1 12 Draining excess water 15.9 50.4 34.5 17.3 43.9 26.6 n/a n/a n/a 2.1 30.0 27.9 13 Testing soil acidity 0.3 19.9 19.6 1.0 25.8 24.8 n/a n/a n/a 0.0 18.6 18.6 14 Adding lime or ash to soil before planting to reduce acidity 0.6 33.7 33.1 1.0 37.9 36.9 n/a n/a n/a 0.0 28.1 28.1 15 Choosing the variety 34.5 79.7 45.2 39.8 73.5 33.7 n/a 81.6 n/a 7.7 n/a n/a 16 De-suckering or removing unwanted side-shoots 63.7 93.8 30.1 n/a n/a n/a n/a n/a n/a 15.3 n/a n/a 17 Spraying for pests and disease 67.3 92.8 25.5 67.3 87.1 19.8 n/a 79.3 n/a 3.0 43.0 40.0 18 Using recommended plant and ridge spacing n/a 98.6 n/a n/a 96.6 n/a n/a 94.4 n/a n/a n/a n/a 19 Using recommended fertilizer and application rates n/a 94.6 n/a n/a 94.3 n/a n/a 91.0 n/a n/a n/a n/a 20 Sterilizing nursery beds before planting n/a 80.4 n/a n/a 81.8 n/a n/a n/a n/a n/a n/a n/a 21 scouting for pests and diseases n/a 77.2 n/a n/a 69.3 n/a n/a 85.0 n/a n/a 44.1 n/a 22 Uprooting infected plants and burning n/a 87.3 n/a n/a 73.9 n/a n/a 74.8 n/a n/a 41.1 n/a 23 Sowing seed in row/groove nursery n/a 81.9 n/a n/a 84.1 n/a n/a n/a n/a n/a n/a n/a 24 Using fish soup/ sugar solution as bait for insects n/a 21.7 n/a n/a 21.2 n/a n/a 21.1 n/a n/a 15.2 n/a 25 Planting in sunken beds n/a 73.2 n/a n/a 80.7 n/a n/a n/a n/a n/a 0.0 n/a 26 Hardening off before planting out n/a 93.8 n/a n/a 93.2 n/a n/a n/a n/a n/a n/a n/a 27 Selecting the best seedlings for planting out n/a 96.0 n/a n/a 93.6 n/a n/a 95.9 n/a n/a n/a n/a 28 Intercropping n/a 51.8 n/a n/a 47.7 n/a n/a 36.1 n/a n/a n/a n/a 29 Green manuring 45.9 n/a n/a 49.0 n/a n/a n/a n/a n/a n/a n/a n/a 30 Water capture 25.8 n/a n/a 19.4 n/a n/a 22.1 n/a n/a 4.3 61.6 57.3 31 Earthing up n/a n/a n/a 79.6 91.7 12.1 83.0 94.4 11.4 n/a n/a n/a 32 Clipping plant ends n/a n/a n/a n/a 71.2 n/a n/a n/a n/a n/a n/a n/a MSIKA Final Evaluation Page 150 kadale@africa-online.net Practice Used Tomato Onion Potato Mango Baseline EoP Diff Baseline EoP Diff Baseline EoP Diff Baseline EoP Diff 33 Using raised beds n/a n/a n/a n/a 78.0 n/a n/a n/a n/a n/a n/a n/a 34 Chitting n/a n/a n/a n/a n/a n/a n/a 63.9 n/a n/a n/a n/a 35 Pruning n/a n/a n/a n/a n/a n/a n/a n/a n/a 62.1 86.8 24.7 36 Grafting n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a 22.8 n/a 37 Budding n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a 15.2 n/a 38 Staking young trees n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a 53.6 n/a 39 Digging of planting holes prior to planting n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a 61.6 n/a 40 Deflowering of first flowers n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a 29.7 n/a 41 Top Working n/a n/a n/a n/a n/a n/a n/a n/a n/a 23.4 n/a n/a Sample size/difference 333 276 -57 98 264 166 289 266 -23 235 263 28