Photo credit: Lucy O’Bryan/Absolute Options SOLUTIONS FOR AFRICAN FOOD ENTERPRISES (SAFE) FINAL EVALUATION REPORT January – June 2016 June 2016 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Gary Woller, David Rinck, and Emily Miller of Social Impact. SOLUTIONS FOR AFRICAN FOOD ENTERPRISES (SAFE) FINAL EVALUATION REPORT JANUARY – JUNE 2016 May 27, 2016 Revised June 30, 2016 Submitted September 14, 2016 AID-OAA-I-10-00013/AID-OAA-TO-14-00021 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 ii CONTENTS Contents .................................................................................................................................................................................................... ii Figures and Tables .................................................................................................................................................................................. iii Acronyms ................................................................................................................................................................................................ iiv Acknowledgements ................................................................................................................................................................................ v Executive Summary................................................................................................................................................................................ vi 1. Introduction...................................................................................................................................................................................1 1.1 Program Background...................................................................................................................................................................1 1.2 Theory of Change........................................................................................................................................................................2 2. Evaluation Overview ...................................................................................................................................................................4 2.1 Evaluation Purpose ......................................................................................................................................................................4 2.2 Evaluation Questions ..................................................................................................................................................................4 3. Methodology .................................................................................................................................................................................7 3.1 Data Collection Methods...........................................................................................................................................................7 3.2 Limitations of the Evaluation Methodology.........................................................................................................................12 3.3 Evaluation Team .........................................................................................................................................................................13 4. Evaluation Findings .....................................................................................................................................................................15 EQ1. ..........................................................................................................................................................................................................15 EQ2. ..........................................................................................................................................................................................................23 EQ3. ..........................................................................................................................................................................................................26 EQ4. ..........................................................................................................................................................................................................32 EQ5. ..........................................................................................................................................................................................................40 EQ6. ..........................................................................................................................................................................................................44 EQ7. ..........................................................................................................................................................................................................48 EQ8. ..........................................................................................................................................................................................................52 EQ9. ..........................................................................................................................................................................................................53 EQ10 ........................................................................................................................................................................................................72 Annexes ...................................................................................................................................................................................................78 Annex 1: Bibliography ..........................................................................................................................................................................79 Annex 2: STAKEHOLDERS INTERVIEWED ................................................................................................................................. 80 Annex 3: CONSUMER FOCUS GROUP DISCUSSIONS ..........................................................................................................83 Annex 4: Household Survey Instrument .........................................................................................................................................84 Annex 5: Discussion Guides ............................................................................................................................................................ 110 Annex 6: Processor Survey ............................................................................................................................................................. 126 Annex 7: In-Depth Case Study Findings ....................................................................................................................................... 128 Annex 8: Household Survey Findings, Kenya .............................................................................................................................. 142 Annex 9: household survey Findings, Zambia ............................................................................................................................. 161 ANNEX 10: PFS Training curriculum ........................................................................................................................................... 196 ANNEX 11: Terms of Reference (TOR) ..................................................................................................................................... 198 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 iii FIGURES AND TABLES Figure 1: SAFE Theory of Change .......................................................................................................................................................3 Table 1: Key Informant Interviews Conducted during Baseline and Endline Evaluation Rounds .......................................8 Table 2: Focus Group Discussions Conducted during Baseline and Endline Evaluation Rounds .......................................9 Table 3: Stakeholder Surveys Conducted during Baseline and Endline Evaluation Rounds ...............................................10 Table 4: Performance Improvements at Assisted Processors’ as a Result of SAFE Assistance........................................15 Table 5: Performance of Assisted Processors from the SAFE Performance Monitoring System: Cumulative as of March 30, 2016 ...........................................................................................................................................................................16 Table 6: Incremental Volume and Incremental Profits Calculated Using New Impact Assessment Methodology: Cumulative as of September 2015 .........................................................................................................................................19 The processor survey asked those SGBs who had participated in the impact assessment how useful they found it to their organization using a 4-point scale. As seen in Table 7, respondents in all three countries said, on average, that the impact assessment was ‘useful’ to their organization. ......................................................................................20 Table 7: Performance of Assisted Processors from the SAFE Performance Monitoring System: Cumulative as of March 30, 2016 ...........................................................................................................................................................................20 Table 8: Performance of Assisted Processors from the SAFE Performance Monitoring System: Cumulative as of March 30, 2016 ...........................................................................................................................................................................21 Table 9: Changes Made in Assisted Processors’ Business as a Result of SAFE Assistance ................................................24 Table 10: Performance of Assisted Processors from the SAFE Performance Monitoring System: Cumulative as of March 30, 2016 ...........................................................................................................................................................................25 Table11: Changes in Commercial Linkages at Assisted Processors’ as a Result of SAFE Assistance .............................32 Table 12: Performance of Assisted Processors from the SAFE Performance Monitoring System: Cumulative as of March 30, 2016 ...........................................................................................................................................................................33 Table 13. 2015 COMACO Procurements .....................................................................................................................................34 Table 14: Performance of Assisted Processors from the SAFE Performance Monitoring System: Cumulative as of March 30, 2016 ...........................................................................................................................................................................36 Table15: SGB Perceptions of the Extent to which Their Products Reach Vulnerable Households ................................43 Table16: WFP Short-Term Trainings ...............................................................................................................................................45 Table 17: SGB Satisfaction with Revised SAFE Onboarding Process ......................................................................................56 Table 18: Usefulness of Quality Audit and Business Assessment .............................................................................................56 Table 19: SGB Satisfaction with Quick Win Projects ..................................................................................................................57 Table 20: Estimated Impact of Quick Win Projects .....................................................................................................................58 Table 21: SGB Perceptions of TNS and PFS ..................................................................................................................................65 Table 22: SGB Satisfaction with TNS and PFS Assistance ..........................................................................................................66 Table 23: Overall Usefulness of SAFE Assistance .........................................................................................................................66 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 iv ACRONYMS BA Business Advisor BAT Business Assessment Tool BDS Business Development Services BFS Bureau of Food Security BSD Business Skills Development COMACO Community Markets for Conservation CSB Corn-Soy Blend DRC Democratic Republic of the Congo ET Evaluation Team EQ Evaluation Questions FGD Focus Group Discussion FBA Food Business Africa FT Food Technologist FTE Full-Time Employment FtF Feed the Future GM General Mills HACCP Hazard Analysis and Critical Control Points HEPS High-Energy Protein Supplement ISF Initiative for Smallholder Finance KCSE Kenya Certificate of Secondary Education KEBS Kenya Bureau of Standards KII Key Informant Interview KSU Kansas State University LOP Life of Program M&E Monitoring and Evaluation MBS Malawi Bureau of Standards MEP Monitoring and Evaluation Plan MOU Memorandum of Understanding MSI Management Systems International MT Metric Ton NASFAM National Smallholder Farmers’ Association of Malawi NPD New Product Development P4P Purchase for Progress PFS Partners in Food Solutions PLWA People Living with AIDS PMP Performance Management Plan SAFE Solutions for African Food Enterprises SBG Small and Growing Business SHF Smallholder Farmer SI Social Impact SWT Sector-Wide Training TA Technical Assistance TNS TechnoServe ToC Theory of Change SOW Statement of Work USAID United States Agency for International Development WFP World Food Programme Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 v ACKNOWLEDGEMENTS The SAFE Performance Evaluation Team wishes to acknowledge the cooperation of the staff at TechnoServe, Partners in Food Solutions, and Infinite Insight, our local research partner in Kenya and Zambia, who worked with us to collaboratively provide insights and access to the data required to complete this initial evaluation report. In addition, the team thanks the many food processors in Kenya, Malawi, and Zambia who welcomed us into their business places and generously shared their time and experiences with us over the course of our fieldwork. Finally, we extend our thanks and gratitude to USAID for its guidance and support throughout the course of the evaluation. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 vi EXECUTIVE SUMMARY This document presents the findings, conclusions, and recommendations for the endline evaluation round of the Solutions for African Food Enterprises (SAFE) performance evaluation implemented during March￾April 2016. The endline evaluation is the follow-up to the baseline evaluation round, which took place during March-April 2015. SAFE is a five-year Feed the Future (FtF) activity funded by the United States Agency for International Development (USAID) Bureau of Food Security (BFS) and implemented by TechnoServe (TNS) and Partners in Food Solutions (PFS). The program is scheduled to run from October 2012 through September 2017. SAFE’s stated goals and objectives are as follows:  High-level goal: Reduced poverty and improved nutritional status among poor and vulnerable households in priority food-insecure countries in Africa.  Goal: Expanded and increased competitiveness of the food-processing sector and increased availability of nutritious foods.  Objective: Strengthened capacity of African-based food-processing companies to produce and market quality, safe, and nutritious foods. The four primary components that make up the SAFE Theory of Change (ToC) are expected to contribute jointly at the outcome level to strengthen capacity among assisted food processors to produce and market quality, safe, and nutritious foods. This in turn is expected to lead at the goal level to an increase in the availability of nutritious foods and a more competitive processing sector. The high-level goal, which is the theoretical culmination of SAFE outputs, outcomes, and goals, is reduced poverty and improved nutritional status among poor and vulnerable households in SAFE program countries. Implementation of SAFE involves technology and knowledge transfer from volunteer experts from the five PFS partner companies of General Mills, Cargill, DSM, Bühler, and Hershey, which is delivered through the in-country facilitation and guidance of TNS food technology specialists and business advisors. (Hershey joined PFS as a partner corporation after the 2015 SAFE baseline.) Operationally, SAFE uses two primary intervention strategies: (1) customized technical assistance (TA) provided to individual food processors and (2) sector-wide trainings (SWTs) provided to food processors and other sector stakeholders on selected topics. Under the PFS volunteer model, volunteers typically provide TA from their home countries using distance communications methods such as Skype. This model may also include occasional on-site visits by PFS volunteers or other staff from the four PFS partner companies. TNS local staff plays an integral role in this process by identifying processors and helping to facilitate communications between the two parties, both in terms of arranging meetings and ensuring that the international experts and local business operators are “speaking the same language.” Local TNS business advisors and food technologists themselves also provide on-site TA to assisted processors in areas falling under their relevant expertise. Work with food processors is organized by charters, which are similar to memorandums of understanding (MOUs) describing the problem being addressed with a set of specific prioritized objectives and an implementation timeframe with clear milestones. Assisted processors have from one to several charters at different stages of implementation with SAFE. A particular charter may involve anywhere from a single to multiple PFS volunteers. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 vii The SWTs are one- to two-day events on topics of general interest to groups of food processors, such as fortification. They are led by consultants contracted by TNS, or are conducted by TNS staff. These events attract a wide range of enterprises, ranging from small-scale companies, to farmer cooperatives, and in some cases, larger companies. As such, they provide networking opportunities to the participants, as well as opportunities for companies of various scale to learn from each other. In selecting food processors to participate in the program, SAFE uses the following three selection criteria: 1. The food processor should source from smallholder farmers; 2. The food processor should reach, or potentially reach, large numbers of consumers; and 3. The food processor should be producing, or plan to produce, nutritious foods. To date, SAFE has provided technical assistance to 36 food processors (21 in Kenya, 10 in Zambia, and 5 in Malawi). An additional 20 new processors have recently been recruited, bringing the total to the program target of 56, while 11 processors have graduated. Since its inception, SAFE has also delivered 18 SWTs to more than 600 individuals from 462 companies in technical areas related to food such things as food quality, fortification, product development, and business strategy. BFS contracted Social Impact (SI) to undertake this performance evaluation of the SAFE program. The purpose of the SAFE performance evaluation is to provide BFS with empirical evidence to inform programming and investment decisions under the FtF initiative, including the following:  How well BFS centrally funded/centrally managed programs are implemented and performing;  How the programs are perceived by beneficiaries;  Whether the programs are achieving targeted outcomes in support of FtF objectives; and  What more needs to be done. This performance evaluation examines the extent to which the SAFE program has achieved the outputs, outcomes, and goals in its ToC. BFS will use the evidence from this evaluation, as well as other analyses, to identify critical gaps in programs in order to map out the way forward for SAFE over the remaining life of the program and to inform future FtF programming decisions in this area. The SAFE evaluation further seeks to answer the 10 key evaluation questions (EQs) found below. To do so, it employed a mixed-methods design using a combination of quantitative and qualitative data collection methods over two periods. As already mentioned, a baseline evaluation round was implemented during March–April 2015 and an endline evaluation round was implemented during March-April 2016. The primary data collection methods used in the endline evaluation round included: (1) key informant interviews (KIIs) with diverse program stakeholders, (2) focus group discussions (FGDs), (3) processor surveys, (4) household surveys, (5) farming system research, and (6) performance information from SAFE’s performance monitoring system. The baseline also included trainer and trainee surveys for the SWTs, although these were not repeated in the endline because, as explained below, SAFE did not implement additional SWTs between the baseline and endline evaluations. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 viii The 10 evaluation questions, and the primary conclusions and recommendations for each, are as follows: EQ1. Does participation in SAFE improve the performance of SAFE-assisted food-processing firms in the following areas: (1) management and technical capacity, (2) volume of nutritious food products produced, (3) sales of nutritious foods, (4) employment, (5) profitability, (6) investment, and (7) marketing of nutritious foods? Conclusions SAFE appears to have generated increases in production/productivity, volume, sales, profits, investment, and employment. While these improvements are not universal across all assisted small and growing businesses (SGBs), the large majority of those SGBs that have adopted recommendations made by local TNS staff and/or PFS volunteers have experienced a number of benefits in terms of improved performance in one or more of the preceding areas. Those supported SGBs who reported improved performance in the baseline were as likely to continue reporting improved performance in the endline indicating that the benefits of SAFE assistance have, for the most part, been sustained, at least over the twelve-month period between the baseline and endline evaluations. That the majority of SGBs interviewed in the baseline and/or endline reported improvements in how they do business as a result of SAFE assistance suggests that the realized improvements are not just a result of improved external circumstances or other fortuitous events but instead are, to a large degree, the result of internal improvements made that were a direct product of SAFE assistance. Thus the widespread evidence of internal, structural improvements among the supported SGBs bodes well for the long-term sustainability of the benefits received, all else equal. The impact assessment methodology developed and implemented by TNS since the baseline to measure SAFE’s impact on the incremental volume of nutritious foods produced and profits earned at supported SGBs (in lieu of the corresponding monitoring and evaluation plan (MEP) indicators) is an innovative approach to measure SAFE’s impact on incremental volume and profits. While this methodology includes an unknown, and likely significant, amount of measurement error, it is almost certainly a more accurate and credible approach than the previous methodology of simply counting the net change in the three indicators and attributing the entire change to SAFE assistance. SAFE Recommendations  TNS should continue to implement and refine the methodology, with an increasing focus on finding ways to streamline and improve it so as to (1) increase its accuracy and (2) lessen the likelihood that at some point in the future it will be viewed more negatively (e.g., as a time burden) than it was during its first round of implementation.  TNS should consider extending the impact assessment methodology to measure other key program outcomes wherever appropriate, including, for example, employment and investment.  TNS should avoid over-using the new impact assessment methodology. Administering the methodology once per year at a set time is sufficient and a much more replicable and scalable approach.  TNS should provide feedback to the participating SGBs after the conclusion of the impact assessment methodology to include a summary of the findings and any subsequent recommendations or final observations. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 ix General Recommendations  TNS’s approach to assessing the impact of SAFE assistance is a significant improvement over the previous outcome ‘counting’ approach in its MEP, in terms of both rigor and credibility. There is ample room to improve the rigor and credibility of outcome measurement methodologies across BFS-funded programs. Doing so will involve tradeoffs in terms of cost, time, and measurement priorities, but this report recommends BFS to encourage and facilitate experimentation with such approaches. It is also recommended that BFS monitor TNS’ impact assessment methodology closely to determine what lessons can be learned from it for application in other programs. EQ2. Have SAFE interventions contributed directly and/or indirectly to changes in knowledge and practices of stakeholders within the food-processing sector as a result of (1) technical assistance, (2) sector-wide trainings, (3) learning activities, and (4) demonstration effects? Conclusions SAFE technical assistance has been effective in improving the knowledge and practices among assisted SGBs. Nearly every SGB interviewed for the endline, and who was far enough along in the process for changes to have occurred, could cite specific examples about how SAFE technical assistance had contributed to improved knowledge and/or practices. For the most part, the changes cited were specific, and the SGBs could tie them directly back to SAFE assistance. In several cases, the changes at supported SGBs have persisted since the baseline. In many cases, the changes represent significant changes to basic production or business practices at the supported SGBs that have required significant changes to internal operations, production technologies, plant layout, software, etc. It would thus be difficult in many of these cases to backslide to former practice; in this respect, we would expect many of these changes to be sustainable over time, although this admittedly tells us little about how well the SGBs are implementing the relevant practices. In any case, after two rounds of talking with the assisted SGBs, we see reason to be optimistic that in many, albeit unknown, cases, the changes occurring have set the SGBs on a different long-term trajectory, for good or ill, than before SAFE assistance. Unfortunately, due to a budget realignment related to SAFE’s expansion into Ethiopia and Tanzania, SAFE scaled back the SWTs after the baseline evaluation. While we believe that the potential for the SWTs remains the same as in the baseline, we have no additional information to confirm or build on that conclusion in the endline. SAFE Recommendations  While the reason for scaling back on the SWTs over the last 12 months is understandable, it is recommended that SAFE scale up the SWTs over the remaining life of the program, which requires in turn recommitting money for their implementation, or, at the very least, in any SAFE extension. EQ3. Have SAFE interventions produced a demonstration effect as measured by the scaling￾up of SAFE-supported activities among assisted processors and the replication of the SAFE model among other development organizations or in other locations and countries? What is the likelihood that these demonstration effects will be sustained? Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 x Conclusions We can find no evidence that SAFE has produced any significant demonstration effects to date. To the extent that SAFE does have a significant demonstration effect in the future, the SWTs offer perhaps the best mechanisms for achieving it. Another option, which SAFE has adopted, is to collaborate with universities or training institutions to integrate SWT material into curriculum offerings. Toward this end, SAFE has engaged in an extensive and, on balance, effective participatory process to develop three learning manuals on topics of significant interest to the Kenyan food-processing sector: new product development, e-book on business strategy, and e-book on fortification. The new product development manual has been completed and is available for public use and has, so far, been integrated into the curriculum of at least one educational institution. The e-books on business strategy and fortification will be made available shortly at the Food Business Africa (FBA) website. SAFE does not have plans, or funding, to conduct post-development marketing activities for any of the three learning manuals. Work on the manuals largely ceased with the conclusion of the post-development launch workshop. The success of the manuals, and thus their contribution to SAFE’s sustainability, which was a primary motivation for their development, now depends heavily on word-of-mouth among industry stakeholders and on people accessing the e-books at the FBA website. SAFE Recommendations  We recommend scaling up the SWTs, possibly using one of the approaches described in this report. Any program extension should include adequate funding to scale-up the SWTs and to continue working to integrate SWT content into university and training institutions’ curricula.  SAFE should implement a post-development marketing strategy for the three industry-wide learning manuals, presuming funding can either be found or reallocated. Regardless, development of future industry-wide learning manuals should include funding for a strategy to market the manuals after development. Marketing activities may include workshops, business-to-business interactions, working more directly with learning institutions, using staff as guest lecturers at educational institutions to deliver excerpts from the manual to students, working directly with educational institutions to incorporate the manual or manual content into course syllabi and content, and so forth.  Development of future learning manuals should include funding and a plan to pilot test the manual with intended users prior to full rollout.  Other illustrative strategies to increase program demonstration effects (among many) include (1) increasing communications and outreach efforts by SAFE staff and clients to professional and technical associations; (2) launching a monthly or bi-monthly informal gathering after work with management and staff of other market development programs; (3) reaching out to business development services (BDS) and financial service providers, perhaps hosting an annual awards of innovators in food processing enterprises and inviting the press to cover the event; or (4) conducting SWTs in financing co-hosted by commercial lending and invoice discounting institutions. General Recommendations  Sustainability strategies should be built into program design at the outset and prioritized over the life of the program in terms of resource allocations even when they may come into conflict with other operational demands. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 xi  Demonstration effects are an important contributor to program sustainability. Like sustainability, however, they are unlikely (or less likely) to occur on their own without planned facilitation by the program and thus require an explicit strategy for achieving as part of the program’s sustainability strategy.  One way to help ensure the creation and implementation of a sustainability strategy is to create a line item and allocate funds specifically for this purpose and to include specific indicators within the program monitoring and evaluation plan to measure its achievement. Grouping sustainability in with other program operations as a funding line item and failing to hold the program accountable for achieving measurable results risks subsuming sustainability to other operational priorities, particularly those against which the program must report achievements. EQ4. Is participation in SAFE associated with new or strengthened market linkages between food-processing firms and (1) smallholder farmers, (2) wholesale buyers, (3) retail buyers, (4) food aid buyers, (5) lending institutions, and (6) other value chain actors? Conclusions SAFE has done a moderately good job facilitating new commercial linkages for its supported SGBs over the life of the program. Of all the different measures of SAFE performance included in the processor survey, SAFE’s success in facilitating new commercial linkages was consistently rated the lowest among respondent SGBs. Limited access to finance, both for working capital and fixed investment, remains as significant a constraint to assisted SGBs in the endline as it did in the baseline. This, combined with the lengthy delays incurred in getting paid by chain supermarkets (who are primary clients of several supported SGBs), is adversely affecting a number of the SAFE SGBs in developing their enterprises and moving forward as planned with the SAFE charters. At the same time, however, SAFE has become more active since the baseline in working to facilitate linkages between its supported SGBs and different sources of commercial finance using a commercialization approach, while also investing in creating an interactive tool to guide Kenyan SBGs in finding and accessing different types of financing. The farming systems research conducted by the Markets Specialist suggests that SAFE assistance can be effective in facilitating the expansion of SGBs that engage smallholder farmers directly through contract farming and out-growers operations, resulting in increases in productivity, which in turn are likely to increase consumption and income of farmers. Opportunities to replicate this success are limited by the prevailing procurement practices of most processors in the region. However, SAFE should leverage these opportunities where possible. SAFE Recommendations  SAFE should continue with its current commercialization approach to financing. There are also ways SAFE can possibly strengthen their approach. For example, SAFE could continue to provide, or link its clients to, business development training and counseling on improved record keeping, which is still a high priority according to TNS staff.  Since financing is not a core area of SAFE support to the SGBs, developing closer linkages to other donor programs, progressive commercial lenders, and government initiatives for SME financing is another possible approach to address financial access. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 xii  Sector-wide trainings on financial planning for agricultural enterprises, internal and external sources of credit are recommended along with linkages to business development services. SAFE could co-host in collaboration with other business service providers, donors, lending institutions, and program implementers to take advantage of existing resources such as the USAID DCA loan guarantee facility.  Extending the interactive tool being developed by the SAFE Kenya office to help SGBs navigate the financial landscape better to the other SAFE country programs would be worthwhile if the tool proves to be a helpful resource.  With regards to increasing its impact on SHFs, SAFE should prioritize and target assistance to SGBs that employ out-grower and contract farming schemes to increase the impact of assistance on smallholders in terms of increased market linkages and improved income and nutrition of farmers. SAFE should further consider providing complimentary assistance to SGBs on procurement models, or partner with projects that specialize in this area to enhance linkage between their assistance and impact on producers. Finally, SAFE should consider adding TA capacity in supplier support to the types of TA it provides, or partnering with a similar program that implements this type of support. General Recommendations  Consistency in philosophy and approach across a program’s diverse set of activities is important and should be adhered to in all (or nearly all) cases, such as in SAFE’s commitment to the commercialization approach in access to finance. While such an approach may not always maximize certain outcomes, it enhances the program’s credibility with stakeholders and helps ensure that the program is sending a consistent set of messages both to external stakeholders and internal staff.  Commercial linkages are much less likely to emerge organically and thus require an active facilitation effort by the program based on mutual advantage to all parties. A particular application of this principle in SAFE’s case is the program’s objective to generate downstream benefits to smallholder farmers supplying supported SGBs. As emphasized in both the baseline and endline, this goal cannot be assumed to occur organically but requires a specific strategy to achieve it, which includes at a minimum targeting SGBs who have existing commercial relationships with SHFs via some kind of out-grower or contract farming scheme. EQ5. Have SAFE interventions increased the availability of nutritious foods, particularly in markets characterized by higher levels of food insecurity among the consumer population? Conclusions Overall, SAFE-supported SGBs are not reaching vulnerable households with nutritious food products. This goal has not been prioritized in SAFE’s design or statement of work (SOW), and, not surprisingly, has not been incorporated into SAFE TA to supported SGBs. Nor, consistent with its mandate, has SAFE made any effort to target support for SGBs that are serving this market. Food consumption patterns did not undergo any change from the baseline to the endline among case study subjects in both Kenya and Zambia, while respondents in both countries remained largely unfamiliar with the concept of nutritious foods, and what constitutes nutritious food. Case study subjects in both countries make the large majority of their food purchases, as measured by frequency of purchases and volume of purchases, at informal market retail outlets. In Kenya, kiosks are Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 xiii the primary retail outlet of choice, followed by open-air markets and supermarkets. In Zambia, open-air markets are the primary retail outlet of choice, followed by kiosks and neighborhood stores. Kiosks and neighborhood stores offer a few key advantages that make them the outlet of choice for so many low-income shoppers. First, they repackage goods in small packs, which low-income shoppers can afford. Second, they often offer credit, which enables low-income shoppers to purchase when cash flow is lower and repay when cash flower is higher. Third, they offer convenience, often being located near the shopper’s place of residence. Fourth, they offer familiarity; kiosk and shop owners often know customers’ names and go to lengths to stock a wide variety of brands and pack sizes according to their familiarity with customer needs. Any food processor seeking to sell to marginal, food-insecure populations in large volumes, therefore, will need to adapt its business model so as to distribute its products through the kiosks and neighborhood stores that dominate the informal food retail market. To do so will require, at a minimum, packaging and pricing strategies targeted to this market. SAFE Recommendations  In light of SAFE’s goal to improve nutritional outcomes in poor and vulnerable households, which requires that nutritious foods produced with SAFE assistance actually reach these households, TNS’ decision to study the informal market is both well considered and overdue. We recommend that BFS actively support this research, as it has important implications for any SAFE extension as well as for future BFS programming decisions.  BFS should give serious consideration to revising the SOW for any SAFE extension to include a provision requiring, or encouraging, the follow-on program to support food processors to increase their presence in and penetration of the informal wholesale and retail food markets for processed foods, subject to the findings of the SAFE informal market research.  As demonstrated in our case study research, informal food markets in different countries share many characteristics, but also involve some important differences. Given that SAFE’s current informal market research is limited to Kenya, we recommend that BFS support additional research on informal food markets in the other countries served by SAFE or by any SAFE follow-on program. General Recommendations  Program design needs to be consistently checked against the program’s higher-level objectives and goals, even if the program is not strictly tasked with achieving those objectives or goals, so as to determine consistency between the design and the stated objective or goal. Engaging in this activity is likely in many cases to uncover hidden assumptions in the objective or goal that can then be checked against what is known about the existing program environment, and/or it can trigger investigations into the validity of key assumptions.  Generally speaking, poor and vulnerable households are likely to perceive and live a very different reality than program designers. To the extent a program aims to generate benefits for these households, whether directly or indirectly, it is critical to take the effort to learn what this alternative reality is and to check program assumptions against it. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 xiv EQ6. To what extent has any increase in the availability of nutritious foods been matched by an increase in access to (and consumption of) nutritious foods, particularly in markets characterized by higher levels of food insecurity among the consumer population? Conclusions By all indications, the short-term milling training delivered by SAFE in collaboration with the World Food Programme (WFP) was a success. The training was well received by all participants interviewed, who universally claimed that it had a significant impact either on their business, their on-farm production, or their household wellbeing. The short-term milling training accomplished several outcomes consistent with SAFE’s objectives. First, similar to the SWTs, they proved to be cost-effective mechanism for delivering useful technical assistance to multiple SGBs at a single time. Second, they reached a cohort of small, relatively disadvantaged SGBs for whom such technical assistance would normally be out of their research. Third, the assisted SGBs were, for the most part, selling directly or indirectly (via institutional buyers) into markets characterized by high levels of vulnerable households. Fourth, the trainings also directly or indirectly benefitted these farmers in terms of increased on-farm production and household income, and possibly even household nutritional status. SAFE Recommendations  If BFS is committed to increasing outreach to marginal communities and vulnerable households via TA to small food processors, the type of short-term miller trainings delivered by SAFE in conjunction with the WFP provide a potentially effective option for pursing these objectives, and should therefore be considered for expansion under the current SAFE program and for continuation under any SAFE follow-up program. General Recommendations  A single training is rarely sufficient to induce long-term, sustainable changes in production practices or outcomes. Thus a general recommendation is to pair up short-term technical trainings with follow￾on activities to reinforce past training topics and/or give training participants an opportunity to implement what they learned. Any number of options for follow-up activities exists; for example, produce and distribute a newsletter with examples of trainees who have successfully implemented practices or technologies covered in the training; offer a help desk for trainees to ask follow-up questions or receive virtual one-on-one mentoring; facilitate cross-organizational visits by training participants; hold contests among trainees to recognize those who have done the best jobs of implementing training concepts; visit trainees at their organizations to provide follow-up mentoring; or conduct post-training assessments to determine their effectiveness in changing trainees’ attitudes and behaviors. EQ7. Is participation in SAFE associated with increased leadership/management opportunities for women in SAFE-assisted food-processing firms? Conclusions Overall, SAFE has not been associated with leadership and advancement of women within its client SGBs. Its primary contribution has been in technical assistance and product development. Socially progressive enterprises that were women-owned or -managed from the outset and already employed a significant Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 xv number of women workers were ready and able to advance women as their businesses expanded with the other technical enterprise assistance from SAFE. The workplace environment of the socially progressive enterprises was already supportive of women and, most notably, the communications lines between management and unskilled and administrative workers were open and transparent. Using networks rather than relying heavily on mass media outlets to identify and recruit women-owned food-processing enterprises has achieved one of the program goals: increasing the number and ratio of women within the SAFE three-country client portfolio. However, most of the women-owned enterprises are micro or small, and some are not yet formally registered in their local district. The greatest impact on women-owned enterprises from the revised strategy has been in its advisement on management practices around women’s issues and workplace environment, such as maternity leave and changing rooms. The enterprises have also expressed their appreciation of one-on-one interaction with SAFE staff, especially female staff members who have made visits to the enterprise and continue to support and mentor the women business owners. SAFE Recommendations  SAFE should continue efforts to recruit women-owned food-processing enterprises in the final year of the program, since women-owned enterprises, and those that are socially progressive, are most likely to be advocates for women’s leadership and advancement opportunities.  Smaller women-owned enterprises need a full suite of business and technical capacity building, especially in business management and financing. If SAFE does not have the staffing resources needed for these labor-intensive inputs, it can strengthen linkages with existing BDS service providers that offer affordable business management services.  To the extent possible, continue site visits and one-on-one follow-up communications with the women business owners. Make introductions to and facilitate linkages with financial institutions and other sources of credit to help mitigate the financial constraints that most of the enterprises are facing. The gender focal person for the three countries covered in this evaluation should continue to share best practices from the TNS quarterly Gender Practice Group forums, especially on reaching out to women in gender-sensitive business management and financing.  A gender strategy and focus for women’s inclusion and empowerment must be in the initial program design. This is recommended for the 2017 solicitation. For women to advance, opportunities and constraints need to be recognized and planned out at the initial design phase. General Recommendations  As with other direct or indirect program benefits, gender impacts are far more likely to occur as a result of as a set of planned activities than they are to occur organically. To the extent BFS desires to produce gender impacts in similar programs, it requires a concerted and consistently implemented strategy, something that has not occurred to date at SAFE.  In the context of business support programs such as SAFE, the gender strategy for women’s economic empowerment should address increasing women’s agency in business management, decision making, negotiating, and access to markets, while also addressing the socio-cultural context in which women operate. Understanding this socio-cultural context will provide insights into changing behaviors with Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 xvi positive socio-economic outcomes and in taking up new practices. Access to information and skill development for enterprise management and line workers needs to be envisioned and articulated. Resources for developing women’s leadership in civic and business environments need to be facilitated.  BFS programming should reflect adequate budgetary resources, and encourage new or expanded programs, for recruiting technical and market development professionals who already possess experience in gender inclusiveness in program design and implementation. Gender focal persons are generally stretched too thinly across a broad spectrum of activities or a large geographic area. An example would be to recruit women intervention managers with a technical or business background who have previous experience working with women entrepreneurs or other female beneficiaries. EQ8. To what extent has SAFE led to increased income generation among women employees of SAFE-assisted food-processing firms? Conclusions SAFE has not been directly associated with increases in income of women workers within its client enterprises. Its primary contribution has been in technical and product development. TNS has rightly stated that this is beyond the program’s sphere of influence. SAFE Recommendations  Even given that increasing the income of female workers at SAFE-supported SGBs is beyond SAFE’s sphere of influence, this should not deter SAFE from encouraging, supporting, and providing a platform for capacity building of SGB staff (women and men) that, if well managed, can lead to greater productivity, growth and profitability in the enterprise and thus indirectly to the financial remuneration of its employees. SAFE can, moreover, contribute to advise its clients on promotion and advancement criteria, policies, and benchmarks as part of the human resource policies inventoried at the initial stages of client engagement with SAFE. General Recommendations  While increased employee income is a laudable objective in technical support programs like SAFE (in which the technical support is provided at the level of the firm), it is more likely to be a second-to￾third-order result, making it challenging to measure, more challenging to achieve, and even harder to attribute to program support. Unless such a technical support program has specific strategies/ initiatives to promote increased income among the supported firms’ employees, there is little purpose to making this an objective of BFS funding and/or holding the implementing agency accountable for it. EQ9. Was the SAFE program implemented as designed? Conclusions From one perspective, TNS and PFS did implement the program as it was designed, broadly speaking. Both adhered closely to the basic approach of selecting SGBs and providing them distance-based TA from international experts at PFS corporate partners, while TNS provided on-the-ground support and acted as an intermediary/translator for the PFS volunteers, who did most of the technical advising heavy lifting. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 xvii From another perspective, however, TNS and PFS did not implement the program as designed. Rather than take the model as is and attempt to impose it within the program countries, both TNS and PFS have been active since the beginning of the program in innovating the SAFE service delivery model to improve its efficiency and effectiveness. Notable innovations to the model include an improved SGB onboarding process; the expansion of SAFE client services via the quick win projects ask-an-expert and industry-wide learning manuals; standardized charters; a more methodologically robust performance measurement approach via its new impact assessment methodology; an increased focus on vulnerable households; and increased full-time support staff at PFS. Of these, the quick win projects and standardized charters represent the most significant innovations to the SAFE service delivery model to date. Key informants at both PFS and TNS are well aware of the constraints to the SAFE model of distance￾based service delivery using volunteer experts (e.g., lag times, communication difficulties, time differences, locally inappropriate solutions, and disruptions in service) and are realistic about its limitations. This relatively clear-eyed approach has abetted what appears to be a culture of learning and innovation within both organizations. In fact, their willingness to acknowledge and confront these limitations to the model directly motivates the innovations to model implemented by both organizations over the course of the evaluation. SAFE is an evolving program, and the evolution is moving progressively toward creating a better program and a better service delivery model, albeit one with persistent limitations. This evolution, however, carries with it an increasing tension, particularly related to the relative roles of local TNS staff and PFS volunteers, as they continue to work out the best mix of providing locally-based technical solutions provided by local or regional experts (including TNS staff) and distance-based solutions provided by international volunteer experts. How PFS and TNS manage this tension will be one of the most defining characteristics of the model’s evolution over time. SAFE Recommendations  PFS and TNS should be encouraged, and given ample space by BFS, to continue to innovate the service delivery model, even to the point where certain innovations fail or fare less well than others. It is clear over the two evaluation rounds that the inherent limitations to the model require ongoing innovation to address, and solutions to these limitations are more likely to be addressed incrementally over time via a process of ongoing innovation and improvement as opposed to a single, point-in-time solution that solves the problem.  In particular, BFS should encourage PFS and TNS to redefine over time and through practice the relative roles of local TNS staff and local/regional experts versus the PFS volunteer experts, as represented to date by the advent and evolving role played by the quick win projects and standardized charters. General Recommendations  Implementing a program ‘as planned’ should not necessarily be assumed to be a good thing. There are often very good reasons for deviating program implementation from planned implementation. In cases where program deviations occur, it is important to understand why the deviations occurred.  SAFE’s experience highlights the general recommendation that programs often need the space and flexibility to innovate and learn by trial and error and to adapt program design and implementation as appropriate. This recommendation is particularly true for programs like SAFE that implement a pre￾packaged, or off-the-shelf, methodology (in this case, distance-based TA by international experts). Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 xviii EQ10. What might SAFE have done differently to improve program implementation? This section summarizes the evaluation team’s initial conclusions about what might have been done differently to this point to improve program implementation and results drawing on the findings and conclusions for Evaluation Questions 1 through 9. In this sense, they should also be interpreted as recommendations for what the program might do to improve implementation and results moving forward.  In light of PFS’ past growth and future growth plans, the operational systems that support and manage its volunteer expert force should be increasingly and, eventually, fully formalized. This process should be continued up to the point where PFS is fully staffed with full-time, paid staff carrying out all important operational/administrative duties.  TNS should refine the SGB selection and onboarding process further so as to bring on more SGBs who (1) are already selling in informal food markets or are looking to sell in informal food markets and (2) have existing, strong commercial linkages with SHFs (e.g., via an out-grower scheme) or are looking to establishing such commercial linkages with SHFs. While targeting either of these two types of SGBs is not mandated in SAFE’s SOW, they both appear fruitful areas for expanding SAFE’s impact as part of any follow-on program.  TNS should limit its impact assessment methodology to one measurement activity per year. The impact assessment methodology is an intensive, time-consuming process that would eat up a significant portion of time for both TNS staff and client SGBs.  TNS should implement the impact assessment with all other SGBs missed during the initial September 2015 measurement activity, and for whom impacts may have realistically occurred, so as to create a complete program baseline and to ensure that the interim between the baseline and follow-up measurements allow sufficient time for measurable change to have occurred.  SAFE should avoid adding significantly to the number of SGBs served by food technologists and business advisors until the additional work requirements involved with providing locally-based TA (e.g., via quick wins and standardized charters) are clarified.  SAFE should conduct routine follow-ups with SWT participants, especially those who appear to be good candidates for working directly under SAFE. This might take the form of follow-up phone calls, workshops, visits to production facilities, linkage meetings (such as was done in Zambia with lending institutions), participation in Ask an Expert, and so forth. SAFE may also consider implementing a voucher scheme for follow-up, whereby SWT attendees can redeem vouchers for TA from pre￾identified trainers at a later date.  BFS should make additional alignments to the budget, if feasible, to fund SWTs during the remaining life of the program, and that funding for SWTs should be given a high priority in any follow-on program to SAFE.  SAFE should increase the number, scope, and outreach of short-term focused trainings, such the short-term miller trainings implemented with WFP, for community-based SGBs with smallholder farmer membership, along with follow-up activities, as a strategy for increasing the program’s outreach to and impact on both smallholder farmers and vulnerable populations. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 xix  SAFE should pursue strategies for scaling-up the standardized charters, both as part of the formal chartering process and outside of the formal chartering process, by, for example, increasing the types of projects that qualify for standardized charters and recruiting SGBs that would benefit from them.  SAFE should develop and disseminate additional sector-wide learning resources, making sure to allocate funding to pilot test and market them once they are completed.  SAFE should investigate the feasibility and process for partnering with local technical experts with the selected charter teams as a strategy for increasing SAFE’s impact on the local food-processing sector and for increasing SAFE’s long-term sustainability.  SAFE should prioritize the selection of SGBs that have formal contractual or out-grower relationships with their SHF suppliers, similar to Community Markets for Conservation (COMACO). Consider, moreover, providing complimentary assistance to SGBs on procurement models, or partner with projects that specialize in this area in order to enhance linkages between their assistance and impact on producers. Finally, increase the number, scope, and outreach of short-term focused trainings for community-based SGBs with SHF membership.  If SAFE chooses to work with its client SGBs to sell in informal markets, it will need to emphasize providing marketing assistance to SGBs covering each of the 4P’s of marketing that reflects a clear and in-depth knowledge of the requirements to sell within informal markets.  Potential program improvements to improve gender sensitization and training include co-sponsoring on-site life skills development and other learning activities for lower and mid-level workers; facilitating gender sensitization workshops or trainings for processor staff; integrating the gender audit into the business assessment tool (BAT) and following up with all processors with whom it has done gender audits to share the results, discuss the implications, and plot the way forward. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 1 1. INTRODUCTION 1.1 PROGRAM BACKGROUND Implementation of Solutions for African Food Enterprises (SAFE) involves technology and knowledge transfer from volunteer experts in the four Partners for Food Solutions (PFS) partner companies (General Mills, Cargill, DSM, and Bühler) delivered through the in-country facilitation and guidance of TechnoServe (TNS) food technology specialists and business advisors. According to its website, PFS’s “corporate partnerships are strategically aligned to core business capabilities. For example, Cargill takes the lead on vegetable oils, General Mills on blended flours, Royal DSM on fortification of staple foods, and Bühler on process engineering.” 1 TNS resources and activities are funded by the United States Agency for International Development (USAID) and include implementation for four years between September 2016 and five years for the regional support team (through September 2017).2 It also includes a technical assistance (TA) fund to be used to catalyze program growth in other priority countries. SAFE’s focus is on increasing the availability of quality, safe, and nutritious foods in program countries by enabling food processors in program countries to meet local and regional food safety and quality standards and to be competitive suppliers into local and regional markets, including food aid markets. Operationally, SAFE uses two primary intervention strategies: (1) customized TA provided to individual food processors and (2) sector-wide trainings (SWTs) provided to food processors and other sector stakeholders on selected topics. Under the PFS volunteer model, volunteers typically provide TA from their home countries using distance communications methods such as Skype. This model may also include occasional on-site visits by PFS volunteers or other staff from the four PFS partner companies. TNS local staff plays an integral role in this process by identifying processors and helping to facilitate communications between the two parties, both in terms of arranging meetings and ensuring that the international experts and local business operators are “speaking the same language.” Local TNS business advisors and food technologists themselves also provide on-site TA to SAFE-assisted processors in areas falling under their relevant expertise. Reflecting its strategic focus on small and medium-sized food processors, SAFE refers to the food processors it assists as small and growing businesses (SGBs). SAFE’s work with its SGB clients is organized by charters, which are similar to memorandums of understanding (MOUs) describing the problem being addressed with a set of specific prioritized objectives and an implementation timeframe with clear milestones. SGBs have from one to several charters at different stages of implementation with SAFE. A particular charter may involve anywhere from a single to multiple PFS volunteers. In selecting food processors to participate in the program, SAFE uses the following three selection criteria: (1) the food processor should source from smallholder farmers (SHFs); (2) the food processor should reach, or potentially reach, large numbers of consumers; and (3) the food processor should be producing, or plan to produce, nutritious foods. Since the baseline evaluation, SAFE has made some important changes to its SGB recruitment and selection process (see below) that include placing a greater emphasis on using 1 PFS website: http://www.partnersinfoodsolutions.com/what-we-do 2 SAFE activities in Malawi were originally scheduled to run only through September 2015 but have since been extended for an additional year through September 2016. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 2 its existing networks to identify and recruit processors, conducting a more in-depth due diligence process, along with an intensified focus on client pull (e.g., client commitment and entrepreneurism), and increased client socialization via a familiarization workshop. To date, SAFE has provided technical assistance to 36 food processors (21 in Kenya, 10 in Zambia, and 5 in Malawi). An additional 20 new processors have recently been recruited, bringing the total to the program target of 56, while 11 processors have been graduated. Since its inception, SAFE has also delivered 18 SWTs to more than 600 individuals from 462 companies in technical areas related to such things as food quality, fortification, product development, and business strategy. 1.2 THEORY OF CHANGE Figure 1 presents the SAFE Theory of Change (ToC), which shows the primary outputs, outcomes, and goals towards which SAFE is working. The four primary program outputs at the bottom of the ToC are expected to contribute jointly at the outcome level to strengthened capacity among assisted food processors to produce and market quality, safe, and nutritious foods. This, in turn, is expected to lead at the goal level to an increase in the availability of nutritious foods and a more competitive processing sector. The high-level goal, which is the theoretical culmination of SAFE outputs, outcomes, and goals, is reduced poverty and improved nutritional status among poor and vulnerable households in SAFE program countries. It should be noted, however, that in practical terms (or in terms of program operating strategy), SAFE’s actual scope of work—or results for which it is held accountable and which are thus examined as part of this performance evaluation—only goes as high as increasing the availability of nutritious foods at the goal level. While the increased competitiveness of the food processor sector and the higher-level goals related to poverty reduction and improved nutrition are considered important higher-level effects to which the program aspires, they do not constitute operationalized goals that motivate program operations. This means that although this evaluation investigates questions related to SAFE’s outreach to and impact on food insecure consumers, the purpose for this line of inquiry is more to inform future BFS programming rather than to hold SAFE accountable for its results in this area. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 3 Figure 1: SAFE Theory of Change Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 4 2. EVALUATION OVERVIEW 2.1 EVALUATION PURPOSE BFS contracted Social Impact (SI) to undertake this performance evaluation of the SAFE program. The purpose of the SAFE performance evaluation is to provide BFS with empirical evidence to inform programming and investment decisions under the FtF initiative, including the following:  How well BFS centrally funded/managed programs are implemented and performing;  How the programs are perceived by beneficiaries;  Whether the programs are achieving targeted outcomes in support of FtF objectives; and  What more needs to be done. This performance evaluation examines the extent to which the SAFE program has achieved the outputs, outcomes, and goal of increased nutritional food availability in its ToC. BFS will use the evidence from this evaluation, as well as other analyses, to identify critical gaps in programs in order to map out the way forward for SAFE over the remaining life of the program and to inform future FtF programming decisions in this area. 2.2 EVALUATION QUESTIONS The SAFE performance evaluation seeks to answer the ten evaluation questions (EQs) listed below. Evaluation questions are grouped according to four categories: (1) food-processing sector, (2) markets linkages, (3) gender, and (4) implementation process. Food-Processing Sector EQ1. Does participation in SAFE improve the performance of SAFE-assisted food-processing firms in the following areas: (1) management and technical capacity, (2) volume of nutritious food products produced, (3) sales of nutritious foods, (4) employment, (5) profitability, (6) investment, and (7) marketing of nutritious foods? If so, how? If not, why? What could be done better to improve the program’s impact on these areas of firm performance? What is the likelihood that these improvements will be sustained? EQ2. Have SAFE interventions contributed directly and/or indirectly to changes in knowledge and practices of stakeholders within the food-processing sector as a result of (1) technical assistance, (2) sector-wide trainings, (3) learning activities, and (4) demonstration effects? If so, how? If not, why? What worked best? What did not work well? What could be done better to facilitate changes in knowledge, attitudes, and practices among sector stakeholders? What is the likelihood that these changes will be sustained? EQ3. Have SAFE interventions produced a demonstration effect as measured by the scaling-up of SAFE-supported activities among assisted processors and the replication of the SAFE model among other development organizations or in other locations and countries? What is the likelihood that these demonstration effects will be sustained? Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 5 Market Linkages EQ4. Is participation in SAFE associated with new or strengthened market linkages between food-processing firms and (1) smallholder farmers, (2) wholesale buyers, (3) retail buyers, (4) food aid buyers, (5) lending institutions, and (6) other value chain actors? If so, how? If not, why? What worked well in developing or strengthening these market linkages? What did not work well? What could be done better to develop or strengthen these market linkages?3 Are these market linkages sustainable beyond the program? EQ5. Have SAFE interventions increased the availability of nutritious foods, particularly in markets characterized by higher levels of food insecurity among the consumer population? If so, how? If not, why? What could be done better to improve program impact on food availability, particularly characterized by higher levels of food insecurity among the consumer population? EQ6. To what extent has any increase in the availability of nutritious foods been matched by an increase in access to (and consumption of) nutritious foods, particularly in markets characterized by higher levels of food insecurity among the consumer population? If so, how? If not, why? In particular, what barriers exist to food-insecure households purchasing and consuming nutritious foods? Gender EQ7. Is participation in SAFE associated with increased leadership/management opportunities for women in SAFE￾assisted food-processing firms? What are the opportunities for advancement? What are the constraints? What is most needed to move women into higher-paying positions with increased income? Is training, skills development, or other capacity building accessible to the women who wish to advance? What are the ability and willingness of the food processor enterprises assisted by SAFE to support the advancement of women in their employment? EQ8. To what extent has SAFE led to increased income generation among women employees of SAFE-assisted food-processing firms? If so, how? If not, why? To what extent do these women employees have control over these resources and are the decision makers in their households as to the use of the earned income? 3 Note that here and elsewhere in this document, a market linkage is evidenced by a successfully completed market transaction or transactions. The existence of two or more market entities that have engaged in market transactions in the past but that now are engaging in more mutually beneficial market transactions is evidence of a strengthened market linkage, while the existence of two entities engaging in market transactions for the first time is evidence of a new market linkage. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 6 Implementation Process EQ9. Was the SAFE program implemented as designed? If not, what changes were made and what factors, both internal and external to the program, explain the changes? How did these changes affect program results? EQ10. What might SAFE have done differently to improve program implementation? Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 7 3. METHODOLOGY 3.1 DATA COLLECTION METHODS The SAFE performance evaluation is a mixed-methods evaluation that uses a combination of quantitative and qualitative data collection methods. It involved a baseline data collection round implemented during March–April 2015 and an endline data collection round implemented during March-April 2016. The primary collection methods included: (1) key informant interviews (KIIs), (2) focus group discussions (FGDs), (3) processor surveys, (4) household surveys, (5) farming system research, and (6) secondary data review, primarily consisting of performance information from SAFE’s performance monitoring system. The FGDs and household surveys were implemented in conjunction with the in-depth case studies conducted in catchment areas served by the two SAFE-assisted processors Classic Foods in Kenya and Community Markets for Conservation (COMACO) in Zambia, while the farming system research was implemented with SHFs supplying raw materials to COMACO. The farming systems research was new to the endline evaluation round. Both the two case studies and the farming system research were done to provide information to help the evaluation team answer EQ4 and EQ6. Key Informant Interviews KIIs are in-depth, semi-structured interviews conducted with program stakeholders face-to-face, by telephone, or over Skype. KIIs use a prepared discussion guide but also allow interviewers the flexibility to investigate other lines of inquiry as appropriate. KII participants were selected purposively based on consultations with TNS and the local research partner selected by the evaluation team and on the evaluation team’s familiarity local markets and stakeholders. The primary criteria for selecting participant stakeholders were their familiarity with SAFE and/or their knowledge of relevant issues. The evaluation team conducted KIIs during the endline with representatives from the nine stakeholder groups shown in Table 1. As seen in Table 1, the number of stakeholders interviewed in the endline increased to 103 as compared to 93 in the baseline. Accounting for most of the increase was an increase in the number of SGBs interviewed in each county, plus the addition of financial institutions and subject matter experts to the list of stakeholders interviewed. Subject matter experts were interviewed about their role in the development of the three primary learning resources developed by SAFE since the baseline, including the new product development manual and e-books in business strategy and fortification. Included among the SGBs interviewed during the endline in Kenya were three small hammer millers who participated in the joint SAFE and World Food Programme (WFP) small group hammer mill trainings for small hammer millers and two small food processors who participated in the joint SAFE and WFP batch fortification training in the Kakamega refugee camp. Market actors interviewed included wholesalers and retailers working in the communities covered by the in-depth case studies. In the endline, the wholesaler and retailer KIIs focused on trying to understand how food products are distributed and sold within the informal markets located in the more marginal urban and rural communities where lower-income and food-insecure people shop. It is the evaluation team’s intention that this information (along with the consumer FGDs, see below) will be useful to inform SAFE’s and BFS’ thinking on how a food processor assist program, like SAFE, can be structured so as to increase the access to nutritious foods among poorer and more food insecure populations. KIIs were recorded and the recordings transcribed whenever possible. Interviewers also took notes of each interview and prepared written summaries of their notes. A complete list of KII participants, disaggregated by country and stakeholder group during the endline, is presented in Annex 2: Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 8 Stakeholders Interviewed. Copies of the KII discussion guides can be found in Annex 5: Discussion Guides. Table 1: Key Informant Interviews Conducted during Baseline and Endline Evaluation Rounds Stakeholder Kenya Zambia Malawi Total B E B E B E B E PFS N/A N/A N/A N/A N/A N/A 8 7 TNS 6 10 1 2 2 2 9 14 SGBs 15 23 8 10 4 8 27 41 Trainers 2 0 3 0 3 0 8 0 Institutional buyers 3 2 4 0 3 3 10 5 Sector stakeholders 3 2 5 4 2 0 10 6 Financial institutions 0 3 0 1 0 0 0 4 Subject Matter Experts 0 10 0 0 0 0 0 10 Market actors 10 8 10 8 1 0 21 16 Total 39 58 31 21 15 13 93 103 Focus Groups Discussions FGDs are in-depth, semi-structured group interviews conducted face-to-face with members of selected stakeholder groups. In the endline, FGDs consisted entirely of consumer FGDs, which were group discussions with eight women in each case who are responsible for making household food purchasing decisions in case study catchment areas. The endline consumer FGDs focused less on issues related to attitudes and behaviors about nutrition (as was the case in the baseline) but more on the participants’ shopping behaviors, in terms of where they shopped and why and what factors motivated where they shopped and their purchasing behaviors. As with the wholesaler and retailer KIIs, the intention here was to gain a better understanding of how the informal markets, in which lower-income and food insecure consumers shop, operate so to guide thinking on how to reach these consumers with programs such as SAFE. FGDs used a prepared discussion guide but also allowed interviewers the flexibility to investigate other lines of inquiry as appropriate. Conducting FGDs typically involved a two-person moderation team, including a trained facilitator who moderated the discussion and a research assistant who took notes and assisted the facilitator. FGDs were recorded, and the recordings transcribed whenever possible. Moderators also took notes of each interview and prepared written summaries of their notes. Table 2 shows the distribution of FGDs across countries and stakeholders. Note that the endline evaluation round did not include FGDs with SWT participants or with women employees. Under the budget realignment, SAFE had to incorporate activities in both Ethiopia and Tanzania under the same budget ceiling meaning that had to make cuts elsewhere, so it scaled back on the SWTs. TNS has included a significant scale-up of SWTs in all countries in its proposed budget ceiling increase for the period through September 2017, but is still waiting for feedback from USAID on this proposal. In lieu of the SWTs, SAFE conduct four small group trainings in conjunction with the WFP for 17 small hammer millers in Kenya, including three hammer millers in the Kakamega Refugee Camp, plus it further Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 9 collaborated with the SWG to conduct a group training on batch fortification for two small processors in Kakamega. A sub-set of hammer millers and processors participating in the WFP trainings were included in the SGB KIIs, as discussed above. With regards to the women employee FGDs, the original purpose for conducting these in the baseline was to explore SAFE’s impact on gender roles and women employees within the assisted processors. Given, however, that SAFE had not implemented any gender-related initiatives from the baseline to the endline, the evaluation team (ET) had no reason to believe that anything would have changed in the interim between the two, and thus decided there was little to gain by repeating the women employee FGDs in the endline, particularly in light of budget constraints, which required the evaluation team to scale back the number of KIIs and/or FGDs conducted during the endline. Participants in the consumer FGDs are listed in Annex 3: Consumer Focus Group Discussions. Copies of the FGD discussion guides for each stakeholder group are presented in Annex 5: Discussion Guides. Table 2: Focus Group Discussions Conducted during Baseline and Endline Evaluation Rounds Stakeholder Kenya Zambia Malawi Total B E B E B E B E SWT participants 2 0 1 0 0 0 3 0 Women employees 4 0 4 0 1 0 9 0 Consumers 4 4 3 4 0 0 7 8 Total 10 4 8 4 1 0 19 8 Processor Survey The evaluation team asked each SGB interviewed during the endline to complete a short self-administered survey. Whereas the baseline evaluation also administered surveys to SWT trainers and participants, these were not repeated in the endline given that SAFE did not implement SWTs in Kenya and Zambia in the interim between the baseline and endline evaluations. Table 3 shows the number of stakeholders responding to each of the stakeholder surveys in each of the SAFE program countries in the baseline and endline evaluation rounds. A copy of the processor survey is provided in Annex 6: Processor Survey. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 10 Table 3: Stakeholder Surveys Conducted during Baseline and Endline Evaluation Rounds Stakeholder Kenya Zambia Malawi Total B E B E B E B E Processor survey 15 184 8 10 3 8 26 48 Trainer survey 2 0 3 0 2 0 7 0 Trainee survey 97 0 41 0 0 0 138 0 Total 114 18 52 10 3 8 169 48 Household Survey The household survey is a repeated cross-sectional survey administered to a simple random sample of households located in low-income catchment areas served by two of SAFE’s assisted processors: Classic Foods in Kenya and COMACO in Zambia. Both the case study processors and communities were selected after close consultations with TNS. Households participating in the survey within each sample community were selected using the random walk method. Survey respondents were the persons within those households most responsible for making food purchasing decisions. The household survey questionnaire included questions covering household demographics, household food consumption, attitudes toward and usage of nutritious foods, attitudes towards and usage of the foods, and brands produced by the two case study processors. In addition, the endline survey added a module of questions asking about where respondents shopped and why. Similar to the market actor KIIs and consumer FGDs, the purpose of these questions was to try to understand better how informal markets (where most of the survey respondents shop) operate and what factors drive shopping and purchasing behaviors within those markets. The household survey sample size in Kenya was 428 households in the baseline and 438 households in the endline spread across ten urban settlements in and around Nairobi. The total endline sample size in Zambia was 466 persons living in Eastern Zambia rural communities of Chipata and Katete. The endline sample size was less than the baseline sample size of 657 drawn from the same two rural communities plus a third rural community of Lundazi. Budget constraints in the endline required a reduction in the Zambia sample; after discussions between the SI evaluation team and Infinite Insight, the decision was made to drop Lundazi from the sample owing to its more remote location and thus greater cost. Copies of the two survey questionnaires used in Kenya and Zambia are provided in Annex 4: Household Survey Instrument, while a detailed summary of the survey findings in Kenya and Zambia (including a breakdown of the sampling distribution) are provided in Annex 8: Household Survey Findings, Kenya and Annex 9: Household Findings, Zambia, respectively. Farming Systems Research As mentioned above, both PFS and TNS see facilitating linkages between SGBs and SHFs to generate increased on-farm productivity and income to be an important aspect of SAFE impact. However, rather than pursue this objective directly by working specifically with SHFs, they have elected to pursue it indirectly under the assumption that expanding the capacity of the food-processing sector will result in 4 Not all of the Kenya SGBs interviewed agreed to complete the processor survey. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 11 complimentary strengthening of the sector’s linkages with their SHF suppliers. However, as pointed out in the baseline evaluation report, while sector-wide expansion of processing capacity may result in overall improvements in market opportunities for SHFs, it would be very difficult to attribute specific improvements to SAFE interventions absent specific strategies to accomplish this. While the baseline evaluation did not include the scope to investigate smallholder supplier linkages in depth, the SI team felt that this topic was important enough to add to the endline scope. With constrained resources for the endline evaluation, however, it was not possible to conduct a broad-based investigation of the issue, thus the ET opted to conducted a more limited, though still in-depth, investigation with a single SGB to serve as a case study. One of the SI team members had substantial experience in conducting farming systems research (described below), and the ET felt this approach would work well here in the context of an in-depth case study, which sought to understand the impact certain SGBs can have on smallholder suppliers in terms of increasing their on-farm performance. The team selected COMACO (based in Chipata, Zambia) as the best case study option due to the company’s well-developed out-grower relationship with its smallholder groundnut and soybean suppliers, which is a central part of its operating mission. During March 2016, SI Markets Specialist David Rinck travelled to Chipata in the Eastern Province of Zambia where he spent the week of March 28-April 2 visiting with COMACO staff and their groundnut and soybean out-grower farmers and documenting the nature of their relationships and lessons learned for the endline evaluation. The research consisted of KIIs with COMACO staff and associated stakeholders and 72 illustrative crop surveys conducted through seven FGDs with soybean and groundnut producers in Eastern Province (37 groundnut producers and 35 soybean producers). The FGDs were conducted in three areas of Eastern Province: Chithaza and Leseti Farms (in the Eastern Province Cash Crop Zone) and Kamphasa (in the Luangwa-Kafuwe Valley Zone). The FGDs met at the producer’ communities in sizes ranging between 8-12 participants. Following the FGDs, the Markets Specialist also visited fields with the farmers to see firsthand their crops and gather additional insights. While COMACO is among the relatively few SAFE-assisted SGBs employing an out-grower model, there are additional examples including Lakeshore Agroprocessors, as well as cooperatives such as the Bvumbwe Dairy Cooperative Society and Chinangwa, Mbatata, Roots & Tubers Enterprise Ltd, all in Malawi. The relevancy of this case study is limited to potential impact only in cases where SAFE assistance is strategically targeted to these types of SBGs. Also, it is important to add here that, while COMACO has a more developed relationship with its smallholder suppliers than most of the SGBs SAFE works with, SAFE has had no involvement in establishing this relationship. Therefore the intent of the farming systems research is not to attribute benefits (or lack of benefits) found to SAFE, but rather to use the findings of this research to understand the potential for a processor support program like SAFE to provide downstream benefits to SHFs supplying assisted food processors in the event that this targeting approach is explicitly adopted and integrated into a strategy for processor selection. In-Depth Case Studies As discussed above, SAFE focuses on increasing the availability of nutritious foods in the three SAFE program countries. Moreover, while improving the nutritional status of vulnerable households sits at the top of the program ToC, SAFE is not actively working toward, and is not tasked to achieve, this high-level goal. It is worth noting, however, that this ‘disconnect’ between SAFE’s high-level goal and its more limited mandate is an artifact of how SAFE was designed. SAFE could have been designed differently to pursue this goal, and, perhaps more relevant, future processor support programs can also be designed so as to pursue this goal. With the intention of informing thinking in this direction, the SI ET undertook the two case studies in the baseline so as to understand better the supply and demand characteristics for nutritious Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 12 processed foods in more marginal communities/markets characterized by higher poverty and greater food insecurity. The data collection methods used for the two case studies include key informant interviews with food wholesalers and retailers, consumer FGDs with women responsible for household food purchasing decisions, and the household survey. The endline evaluation repeats the two cases studies, although with a few revisions over the baseline. As mentioned above, these revisions include the following:  Shift in the focus of the wholesaler and retailer KIIs from the demand for and supply of nutritious foods to the channels by which food products are distributed within informal markets.  Shift in the focus in the consumer FGDs from attitudes and behaviors related to nutritious foods to shopping preferences and behaviors, particularly where the respondents shop, why they shop there, and what they buy there (product and packaging).  Addition of a module to the household survey asking about the respondents’ shopping preferences and behaviors.  Reduction in the Zambian sample size (for budgetary reasons) from 657 to 466 by removing Lundazi from the sample. The two SGBs selected for the in-depth case studies were Classic Foods in Kenya and COMACO in Zambia. Classic Foods operates in relatively poor urban settlement areas surrounding Nairobi (e.g., Kibera, Kitengela, Juja, and Ruiru) where it sells corn-soy blend (CSB) porridge and fresh milk under its Classic brand name and fortified maize flour under its Capital brand name to retail chains and small independent retailers. COMACO operates in the Eastern Province in the relatively sparsely populated cities and towns of Chipata, Katete, and Lundazi, where it sells two nutritious food products in retail chains and small independent retailers under the It’s Wild! Brand name: Yummy Soya (CSB) and Peanut Butter.5 3.2 LIMITATIONS OF THE EVALUATION METHODOLOGY The evaluation methodology has a number of limitations that have important implications for the types of findings and conclusions that can be drawn from the evaluation, as discussed below. Attribution The primarily qualitative nature of the evaluation, which in turn entails the absence of comparison measurements and (at times) small sample sizes, means that the evaluation methodology is not able to generate sufficient counterfactual evidence to attribute observed results to SAFE activities. The evaluation methodology seeks to mitigate this limitation through the triangulation of data and by drawing on multiple sources of data to reach informed an informed set of conclusions related to SAFE’s role in contributing to these observed results. Measuring Incremental Change As mentioned above, the evaluation methodology was designed to measure what changes were occurring at SAFE-assisted SGBs and to understand SAFE’s role in these changes; it was not designed to measure the incremental change from the baseline to the endline or how the incremental changes varied from SGB 5 Due to budget constraints, and the absence of a SAFE-assisted processor who met the selection criteria, we did not conduct a third in-depth case study in Malawi. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 13 to SGB or across the three SAFE program countries. The best information on incremental changes, and how they vary across SGBs and countries, is found in the SAFE performance monitoring system, which is referenced in multiple places in this evaluation report. Sampling Owing to time and resource constraints, the ET was not able to interview all SAFE stakeholders, and thus was only able to interview a subset of purposively selected stakeholders. While the evaluation team worked closely with TNS to identify a reasonably representative sample set of stakeholders, these may not give a fully comprehensive picture of program impact, its activities, and its results. Limitations of Case Study Methods The two in-depth case studies and the farming system research with COMACO were intended to provide insights on questions related to the availability and access of nutritious foods among food-insecure households and to the distribution of food products and consumer shopping behaviors within informal markets. The case study SGBS were selected after discussions with TNS and among the SI team members as being potentially good cases from which generalizable lessons might be learned. However, there are limitations to using case studies to draw generalizable lessons learned related to issues such as sample size and context limitations. Indirect Effects The evaluation methodology and timeframe meant that evaluators were obliged to focus primarily on the program’s direct beneficiaries, which are the assisted SGBs. At the same time, SI recognizes that there may be direct or indirect beneficiaries of program activities, including SWT trainees and also smallholder farmers supplying the assisted processors. Aside from the farming systems research done during the endline evaluation, the evaluation methodology did not allow the evaluation to determine what or how substantial these other direct or indirect benefits were. Subjective Measurements A final implication of the qualitative approach used is that the measurement of results largely consists of subjective perceptions of KII and FGD participants. Thus, the evaluation lacks specific quantitative measurements of certain key results. 3.3 EVALUATION TEAM The Social Impact ET consisted of three US-based consultants, including a Team Leader, an Agricultural Markets Specialist, and a Gender Specialist, in addition to a local research partner. Among the US-based consultants, the Team Leader and Gender Specialist are working under contract with SI, and the Agricultural Markets Specialist is working under contract with Management Systems International (MSI). The local research partner is Infinite Insight, is a Nairobi-based market research firm with experience working in multiple countries in East and Southern Africa. The US-based consultants and local research partner shared data collection responsibilities during both evaluation rounds. The former conducted all KIIs with PFS and TNS staff in addition to a sub-set of KIIs with SGBS, institutional buyers, subject matter experts, financial institutions, and other sector stakeholders in Kenya and Zambia, in addition to all KIIs in Malawi. For its part, Infinite Insight conducted the greater share of SGB KIIs in addition to a sub-set of KIIs with all other program stakeholders, all wholesaler/retailer KIIs, all consumer FGDs, and all household surveys. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 14 The following sections present the findings, conclusions, and recommendations from the SAFE endline evaluation round organized by the ten evaluation questions listed above. Recommendations include both SAFE Recommendations, which apply specifically to the SAFE program, and General Recommendations, which apply more broadly to BFS programming. Note that the quotes presented below and elsewhere to demonstrate the evaluation findings are not exhaustive of all quotes made by KII and FGD respondents, but are instead intended to be reasonably representative of the statements made and the common themes that emerged during the KIIs and FGDs. For the sake of brevity, only a limited number of quotes are provided to demonstrate a particular point. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 15 4. EVALUATION FINDINGS EQ1. DOES PARTICIPATION IN SAFE IMPROVE THE PERFORMANCE OF SAFE￾ASSISTED FOOD-PROCESSING FIRMS? Volume, Sales, Profits, and Investment The endline evaluation found little change from the baseline in terms of the benefits SGBs have received, or perceive to have received, from participation in the SAFE program. Overall, participation in SAFE appears to have contributed to improved SGB performance in terms of increased production efficiency and improved quality, which in turn have contributed to an increased number and volume of nutritious food produced, sales, profits, and investment. The processor survey asked the respondent SGBs to rate their improvement in each of the above areas on a scale of 1–3. Table 4 shows their responses in the endline compared to their baseline responses. Overall, the ratings in the baseline and endline are similar across the board, in some cases increasing and in other cases decreasing. In no case does the increase or decrease appear significant. According to the results in Table 4, assisted SGBs responding in the endline reported a significant (or near significant) increase in product quality, some increases in productivity, sales and profits and less increase in the number of product lines and investment. Table 4: Performance Improvements at Assisted Processors’ as a Result of SAFE Assistance 1. Kenya Zambia Malawi Total B E B E B E B E Number of product lines 1.5 1.6 2.6 2.0 2.3 1.4 2.0 1.7 Productivity 1.9 2.0 2.8 2.3 2.5 1.9 2.3 2.1 Sales 1.9 2.1 2.6 2.6 2.3 1.6 2.2 2.1 Profits 1.7 2.1 2.4 2.4 2.2 2.0 2.0 2.2 Product quality 1.9 2.2 2.9 2.9 2.6 2.4 2.4 2.5 Investment 1.4 1.8 1.5 2.0 1.4 1.3 1.4 1.7 Which of the following areas of your business has grown as a result of this assistance: 1=No increase; 2=Some increase; 3=Significant increase Among Kenya SGBs, ratings went up across the board, with the largest increases reported for profits, investment, and product quality. In contrast to Kenya, in Zambia ratings fell significantly for the number of product line and productivity, rose more sharply for investment and remained the same otherwise. Finally, in Malawi the ratings fell in all six cases, showing the largest declines for number of product lines, productivity and sales. It should be noted, however, that since the baseline evaluation, SAFE has onboarded 10 new SGBs in Kenya, eight new SGBs in Zambia, and six new SGBs in Malawi. Thus these SGBs were fairly early into the process and may not have had time in several cases to experience much change in the above areas. Table 5 presents the cumulative results from the SAFE M&E Plan as of March 30, 2016 for indicators measuring the number of firms operating more profitably, value of incremental net profits earned, total and incremental volume of food produced, value of new investments made, and the number of new Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 16 nutritious food products sold. The values for the incremental volume produced and incremental profits earned are taken from the results of SAFE’s impact assessment conducted during September 2015 using its new methodology to estimate its incremental impact on supported SGBs (see below). As seen in Table 5, SAFE has already exceeded its life of program (LOP) targets for two indicators and is on its way to meeting its targets for two others. (Given that SAFE has only recently piloted its new impact assessment methodology, they have yet to develop corresponding performance targets for incremental volume and profits.) Table 5: Performance of Assisted Processors from the SAFE Performance Monitoring System: Cumulative as of March 30, 2016 Indicator Kenya Zambia Malawi Total Target % of Target Number of firms operating more profitably6 11 9 1 217 42 50.0 Incremental volume of processed foods sold by assisted food processors (metric tons)8 5,705 1,915 92 7,712 N/A N/A Value of incremental net profit by assisted food processors (US$)9 1,471,308 2,385,463 136,388 3,993,159 N/A N/A Volume of improved nutritious food products sold by assisted food processors (metric tons)10 5,672 9,833 7,380 22,886 30,000 76.3 Value of new investments by assisted processors (US$)11 4,788,291 369,180 960,349 6,117,820 1,200,000 509.8 Number of nutritious food products sold by assisted food processors12 15 7 4 26 25 104.0 The results in Table 5 are consistent with the findings of the processor KIIs that SAFE assistance has, on balance, helped the supported SGBs improve their performance in terms of production/productivity, sales, profits, and investment. Surprisingly, investment is the one area in which SAFE has most exceeded its 6 Firms (assisted processors) are counted for this indicator for each reporting period they report higher profits than in the previous reporting period. 7 Eleven of the 21 SGBS have reported increased profits in successive years. 8 The ‘incremental volume of processed foods sold’ is the total volume in metric tons of processed foods sold in the current reporting period minus the volume of processed foods sold in the previous reporting period. Note that this refers to all processed foods sold, including, but not limited to, nutritious food products. 9 ‘Incremental net profit’ is the US$ net profit reported during the current reporting period minus the net profit reported during the previous reporting period. The negative value of incremental net profit thus means that the profit reported in one reporting period was less than the net profit reported in the previous reporting period. 10 This is the total (not incremental) volume in metric tons of nutritious food products sold in the relevant reporting period. 11 ‘Investment’ is defined as any use of resources by food processors intended to increase future production output or income, to improve the sustainable use of agriculture-related natural resources (soil, water, etc.), to improve water or land management, etc. This indicator is measured in US$. 12 ‘Nutritious food products’ refer specifically to food products under production by assisted food processors that have undergone enhancement in their nutritional value, their availability of nutrition, their quality of nutrition or any improvements related to a nutritious food product due to SAFE support. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 17 performance target, yet it is also the one area in which supported SGBs consistently rated the lowest in terms of improved performance. As in the baseline evaluation, SGBs participating in the KIIs were able to offer numerous examples as to how the SAFE assistance has improved their performance in the above areas. The following is a small but illustrative sampling of the relevant quotes.  “Customer complaints have gone down, and the customers have more faith in the product, and of course our sales have gone up.”  “It’s better, it has improved. For example, if we look at cleaning the green grams, we used to employ four ladies to clean them, but now we have a machine that cleans so productivity has increased.”  “I can now handle bigger volumes of crops. I used to deal mostly in sorghum, the one you take to breweries, but now I can work on green grams. I was at BIDCO last week to sign a contract of sunflower and soya beans. We have more clients than before.”  “Very successful, especially on the new formulation of the mount chili sauce, which helped us cut down costs by almost half.”  “It has made a good impact on our quality product because we follow that procedure so our quality control has become really good. Our cleanliness and hygiene has become good. Also, our managing of finances in terms of managing day to day operations has become good.”  “I think 90% of them were beneficial and quite successful. I would say we have improved quality of our product.”  “After five months, the processing came to life. It was very vibrant, very on point and the temp was good because every two weeks we could have some progress. There were a lot of details where I felt like an expert.” New Impact Assessment Methodology to Estimate Incremental Volume and Profits As discussed above, SAFE’s M&E Plan includes two indicators that measure SAFE’s incremental impact: the volume of nutritious foods produced and net profits. As defined in its M&E Plan, the full amount of annual changes in these indicators are attributed to SAFE’s impact, regardless of any other factors occurring internal to or external to the SGB (e.g., investments made, nature or size of other product lines, external economic events, etc.) to which SAFE did not provide TA. This means in practice that SAFE “ended up either taking credit for improvements we did not stimulate, or we are forced to explain negative movement in net profits our efforts had nothing to do with.”13 The TNS M&E team in Kenya puts a high priority on the credibility of the performance information it reports to its internal and external stakeholders, and it felt that the information it was reporting related to the above three indicators threatened this credibility. Consequently, the TNS M&E team developed, pilot tested, and is now rolling out a new, rigorous impact assessment methodology to measure the incremental impact of its TA to supported SGBs with regards to volumes of nutritious foods produced and net profits earned. The new impact assessment methodology is one of the primary new initiatives launched by SAFE since the baseline evaluation (see below). 13 Mukabana, Florah: Study of Feasibility of Measuring Financial Benefits Directly Attributable to Project Interventions: Final Report, September 2015, p. 7. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 18 The impact assessment methodology involves the following three activities: 1. Discussions among the SAFE team to develop an understanding of the types of measures needed to make causal linkage between SAFE TA under the different quick win and PFS charters and SGB performance. 2. Review of SGB documents, including financial statements, bank records, audited accounts, management reports, etc. 3. KIIs with members of the SGB management team fleshing out details of charters implemented, changes made in response to SAFE TA, and how these changes have contributed to SGB performance in areas such as production volumes, production costs, revenues, labor, investments and so forth. The KIIs are the heart of the methodology and involve a back-and-forth between the SAFE assessment team—consisting of the food technologist, business advisor, and M&E team member—and the SGB to arrive in an iterative manner to a shared understanding of SAFE’s incremental impact on the areas of interest. At the conclusion of interview, the SGB representative signs a form stating his or her agreement with the results. TNS conducted a pilot test of its impact assessment methodology with three Kenyan SGBs during 2015 and concluded that the methodology produced credible estimates of SAFE’s incremental impact on the outcomes of interest. (SAFE acknowledges that the methodology can only at best estimate SAFE’s incremental impact, but it also argues the estimates, such as they are, are far more accurate measures of program impact than the measures used previously.) TNS subsequently rolled out the methodology to 22 SGBs across Kenya (12), Zambia (9), and Malawi (1) during September 2015. Moving forward, TNS intends to implement its impact assessment methodology with each supported SGB at least once a year and will report the results from this impact assessment to report on incremental volume and profits replacing the previous methodology (see above). An implication of the new methodology is that it will typically produce much different results than the old methodology, thus rendering the values generated by the two methodologies incomparable. To demonstrate, Table 6 shows the values for incremental volume and profits as of September 30, 2015 (reported in Table 5) to the corresponding values in the baseline as of March 30, 2015 for Kenya and Zambia, as reported in the baseline. Looking first at incremental volume, the new methodology has resulted in significantly fewer metric tons produced for both Kenyan and Zambian SGBs indicating that the old methodology was systematically giving SAFE credit for improvements in production volume to which it did not contribute. Turning next to incremental profits, the new methodology produced a net positive increase in incremental profits in Kenya compared to a significant negative increase in the baseline, suggesting that in this case, the old methodology was blaming SAFE for reductions in net profits for which it was not responsible. In Zambia, the value for incremental net profits using the new methodology was very similar to the results using the old methodology. 14 Because the results from the old and new methodologies produce (or can produce) such different results, SAFE will use only the new methodology from this point forward, and will not compare future results to result previous to September 30, 2015. 14 The March 30 and September 30, 2015 values for the two indicators are also not strictly comparable because they do not include all of the same SGBs. Thus, these comparisons are presented for illustrative purposes only to demonstrate how the two methodologies produce different results, both by under estimating and over estimating SAFE’s impact. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 19 Table 6: Incremental Volume and Incremental Profits Calculated Using New Impact Assessment Methodology: Cumulative as of September 2015 Indicator Kenya Zambia Baseline Endline Baseline Endline Incremental volume of processed foods sold by assisted food processors 25,122 5,705 11,871 1,915 Value of incremental net profit by assisted food processors -5,143,511 1,471,308 2,352,643 2,385,463 In addition to generating more accurate, and thus more credible, estimates of program impact, another motivation for implementing the new impact assessment methodology, according to TNS, was to help foster an improved relationship between the SAFE team and the SGB by, for example, involving the two in a mutual exploration and by demonstrating more explicitly how the SGB has benefited from SAFE assistance. It was TNS’ hope, moreover, that by engaging with the SGB in this process, and by demonstrating how it has benefitted from SAFE assistance, the SGB would perceive greater value to the data collection process, in contrast to the baseline findings in which numerous SGBs complained about the burdensome reporting requirements and intrusive collection methods of the previous data collection system, from which they further perceived no value-added for themselves. During the KIIs with the supported SGBs, the evaluation team probed to determine whether the SGBs saw the impact assessment process in the favorable light as assumed by TNS. It turns out that, consistent with expectations, the SGBs did, on balance, express favorable attitudes towards the impact assessment process, with several mentioning that they saw the value to it, notwithstanding its time demands, whether because it helped them understand better how they were progressing or because it helped both parties identify gaps that needed to be addressed.  “It helped us to see how we can continue assessing ourselves, because the bottom line is the bottom line, it’s very important.”  “It was a moment of reflection. We were able to look back and see where we were before they came in; we came to know the miles that we had made due to the coming in of TechnoServe”  “In fact it was through that same interview that I noticed something. They posed the question relating to the soup, and we realized we were almost done, but we had not started on the porridge. He was very concerned, and he said they would pick it up. So the next thing that I got was a communication was to say that they were reconvening to start on the porridge. I figured that was because of the conversation we had.  “From this exercise, which was done by people from TechnoServe, we learnt the progress we had made in terms of sales. Our sales had grown in terms of improving our business operations; that is how we serve the customer. We also learnt to some areas that we need to do further in order to improve our company; like what we can do in order to grow the sales.”  “It was useful, although it is time consuming but it is useful at least it is positive.” Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 20 The processor survey asked those SGBs who had participated in the impact assessment how useful they found it to their organization using a 4-point scale. As seen in Table 7, respondents in all three countries said, on average, that the impact assessment was ‘useful’ to their organization. Table 7: Performance of Assisted Processors from the SAFE Performance Monitoring System: Cumulative as of March 30, 2016 Kenya Zambia Malawi Total 3.2 3.0 3.2 3.1 Overall, how useful was the recent SAFE impact assessment to your company: 1=Not at all useful; 2=Not very useful; 3=Somewhat useful; 4=Very useful A number of SGBs saw sufficient value in the impact assessment, that they felt it should be done repeated two or more times per year.  “I think regularly, about once a year or quarterly. Quarterly would help because you would be able to remember something. Right now I’m fishing for answers because it happened a year ago. If it was quarterly, I would be able to say we are building up.”  “I would like it repeated twice a year, and the importance of having this exercise is that it keeps you on your toes to look on your deliverable so that next time you are accountable to the goals that you have already set for yourself.”  “I think you should do it at least twice a year.”  “The hygiene one quarterly would be good, because it gives us a quarter to keep working and improving, so enough time to get progress, and it seems like a good enough time. One month is a bit too short, two months is a bit tight, three just sounds about right for many things but more than that six months or once a year there’s no point, six months is on the verge of no point also.” Not all SGBS, however, had a positive opinion about the impact assessment process. Complaints centered around two issues, lack of perceived value and lack of feedback.  “They have gotten the information, and they have gone, then what value does it have to us dishing out information - we would say its zero value. If it does not increase sales, promote our production or anything, it’s basically just dishing out information to them, and we find it not much beneficial to us.”  “No feedback; one year and no feedback. In business you take it like someone did not see any sense in what we discussed. Maybe they just threw the information away.”  “It is a bother in terms of collection of data.”  “It is useful but what they never did is give us the feedback of that audit. What I was talking about the assessment and all that is there should be feedback after the assessment. Maybe there are areas that we need to improve on.” Employment Table 8 shows the number of FTE jobs created as a result of SAFE assistance, as estimated by SAFE, divided into male and female FTEs. These results confirm responses by the interviewed SGBs, many of whom indicated that SAFE support has been instrumental in allowing them to hire additional workers, both male and female. SAFE has comfortably exceeded its LOP targets for creating new male and, particularly, female FTEs, although as learned from the processor interviews, the female FTEs created reported in Table 8 include a much larger share of part-time workers than the male FTEs created. For the most part, females continue Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 21 to be hired more so for basic labor roles, while males, although still hired largely for basic labor roles, are far more likely still be to be hired for higher-level and management positions. Table 8: Performance of Assisted Processors from the SAFE Performance Monitoring System: Cumulative as of March 30, 2016 Indicator Kenya Zambia Malawi Total Target % of Target Number of male FTE jobs created at assisted processors15 80 24 25 129 120 107.5 Number of female FTE jobs created at assisted processors 89 8 15 112 55 203.6 Total 169 32 40 241 175 137.7 Conclusions SAFE appears to have generated increases in production/productivity, volume, sales, profits, investment, and employment. While these improvements are not universal across all assisted SGBs, the large majority of those SGBs that have adopted recommendations made by local TNS staff and/or PFS volunteers have experienced a number of benefits in terms of improved performance in one or more of the preceding areas. In terms of the sustainability of these benefits, as in the baseline, it is too early to reach informed judgments about the long-term sustainability of the above performance improvements. However, those supported SGBs who reported improved performance in the baseline were as likely to continue reporting improved performance in the endline indicating that the benefits of SAE assistance have, for the most part, been sustained, at least over the 12 month period between the baseline and endline evaluations. That the large majority of SGBs interviewed in the baseline and/or endline reported improvements in how they do business as a result of SAFE assistance (e.g., production lines, product formulation, marketing strategy, accounting and financial controls, etc.) suggest the realized improvements are not just a result of improved external circumstances or other fortuitous events but instead are, to a large degree, the result of internal improvements made that were a direct product of SAFE assistance. Thus the widespread evidence of internal, structural improvements among the supported SGBs bode well for the long-term sustainability of the benefits received, all else equal. The impact assessment methodology developed and implemented by TNS is an innovative approach to measure SAFE’s impact on incremental volume and profits. While this methodology includes an unknown, and likely significant, amount of measurement error, it is almost certainly a more accurate and credible approach than the previous methodology of simply counting the net change in the relevant indicators and attributing the entire change to SAFE assistance. A member of the SI evaluation team accompanied TNS on an impact assessment visit during the endline evaluation to observe the process. Overall, he found the process to be well-conceived and reasonably 15 Full-time employment (FTE) is calculated in the SAFE performance monitoring system based on a 40-hour workweek and a full, 230-day (11-month) work year. A part-time job would be converted into FTE terms by the following formula: FTE = (hours per week /40) x weeks/11 or (hours per week /40) x days/230. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 22 rigorous. The back and forth discussion among the TNS assessors and the SGB appears to be particularly useful in pinpointing areas in which SAFE had contributed to changes within the SGB, and it offered both parties an opportunity for reflection of what had been accomplished and what remained to be accomplished. The KII findings, on balance, corroborate TNS’ claim that the assessment process is appreciated by the SGBs, and that they see the value to it. It is too early, however, to reach a conclusion as to whether it also serves to improve TNS’ relationship with the SGBs, as TNS claims. There are, however, caveats to the above conclusions. The first caveat is that the impact assessment methodology is a time-consuming, intensive process that, to be done correctly, requires a least a couple hours of the SGB’s time. This raises two risks. The first risk relates to TNS’ internal capacity to carry on the impact assessments year after year with an expanding number of SGBs. The second risk relates to potential interview fatigue at the supported SGBs. Although most SGBs interviewed did not find the impact assessment to be an undue burden the first time around, there is no guarantee that they will continue to feel that way after repeating the process a few more times. There were enough SGBs who did find the process to be unduly time consuming this time (although a minority) that this possibility remains a legitimate concern. A combination of both these factors may make it difficult for TNS to achieve comprehensive coverage of its supported SGBs in the future. Nonetheless, near comprehensive coverage of SGBs is still likely to produce more accurate estimates of SAFE impact that the previous method. The key to the success of this methodology, as TNS rightly acknowledges, is that the SGBs perceive a value to it, unlike their largely negative attitudes toward the previous method of data collection. The SGBs’ responses in the endline offer a reason to be optimistic about this outcome, but, again, a sufficient number of SGBs in the endline failed to see a benefit to the impact assessment that there is also a good reason to remain vigilant that the SGBs perceive value to the process and to take actions to achieve this outcome. The complaints by some SGBs that they received no feedback after the assessment points to one possible strategy for increasing the perceived value of the methodology. Recommendations Recommendations for improving the impact of SAFE assistance on assisted processors’ performance are captured under EQ10, which asks what SAFE might have done differently to improve program implementation and results. SAFE Recommendations  TNS should continue to implement and refine the methodology, with an increasing focus on finding ways to streamline and improve it so as to (1) increase its accuracy and (2) lessen the likelihood that at some point in the future it will be viewed more negatively (e.g., as a time burden) than it was during its first round of implementation. In saying this, we are cognizant that it may take several iterations of implementation, and thus several years, to get to this point. This implies, moreover, that BFS will require patience to allow TNS to work through these details. Allowing the process to proceed in this way may produce, for a time, incomparable time-series data on what are key performance indicators, but in the long-run, it will produce more credible information on program impact.  TNS should consider extending the impact assessment methodology to measure other key program outcomes, wherever appropriate, including, for example, employment and investment. The methodological approach is by no means limited to volume and profits but can be extended to any range of outcomes for which BFS wants to measure incremental changes over time. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 23  TNS should avoid over-using the new impact assessment methodology. We believe that administering the methodology once per year at a set time is sufficient and a much more replicable and scalable approach. Despite comments among interviewed SGBs that they would not mind participating in the assessment more than once per year, we are not confident that this attitude would be sustained given repeated administration of the methodology over the course of a year or two.  TNS should provide feedback to the participating SGBs after the conclusion of the impact assessment methodology to include a summary of the findings and any subsequent recommendations or final observations. This may be done in writing using a standardized form and, if time and resources allow it, a follow-up visit to provide feedback, perhaps piggybacking on a routine visit made by the local TNS food technologist or business advisor. General Recommendations  TNS’s approach to assessing the impact of SAFE assistance is a significant improvement over the previous outcome ‘counting’ approach in its MEP, both in terms of both rigor and credibility. While such an approach has its flaws, including inherent measurement and attribution error and significant time demands, the degree of measurement and attribution error is almost undoubtedly far less than the previous approach. The time additional demands are an important downside of this approach, but there are ways to manage these demands, and as programs move up the learning curve, they should fall. This has general relevance for other BFS programs, which similarly are in a position to measure and attribute outcomes to program interventions. There exists ample room to improve the rigor and credibility of outcome measurement methodologies across BFS-funded programs. Doing so will involve tradeoffs in terms of cost, time and measurement priorities, but it is recommended to BFS to encourage and facilitate experimentation with such approaches. Further, we recommend that BFS monitor TNS’ impact assessment methodology closely to determine what lessons can be learned from it for application in other programs. EQ2. HAVE SAFE INTERVENTIONS CONTRIBUTED DIRECTLY AND INDIRECTLY TO CHANGES IN KNOWLEDGE AND PRACTICES OF STAKEHOLDERS WITHIN THE FOOD-PROCESSING SECTOR AS A RESULT OF TECHNICAL ASSISTANCE, SECTOR WIDE TRAININGS, LEARNING ACTIVITIES, OR DEMONSTRATION EFFECTS? The endline processor KIIs revealed a large number and wide variety of changes in knowledge and practices among SAFE-supported SGBs. The large majority of SGBs cited at least one, and many several, examples of changes they made to their production, marketing, financial, etc. practices as a result of SAFE assistance. These findings are similar to the baseline, with the primary difference being the larger number of SGBs interviewed in the endline where we found evidence of such changes occurring. The following quotes are illustrative of the types of improvements noted by interviewed SGBs.  “It’s better, it has improved. For example if we look at cleaning the green grams, we used to employ four ladies to clean them but now we have a machine that cleans so productivity has increased.”  “The one we are implementing with them right now is the one for the cooling system. They are also helping us in improving the quality of mala (fermented milk).”  “There is business development, they taught my accountant, and now we know how to keep our records well, how to do reconciliation using a computer. We used to do it manually.”  “More customers are purchasing from us. They have told us that our mala has improved so much; the quality is good, and they have seen the volume is increasing. We started with 25 liters, and they have gone now to 50 liters and still improving.” Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 24  “There were issues that we addressed here in terms of food safety and also training on food hygiene that were done with them and we made other improvements in terms of the structure of this place.”  “Number one we implemented the new labels, and number two on the mango bar we managed to implement that one. We also improved the packaging for our dried products and made improvements in the sauce line, how the process of all the sauce goes.”  “We have been working on the formulation of the coy soy blend porridge, which we have just done successfully.”  “It has made a good impact on our quality product because we follow that procedure so our quality control has become very good.”  “We have been working on the formulation of some of the others like corn soy blend with PFS, which we have just done successfully.” When asked to rate the amount of change they had made in different areas as a result of SAFE assistance using a 4-point scale, SGBs interviewed in the baseline gave the answers shown in Table 9. With only two exceptions, the SGBs either gave higher ratings in the endline or ratings that were only marginally lower than those given in the baseline. Again is should be remembered that the endline results included numerous SGBs onboarded since the baseline, meaning they had less time to make changes than other SGBS. In Kenya, the ratings rose in case indicating on average ‘some changes’ made. Among Zambian SGBs, ratings fell for changes in production methods and technical capacity but rose for changes in product quality and management capacity indicating ‘some change for production methods, product quality, and technical capacity, and significant change for product quality. Finally, among the SGBs in Malawi, the average rating rose only for changes in management capacity but remained at or near 3 in all cases again indicating ‘some change.’ Table 9: Changes Made in Assisted Processors’ Business as a Result of SAFE Assistance Kenya Zambia Malawi Total B E B E B E B E Production methods 2.5 3.1 3.6 3.1 3.2 3.1 2.9 3.1 Product quality 2.5 3.1 3.7 3.9 3.3 3.1 3.0 3.4 Management capacity 2.9 2.9 3.2 3.3 2.9 3.0 3.0 3.1 Technical capacity 2.5 2.7 3.5 2.7 3.2 3.1 2.9 2.9 How much change have you made in your business as a result of the assistance you received: 1=No change; 2=A little change; 3=Some change; 4=Significant change In support of the results in Table 9, Table 10 presents the cumulative results from the SAFE performance monitoring system measuring the number of assisted processors applying new management practices and the number of nutritious food products developed for Kenya and Zambia up through March 30, 2016. In both cases, performance has already exceeded the LOP targets. If we compare the number of processors applying new management practices to the number of processors that have received SAFE assistance since program inception, the percentage success rate is 55.6%, again indicative of fairly widespread adoption of new practices across the supported SGBs. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 25 Table 10: Performance of Assisted Processors from the SAFE Performance Monitoring System: Cumulative as of March 30, 2016 Indicator Kenya Zambia Malawi Total Target % of Target Number of processors applying new management practices 22 11 9 42 35 120 Number of improved nutritious food products developed 8 9 4 21 20 105.0 Number of processors certified 6 2 2 10 6 166.7 A particularly useful evidence of new practice adoption occurs when SGBs are able to qualify for certification from government or other industry certification entities. As seen in Table X, SAFE has assisted 10 SGBs to achieve certification, exceeding its LOP target by 67%. The numbers in Table X have since risen, or will soon rise, as a result of SAFE’s work with its Malawian SGBs to achieve the Malawi Bureau of Standards MS21 standard (see below). The following quotes give a flavor to what SAFE has been able to accomplish in this regard.  “It has enabled us to get the certification of our products through the Kenya Bureau of Standards (KEBS). With this in place, we can be able to sell our products in the supermarket. There is improvement on service delivery to our customers in that we have fewer cases of customers returning spoilt milk or complaining maybe the milk is smelling.”  “One of the recommendations was on trainings of our farmers on good agricultural practices with regard to the standards of the UN, which are important in terms of for us to get into the European market.”  “The one about using good manufacturing practices was a good recommendation because if you don’t practices it, your products may not be approved by the KEBS, UNICEF or World Food Programme.”  “They helped us a lot because we get quarterly inspections from the Zambian Bureau of Standards and because of the interventions from PFS and TechnoServe we always pass. There are many compliments from the Bureau of Standards when they inspect because of what we learned from the program.”  “Now we are working on MS21, but we are waiting for machines to arrive from China in order to complete this process. We expect to obtain the certification when the machines arrive.”  “TNS required us to obtain certification from MBS before continuing to assist us. After the initial assessment, we set a target of three months to obtain MS21.”  “We had HACCP certification under Nutraset, and when we separated, GM helped us continue the certification.” Conclusions SAFE technical assistance has been effective in improving the knowledge and practices among assisted SGBs. Assisted SGBs reported, on average ‘some change’ in production methods, product quality, management capacity and technical capacity. Notwithstanding, nearly every SGB interviewed for the endline, and who was far enough along in the process for changes to have occurred, could cite specific examples about how SAFE technical assistance had contributed to improvements in one or more of the above areas. For the large part, the changes cited were specific and the SGBs could tie them directly back to SAFE assistance. Thus to the extent such changes are occurring, they are more likely than not to be a direct result of SAFE assistance. In the baseline, SWT participants gave high to very high marks to all aspects of the SWTs and their feedback suggested that SWTs were an underexploited tool that could be an effective mechanism for Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 26 facilitating more broad-based changes in SGB practices beyond those receiving direct TA from SAFE. Unfortunately, due to a budget realignment related to SAFE’s expansion into Ethiopia and Tanzania, SAFE scaled-back the SWTs in the three program countries under study here after the baseline evaluation. In lieu of SWTs, SAFE delivered a batch fortification training for micro millers operating in the Kakuma refugee camp followed by four hammer milling trainings for small food processors working with the WFP. While we believe that the potential for the SWTs remains the same as in the baseline, we have no additional information to confirm or build on that conclusion in the endline. At this point, it is too early to reach definitive judgments about the sustainability of the above changes in knowledge and practices. In several cases, however, the changes at supported SGBs have persisted since the baseline. In many cases, moreover, the changes represent significant changes to basic production or business practices at the supported SGBs that have required in turn significant changes to internal operations, production technologies, plant layout, software, etc. It would thus be difficult in many of these cases to backslide to former practice, so in this respect, we would expect many of these changes to be sustainable over time, although this admittedly tells us little about how well the SGBs are implementing the relevant practices. In any case, after two rounds of talking with the assisted SGBs, we see reason to be optimistic that in many, albeit unknown, cases, the changes occurring have set the SGBs on an improved long-term trajectory than before SAFE assistance. Recommendations Recommendations for improving the impact of SAFE assistance on changes made at assisted SGBs are captured under EQ10, which asks what SAFE might have done differently to improve program implementation and results. SAFE Recommendations  While we understand the reason for scaling-back on the SWTs over the last 12 months, we wish to re-emphasize the recommendation made in the baseline report to scale-up the SWTs, which requires in turn recommitting money for their implementation, preferably during the remaining life of the program (LOP) or, at the very least, in any SAFE extension. EQ3. HAVE SAFE INTERVENTIONS PRODUCED A DEMONSTRATION EFFECT AS MEASURED BY THE SCALING-UP OF SAFE-SUPPORTED ACTIVITIES AMONG ASSISTED PROCESSORS AND THE REPLICATION OF THE SAFE MODEL AMONG OTHER DEVELOPMENT ORGANIZATIONS OR IN OTHER LOCATIONS AND COUNTRIES? WHAT IS THE LIKELIHOOD THAT THESE DEMONSTRATION EFFECTS WILL BE SUSTAINED? We did not find any evidence that SAFE has produced a significant demonstration effect to this point. In the baseline we found that “if there is an aspect of the SAFE approach that has the greatest potential for creating demonstration effects, it is the SWTs.” As discussed above, however, the SWTs were scaled￾back over the 12 months since the baseline evaluation due to a budget realignment to accommodate the addition of Tanzania and Ethiopia under the SAFE umbrella. In this light, we feel that it is appropriate to restate our findings, conclusions and recommendations from the baseline with regards to the SWTs. According to key informants at TNS, the primary benefit of the SWTs is the knowledge acquired by participants that forms the basis for future action on a larger scale than possible from the one-on-one TA provided, particularly in terms of opening sector actors’ eyes to ideas or options they would have Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 27 otherwise not considered and in terms of alerting them to sources of information and other resources that they can access. The following are illustrative approaches for scaling-up the SWTs:  Offer more SWTs, including more SWTs on diverse topics or more SWTs on topics of particular interest or importance.  Offer a wider variety of SWTs more finely tuned to the participants’ background and level of experience. Examples include SWTs on advanced technical or business management topics for more advanced participants and SWTs on basic technical or business management topics for less experienced participants.  Offer virtual SWTs using online technologies, to the extent possible. One opportunity to scale-up the SWTs we recommended in the baseline is to work with local universities or other training institutions to integrate the SWT content and training materials into their own curriculum or training programs. It turns out that SAFE has adopted this course of action since the baseline evaluation. As part of its sustainability strategy, the SAFE Kenya team worked on developing a set of learning manuals since the baseline, which it will make available to the public and, particularly, to learning institutions teaching courses on topics of relevance to the food processing sector. These learning manuals consisted of a New Product Development (NDP) Manual and e-books on Fortification and Business Strategy. The e-books are intended to be on-line learning manuals in a format that encourages wide distribution to learning institutions and other users. The NPD Manual is not currently formatted for on￾line distribution, although SAFE would also like to convert it into an e-book if resources allow. The content for these three learning manuals came from SWTs, which SAFE had previously conducted on each topic. At the conclusion of each SWT, SAFE solicited feedback on the content of the SWT and, using this feedback along with the original SWT materials hired consultants to develop the manual/e-book content. Prior to finalizing each learning resource, moreover, SAFE engaged a number of local subject experts to review and comment on the manual/e-book content wrapping up in the end with a workshop during which the contents of the manual/e-book were shared with industry stakeholders at a post￾development launch workshop. In preparing and reviewing the three learning manuals, SAFE engaged a large and diverse number of Kenyan stakeholders, including, for example, the Ministry of Health, Ministry of Industrial Production in the Kenya Bureau of Standards, University of Nairobi and the Kenya Nutritionist and Dietary Institute for the e-book on nutrition; United States International University, Strathmore Business School, and University of Nairobi Business School for the e-book on business strategy; University of Nairobi, Kenyan Industrial Property Institute, Kenya Bureau of Standards, Kenya Industrial Research and Development Institute and Kiambu Institute for the NPD manual. In selecting stakeholders to work with it on the learning manuals, SAFE strategically focused on (1) government agencies involved in the relevant topics and (2) learning institutions where SGB staff might go for training. Each of the three learning manuals is targeted specifically at a practitioner audience using case studies, along with examples to highlight primary learning points and tools to implement them. Although the manuals are intended for general use, each of the concepts covered is relevant to the food processing sector, while the e-books include a module on food processing. Evaluators spoke with a number of the subject matter experts who participated either in the drafting or review of each of the three learning manuals. These subject matter experts were unanimous in recognizing that the purpose of the learning manuals was to provide practical, hands-on guidance and tools for practitioners working in the food processing sector. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 28  “The purpose was to furnish them with the knowledge of creating a thriving business and the role of product development to solve the problems that they have, that was the first thing. The second one is how to go about it. Basically, those are the main two components, how you do it. Thirdly the other statutory services that is available to help you do this so that you are not alone.”  “The explanation I got was that they want to have these materials readily available that can be accessed by anyone easily and can be reference material by all industries or whoever wants to know about fortification.”  In line with this purpose, one respondent saw this as an opportunity to take his current course materials and update them by making them more practical-oriented:  “I borrowed the majority from the course content from what I teach in our department of food science and technology for 3rd years. So I already had the theoretical script the background and everything, all the details. But I think with the appointment by TechnoServe I took that as an opportunity to make it more hands on. At that particular point this course was more theoretical but now with the factor that our trainees were the actors in the field, the SMEs…make it more hands on which I think did a wonderful thing to improve and we came up with this manual.” The subject matter experts were split, however, as to how practical the final product was. With regards to the business strategy e-book, some respondents felt that the final product was an effective hands-on manual that covered a wide range of important topics for small business owners. Moreover, as an e-book, it should be easy to revise.  “Right now, I think the book is quite comprehensive, so I would say there are changes that are happening every day and the future is quite, how do we call it… for the lack of word let me say… dynamic. It is actually evolving, and technology is also evolving so there is need to actually have more input in the book. And the fact that it is an e-book it is actually easy to put in some changes as they come but for now I think it is… based on the original copy it is quite extensive.” It is loaded with such things, the diagrams, illustrations and pictures, and in fact we also have a practical section, particularly in the ideation process.”  “Let me tell you, I don’t know how it’s going to be distributed, but I can tell you that all the SMEs and start-up companies can benefit from this.”  “The e-book simple and easy for a layman to understand. This book will help, as it covers everything an enterprise needs to grow.”  “The manual is applied, not just theoretical.” Still, others were not sure how easy they will be in the end for ordinary users to understand:  “I think it is more on the theoretical side.”  “It is a good book, but it needs to be simplified as much as possible so that the reader who is out there who will be using that book knows exactly what is there.”  “It is practical for people of a certain educational level, people who have access to a computer and you will find in the mills there are not so many people who have access to some of these things.”  “I would say this course was well suited for people who have at least gone to the university or those who have gone through technical diploma training so that it is post-secondary training which would be diploma level, degree and even beyond.”  “You will have to completely tone down and yet you don’t want to. So for example we could use the same manual but selectively to train the Juakali people in Gikomba, probably they just have primary education so how do you tone it down.” There was, however, general agreement among the subject matter experts that the learning manuals had broad relevance outside of the food processing sector: Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 29  “It can extend to other products and then if there were SME food producers you can also expand and include those that want to start. So those scattered companies and those who are thinking of coming up, the start-up companies so that you can make them to appreciate the importance of this”  “At that time we were talking about food processing, but like for me it was not just for food manufacture, it could extend to any other companies because it is important even if you are manufacturing some equipment.”  “It is not that only for the SMEs but even for the government policy they can use this.”  “As I said, this principle is not just in the food. This is what I would say most of the bulk of the information is basic, generic and then exemplified using food because that’s where there is a lot of new products. Even cosmetics, shampoos, and pharmaceuticals.” By engaging multiple stakeholders from multiple stakeholder groups in the manual development and review process, SAFE aimed to create a built-in constituency for the learning manuals, which would then help ensure their dissemination and use, particularly in the absence of a formal marketing strategy. The subject matter experts participating in the KIIs had varying perspectives on the effectiveness of SAFE’s stakeholder engagement strategy, most of it positive, although with some caveats. For example, a lead consultant involved in developing one of the e-books felt that there was a good deal of collaboration throughout the process, and he was particularly grateful to TNS for bringing on board experienced practitioners and other experts. In particular, he thinks that the input from international food processors added important dimensions to the final product:  “So in other words, they had companies and corporations who are involved in food processing and development that took part and also provided their input. In fact, several times even as we perfected this particular manual, we were engaged in Skyping their experts in the US and all that.” A contributing consultant for another learning manual also found the collaboration useful, even if it did not always function smoothly:  “So when we were required to come together and work on the project, the collaboration is good because everyone of us has a piece to contribute to that, and in this case, if we sit together and agree this is what you should do, then I think that is more collaboration that going out and saying, ‘I want you do this’ or TechnoServe is going out on their own and saying, ‘I want to do this.’ So yeah, I think the collaboration most of the time is planned and well executed, but it needs to be better planned and coordinated.” Another consultant providing feedback on one of the e-books, and who was part of one of several three￾person teams organized by SAFE to review the e-book, found the collaboration/consultation process to be very productive, not only in terms of the internal collaboration with her fellow team members but also with the lead consultant drafting the e-book and with TNS staff. She reported “lots of communication back and forth with my team and with TechnoServe by email and phone. TNS had a good communication flow, they followed up and reminded about deliverables, they knew we were really busy people.” On balance, she found the process “very practical, we really argued as consultants and came up with lots of input.” The lead consultant, moreover, proved to be very open to their feedback, “he was very responsive, he did not argue, he did not get defensive, he really tried. In the end, he made the changes we recommended.” The same consultant also praised TNS’ role in this process, particularly the fact that TNS stayed in the background and let the subject matter experts do the work, “TNS did not participate much. It organized logistics, hotel, meals, etc. and facilitated, but it remained low key and allowed the consultants to run show … If TNS had been seen to be imposing, it would have lost many. They allowed us to work through the issues to develop a good manual.” Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 30 Others, however, reported a less positive experience in terms of collaboration:  “But the truth of the matter is that the terms of engagement, earlier it was that we were going to have, I think, four or so contacts or meetings, but they called us for one.”  “I think personally when you do an e-book like this you should try to have one or two workshops taking place after a certain time. Bring people together and say ‘OK what can be done?’ ‘What works and where?’ ‘Do we have to deepen it?’ and have all the parties put their input on the table so you can discuss.” Of course, the three learning manuals will contribute to SAFE’s sustainability only to the extent that they achieve reasonably wide dissemination and usage. Toward this end, SAFE has achieved some success to date. The on-line magazine, Food Business Africa (FBA), has agreed to host the two e-books. (FBA also provides a print and digital version.) FBA is a publication of Food World Media and covers the entire sub￾Saharan region with news and information related to the food and beverage processing sector. According to its website, FBA’s target market includes CEOs, General Managers, Quality Assurance and Research and Development Managers, Production Managers, Engineering Managers, Supply Chain Managers, Procurement Managers, Marketing/Brand Managers and other professionals that are tasked with decision making in their roles in the food and beverage manufacturing sector. The Kenyan MOH has adopted the fortification e-book for its own use such that anyone receiving training on fortification through the ministry will need to use this e-book. The Hospitality Department at the Kiambu Institute of Science and Technology – a denominational technical industrial, vocational and entrepreneurship training institute under the Ministry of Higher Education Science and Technology – has adopted the NDP manual as one of its required texts. Unfortunately, SAFE’s funding to develop the three learning manuals did not include money for post￾development pilot testing and marketing. When asked about the possibility of conducting pilot tests of the manuals, SAFE insisted that they had been pilot tested via participants’ feedback at the relevant SWTs. Although the manuals were designed to be user friendly and accessible to wide practitioner audiences, (as noted above) a number of the subject experts still felt that they were still at too high a level to be useful for non-technical audiences, suggesting the need for further pilot testing and refinement beyond that done to date. As for post-development marketing, once SAFE completed the manuals and shared with the industry at the post-development launch workshops, it doesn’t have the resources to go beyond this. Conclusions We can find no evidence that SAFE has produced any significant demonstration effects to date. To the extent that SAFE does have a significant demonstration effect in the future, the SWTs offer perhaps the best mechanisms for achieving it. There exist a variety of options to scale-up the SWTs—by increasing the number, scope, diversity and outreach of SWTs or by offering virtual SWTs using online technologies. SAFE might also conduct follow up trainings with additional assistance to the participating stakeholders, possibly via a voucher scheme as described above. Another option, which SAFE has adopted, is to collaborate with universities or training institutions to integrate SWT material into curriculum offerings. Toward this end, SAFE has engaged in an extensive and, on balance, effective participatory process to develop three learning manuals on topics of significant interest to the Kenyan food processing sector: new product development manual and e-books business strategy and fortification. The content for the learning manuals were developed and vetted in SAFE’s sector wide trainings on the same topics. In creating the learning manuals, SAFE engaged consultants (subject matter experts) representing a wide range of stakeholders and stakeholder groups who both contributed content and reviewed draft versions. While SAFE played the key role in organizing and overseeing the process and facilitating collaboration between the various actors in developing the three manuals, it granted the consultants wide latitude in developing the manual content. The new product Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 31 development manual has been completed and is available for public use and has, so far been integrated into the curriculum of at least one educational institution. The e-books on business strategy and fortification will be made available shortly at the Food Business Africa website. Subject matter experts who worked on the manuals generally agree that they provide a fairly comprehensive coverage of key topics and are reasonably practical and user-friendly. They disagree, however, as to just how practical and user-friendly they are, with some saying that they are accessible to wide audiences, including less advanced users, and others saying that they are still too theoretical and accessible only to more advanced users. Overall, the evaluators find that the process SAFE used to develop the three learning manuals was well￾conceived and effective. Developing such learning tools is a potentially effective way both to expand and enhance SAFE’s impact on the food processing sector over time. The absence of a post-development marketing plan, however, is a significant weakness in the process that threatens the achievement of the above objectives. Recommendations SAFE Recommendations  We recommend scaling up the SWTs, possibly using one of the approaches described above or through another approach. Any program extension should include adequate funding to scale-up the SWTs and to continue working to integrate SWT content into university and training institutions’ curricula.  It is not too late to develop and implement a post-development marketing strategy for the three learning manuals, presuming funding can either be found or reallocated. Regardless, development of future industry-wide learning manuals should include funding for a strategy to market the manuals after development. Marketing activities may include such things as workshops, business-to-business interactions, working more directly with learning institutions, using staff as guest lecturers at educational institutions to deliver excerpts from the manual to students, working directly with educational institutions to incorporate the manual or manual content into course syllabi and content, and so forth.  Development of future learning manuals should include funding and a plan to pilot test the manual with intended users prior to full rollout. While we acknowledge TNS’ argument that the manual content was pilot tested during the SWTs, we believe that the final product itself should also undergo pilot testing prior to full rollout. That some of the subject matter experts claimed that the manual content was still at too high a level for its intended users supports our recommendation. One subject matter expert, for example, suggested using the consultants hired to provide or review content to pilot the manual, or parts of the manual, with their own students as part of their contracted duties.  Other illustrative strategies to increase program demonstration effects (of which there are many) include (1) increasing communications and outreach efforts by SAFE staff and clients to professional and technical associations of which the three program countries have many, for example, the gender focal person or a female nutrition expert could do a presentation and network at a women’s business association for example; (2) launching a monthly or bi-monthly informal gathering after work with management and staff of other market development programs and leveraging each other’s expertise Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 32 and resources; (3) reaching out to BDS and financial service providers, perhaps hosting an annual awards of innovators in food processing enterprises and inviting the press to cover the event; or (4) conducting SWTs in financing co-hosted by commercial lending and invoice discounting institutions. General Recommendations  Sustainability strategies should be built into program design at the outset and prioritized over the life of the program in terms of resource allocations even when they may come into conflict with other operational demands. For example, in the case of SAFE, the SWTs were (according to TNS staff) a primary sustainability strategy, but when funding conflicts arose due to the program expansion, they were cut out of the budget in favor of expanding one-on-one assistance to SGBs. In the case of the sector-wide learning manuals, these were also implemented as part of a sustainability strategy but ironically lacked their own sustainability strategy in that no funds had been allocated to pilot test the manuals or market them post completion.  Demonstration effects are an important contributor to program sustainability. Like sustainability, however, they are unlikely (or less likely) to occur on their own without planned facilitation by the program and thus require an explicit strategy for achieving as part of the program’s sustainability strategy.  One way to help ensure the creation and implementation of a sustainability strategy is to create a line item and allocate funds specifically for this purpose and to include specific indicators within the program MEP to measure its achievement. Lumping sustainability in with other program operations as a funding line-item and failing to hold the program accountable for achieving measurable results risks subsuming sustainability to other operational priorities, particularly those against which the program must report achievements. EQ4. IS PARTICIPATION IN SAFE ASSOCIATED WITH NEW OR STRENGTHENED MARKET LINKAGES BETWEEN FOOD-PROCESSING FIRMS AND SMALLHOLDER FARMERS, WHOLESALE BUYERS, RETAIL BUYERS, FOOD AID BUYERS, LENDING INSTITUTIONS, AND OTHER VALUE CHAIN ACTORS? Endline findings related to SAFE’s effectiveness in facilitating new market linkages for supported SGBs closely track those of the baseline. Looking at Table 11, the baseline and endline responses of SGBs in all three countries to the processor survey indicate that raw materials sourcing and commercial linkages have changed by somewhere between a little change and some change in both evaluation rounds with relatively little variation from the baseline to the endline to rate how much change in different types of commercial linkages they had experienced as a result of SAFE assistance using the same 4-point scale as above. Table11: Changes in Commercial Linkages at Assisted Processors’ as a Result of SAFE Assistance Indicator Kenya Zambia Malawi Total B E B E B E B E Raw materials sourcing 2.4 2.8 2.8 2.3 2.5 2.4 2.5 2.6 Commercial relationships 2.6 2.7 2.9 3.0 2.6 2.0 2.7 2.6 How much change have you made in your business as a result of the assistance you received: 1=No change; 2=A little change; 3=Some change; 4=Significant change Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 33  “They told us to use various channels to reach people, not just to use word of mouth, but to also use the radio, television, website and friends.”  “I was told that I have to complete the value chain by supporting the farmers in farming, sell them the seeds, sell them the sacks, and buy their produce and everything. That is something I have taken very seriously. You move one step, and you find that there is need for the farmers to get this, for example, they have grown the food and have harvested, but don’t know how to dry and they are using nylon. To improve the quality you give them services. You have earned some money and actually improved the quality and reduced wastage.”  “You see how come we are known and we just put the machines in last month? They taught me to talk to people, word of mouth. For example, when I go to meetings, I talk about my business and the services I provide, that is word of mouth. When we go to forums that deal with agriculture, we talk about ourselves, and we also use the Internet; sending people emails and the like.”  ”I can now handle bigger volumes of crops. I used to deal mostly in sorghum, the one you take to breweries, but now I can work on green grams. I was at BIDCO last week to sign a contract of sunflower and soya beans. We have more clients than before.” The findings in Table 11 and the quotes provided above find corroboration in the results from the SAFE performance monitoring system in Table 12 where we see that SAFE has facilitated a total of 87 new market linkages over the LOP to date, including 46 in Kenya, 39 in Zambia and 2 in Malawi. The 87 new market linkages represent 87% of the LOP target, indicating that SAFE is on its way to meeting its LOP target. Dividing by the number of SGBs assisted to date, this yields an average of 1.6 new market linkages per supported SGB. Table 12: Performance of Assisted Processors from the SAFE Performance Monitoring System: Cumulative as of March 30, 2016 Indicator Kenya Zambia Malawi Total Target % of Target Number of new market linkages created 46 39 2 87 100 87.0 Linkages to Smallholder Farmers While sector-wide expansion of food processing capacity may result in overall improvements in market opportunities for SHFs, it would be very difficult to attribute specific improvements to SAFE except in cases where the assisted SGBs have direct “operational” business links with smallholder suppliers, for example, through contract farming or out-grower schemes. Unfortunately, very few of the SAFE-assisted processors implement these types of supplier arrangements due to challenges in contract enforcement, which makes it very challenging for processors to recover investments in inputs when farmers do not perform, e.g., as in the case of ‘side selling’ when better alternative market opportunities present themselves. As a result, most SGBs procure commodities from wholesale markets or from community￾based traders. In certain cases, SGBs provide guidelines on quality requirements to the traders, such as appearance or variety preferences, which may have a positive impact on smallholder access to markets overall. However, even here it is difficult to attribute any impact to SAFE. COMACO in Zambia is a notable exception. Because the commercial viability of its products depends on accessing more demanding consumers in urban markets, COMACO engages intensively with its SHF suppliers. In line with its social welfare mission, COMACO focuses on income generation as well as on food security and nutritional outcomes for SHFs, which comprise a network of over 61,000 out-growers, 40,000 of whom reside in Eastern Province. Out-growers are organized into groups, which receive seeds and fertilizer, following a lead farmer model. The company provides seed loans to lead farmers, who are Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 34 expected to repay the loan one-to-one. The lead farmers also receive a bicycle and are expected to monitor rotation and yields of other farmers in their group. To ensure quality, COMACO works extensively with out-growers on production and handling through an extension team and through demonstration farms, field days and Farm Talk radio. In 2015, COMACO procured about 401 MT of groundnuts from 1,923 farmer groups and about 406 MT of soybeans from 1,495 farmer groups through its out-grower scheme (see Table 13). Table 13. 2015 COMACO Procurements Crop MT Groups Ground nuts 401 1,923 Soybeans 406 1,495 Groundnut: Nationwide, 35-49% of small and medium-scale farmers grow groundnuts, making it the second most grown crop after maize, a figure typically highest in Eastern Province. Groundnut is typically grown for home consumption (only 20% is sold, versus 50% for maize).16 One hundred percent of farmers interviewed reported previously growing groundnuts. Although COMACO trains farmers in production techniques to control aflatoxin, the company considers buying shelled groundnuts peanuts too risky because farmers soak the nuts to loosen the shells, which is a prime cause of contamination. As a result, COMACO has a shelling machine at the facility that handles about three MT per hour, as well as an on￾site aflatoxin testing lab. Farmers reported their average area under groundnut at approximately 0.4 HA in Kamphaza.17 Farmers also reported that groundnut requires more labor than other legumes or maize, as a result of three seasonal weedings recommended by COMACO but that the cost of other inputs, including soil preparation, for which farmers hire labor, is the same for maize, soybean and other legumes, and farmers are able to use tools they already possess to produce maize. All farmers interviewed reported improving groundnut productivity from just under one MT per hectare to between 1.6 and 1.8 MT per hectare since beginning to receive inputs from COMACO. They also report varying degrees of increased sales. COMACO purchases groundnut at ZMK 2.6 per kilo, or ZMK 130 for a 50 kilo bag. Soybean: In contrast, although soybean is the fourth most widely planted crop in Eastern Province (following maize, groundnut, and sunflower),18 only 32% of farmers interviewed had prior experience with it, mostly through a previous extension initiative, possibly by Dunavant. COMACO provides the hybrid Soprano variety from SeedCo, in conjunction with ZARI, to soybean farmers. This is a four-month maturing seed, but as with groundnut, most farmers only get one crop per year. Soybean is more profitable than maize per MT, but with soybean productivity highly sensitive to cleaning (weeding), production is 16 Mofya-Mukuka, Rhoda, Arthur Shipekesa “Value Chain Analysis of the Groundnuts Sector in the Eastern Province of Zambia” IAPRI Working Paper No. 78, September 2013 17 Farmers in the region measure land in limas: One lima = 0.25 hectares (HA) 18 Lubungu, Mary, William J. Burke, Nicholas J. Sitko “Analysis of the Soya Bean Value Chain in Zambia’s Eastern Province” Indaba Agricultural Policy Research Institute, (IAPRI) Working Paper 74, May 2013 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 35 lower.19 Farmers who initiated soybean production through COMACO support reported an average 3.5 years of experience with the crop. HH consumption is limited, and with an average of 72 percent of the crop sold, soybean is generally considered a cash crop, albeit one with an “unreliable market”. 20 Nonetheless, 100 percent of farmers interviewed reported consuming at least some of the crop, either as roasted whole beans, or blended with maize meal as a fortificant, largely as a result of COMACO training in home processing. Reported areas under soybean production were 0.5 hectares per farmer in Jeseti Farms and one hectare per farmer in Chithaza. All farmers reported intercropping soybean with maize and cowpea, which like maize are for home consumption. Like groundnut, farmers reported that soybean requires more labor than maize (COMACO recommends two weedings). Like groundnut, farmers are able to use tools they already possess for maize production. Reported productivity for soybean is approximately 1.7 MT per hectare, which compares favorably with regional averages, most likely as a result of hybrid seed use (for example, estimated average productivity is approximately 0.75 MT per hectare in Malawi21). Farmers reported selling 70-100% of their soybean crop to COMACO. Like groundnut, COMACO purchases soybean at ZMK 2.6 per kilo, or ZMK 130 for a 50 kilo bag. COMACO provides a good case study of an SGB whose business model has (or appears to have) a significant impact on its SHF suppliers, not only in terms of developing market linkages and increasing on￾farm production and income but also in terms of farmer nutrition. Key to these outcomes, however, is that COMACO made a strategic decision to invest significantly in production systems consistent with its mission on improving producer livelihoods. Access to Finance Since the baseline, SAFE has increased its efforts to help link its supported SGBs to financial institutions in its role as a facilitator. This is especially true with the SAFE Kenya program in linking its clients with lenders and the development of interactive financial planning software tools to identify financing needs and alternative sources of credit. Each of the seventeen SGBs SAFE assisted to access bank loans successfully secured a bank loan, while an additional four SGBs obtained bank loans without SAFE assistance (see Table 14). A limited number of other supported SGBs were self-funded from private investors (e.g., Java Foods and High Protein Foods in Zambia), although this type of self-funding is generally not a replicable model for most other SGBs. 19 Deaker, Rosalind et al “Legume Seed Inoculation Technology – A Review” Elsevier University, School of Land Water and Crop Sciences, 2004 20 Ibid 21 “Assessment of Five Value Chains in Malawi” Absolute Options (SO) for Land O’Lakes, October 2015 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 36 Table 14: Performance of Assisted Processors from the SAFE Performance Monitoring System: Cumulative as of March 30, 2016 Indicator Kenya Zambia Malawi Total Target % of Target Number of MSMEs receiving assistance to access bank loans 13 2 2 17 20 85.0 Number of processors accessing bank loans 18 2 1 21 12 175.0 For those SGBs who were not, or who are not, able to secure financing, however, this outcome imposes significant constraints on their business growth prospects. There are several facets to this financing constraint. First is the high cost of credit, which was cited by key informants to range from 19% to 40% when bank loan fees are factored in the cost. A second constraint, applicable primarily to small and women-owned enterprises, is that the enterprise must be registered, and in order to be registered, the enterprise must meet certain minimum standards for food processing, which takes investment capital. In the case of Fresh n’ Crunchy in Kenya, for example, the owner has limited assets that she can use as collateral, and to get certified as a registered business, improvements must be made to the facility, such as new flooring and dry storage areas. Slow paying customers, such as the major supermarket chains, lead to SGBs being unable to finance their operations internally and thus to take advantage of bulk purchases. Several SGBs say that they have invoices exceeding 90 days and are facing severe cash flow issues due to delayed payment. As SMEs, they lack bargaining power and are thus not usually in the position to insist on quicker payment. In Kenya, invoice discounting, such as that offered by Umati Capital, may be available to clients as a stopgap measure. In Zambia, this financing option is just being introduced. (The long payment lag time with large retail chains is one possible motivation for processors to target the informal market, which tends to operate on a cash basis, although in much smaller volumes and packaging. See the discussion in below.) Financing constraints among SGBs are in turn creating constraints on SAFE’s ability to execute charters with certain SBGs. A small number of SGBs, for example, reported that they have run out of inventory due to a lack of working capital financing. Take the case of SHARE Africa in Zambia, for example. According to the Managing Director of SHARE Africa, “From the time we identified early wins, with financing, we could have been certified (and had access to institutional clients) and it’s been frustrating.” As of March 2016, SHARE Africa was out of cash while awaiting payments from two large supermarket chains. The enterprise has a new equipment line but is unable to buy raw materials, and their existing inventory is low. The Managing Director would like to access invoice discounting credit, but he is unaware of this option in Zambia. As a stopgap measure, he is planning to sell off assets, such as trucks. A female Managing Director of another Zambian SGB said that she was so desperate for financing that she was even willing to pay the exorbitant interest rate offered by a commercial bank. Notwithstanding, the bank turned down her loan request. SAFE uses a variety of approaches to help link its supported SGBs to sources of financing. These include inviting representatives of financial institutions to make group presentations to assisted SGBs, passing information about certain SGBs to potential funders, working with SGBs directly to improve their financial reporting or other systems, and preparing loan documents, as reflected in the following quotes:  “They have never been involved in funding but they did connect us to a company called Root Capital. They gave us a website in fact the Root Capital did visit us.” Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 37  “The main project that we have implemented with them is that business plan. This business plan was to enable us negotiate with financial institutions and also be able to have discussions with even donor and also the government”  “They did introduce us to a financier named Root Capital, and we had some interactions with Root Capital. We were interacting on the basis that we need a loan of five million and the intention was of getting this loan was to be able to finance the contents of the business plan.”  “They improved our reporting and financial management … how to do those kinds of presentation and how to prepare the reports that are needed.” SAFE takes, in the words of one financial institution key informant, a “commercialization approach” to linking food processors with financing sources, which, according to the same key informant, makes SAFE unique among the other NGOs providing access to finance support. As part of this commercialization approach, SAFE does not act as an advocate, per se, for its supported SGBs with financial institutions, nor does it seek in any way to influence the latter’s credit decisions. Once SAFE has facilitated a linkage, it is content to let the process play out. Its strategy is instead to bring the SGBs to the point where they are creditworthy before attempting to link them up with financing. As noted by a key informant at a Kenyan financial institution:  “SAFE provides us a unique pipeline in clients. It helps us find them, in most instances already provided with some capacity development, and helps formalize and make them ready, which really helps our ability to provide finance.” The same key informant says that he talks frequently with TNS who time-to-time provides him leads on potential clients that “it has pre-screened and is confident will be a good fit. It then takes a back seat and lets the process proceed.” Sometimes this approach works:  “They were not actually involved in the process. But the advice they gave us was enough for us to be able to satisfy the demand or the request from the bank.”  “The money that we got was actually as a result of the preparation that we got from TechnoServe.” Sometimes it does not:  “They introduced me to Root Capital. We did the cash flows with them, but I didn’t get the loan.”  “They have introduced us to the finance institutions like investors, but we have not gotten any financing, but maybe we will get in future now that we know them.” Judging, however, by the number of supported SGBs who have accessed commercial loans since the beginning of the program, relative to the number who were assisted by SAFE to access loans, SAFE’s commercialization approach appears to have been highly successful. In addition to the above, the SAFE Kenya office is in the process of conducting a financial products survey to determine what types of lenders there are in the markets and what types of products they offer, and from the results of this survey to develop an interactive Excel-based tool for SGBs to help them determine which type of financial products are best for them. SAFE expects the guide to be available by end of June. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 38 Conclusions SAFE has done a moderately good job facilitating new commercial linkages for its supported SGBs over the life of the program. Of those SGBs it has assisted to access loans, most have successfully secured loans. Notwithstanding, the number of SAFE-supported SGBs receiving loans remains a relatively small portion of the total number of SGBs has supported to date. Of all the different measures of SAFE performance included in the processor survey, SAFE’s success in facilitating new commercial linkages was consistently rated the lowest among respondent SGBs. Limited access to finance, both for working capital and fixed investment, remains as significant a constraint to assisted SGBs in the endline as it did in the baseline. This combined with the lengthy delays in incurred in getting paid by chain supermarkets (who are primary clients of several supported SGBs) is adversely affecting a number of the SAFE-supported SGBs in developing their enterprises and moving forward as planned with the SAFE charters. At the same time, however, SAFE has become more active since the baseline in working to facilitate linkages between its supported SGBs and different sources of commercial finance, while also investing in creating an interactive tool to guide SBGs in finding and accessing different types of financing. In doing so, SAFE follows a strict commercialization approach, in which it limits its role to that of facilitator, both by helping SGBs develop their capacity to apply for and qualify for commercial financing and by directly linking SGBs with financial institutions, whether individually or in a group setting. While SAFE’s commercialization approach to financing has produced a number of successful linkages, the approach has natural limitations, being constrained by the capacity and creditworthiness of its assisted SGBs and by the credit policies at financial institutions. The commercialization approach works well for those relatively few SGBs selected to receive assistance, but the selectiveness and hands-off nature of the approach inherently limit the number of SGBs who will successfully access loans as a result. Notwithstanding, we conclude that SAFE’s commercialization approach to financing is fully consistent with its broader strategy of SGB support, which also employs a commercialization approach. Given the nature of the program, and its mandate, it would be, in our view, inappropriate for SAFE to take a more hands￾on role in providing, or finding, commercial financing for its supported SGBs. SAFE’s strategy of preparing and pre-screening its SBG clients before attempting to link them with financial institutions is a more cautious approach that, nonetheless, should not affect its outcomes too significantly (e.g. a more scattergun approach would most likely result primarily in more loan denials) and is one that should also significantly enhance its credibility with lending institutions thereby paving the way for greater success down the road. The farming systems research conducted by the Markets Specialist suggests that SAFE assistance can be effective in facilitating the expansion of SGBs that engage smallholder farmers directly through contract farming and out-growers operations, resulting in increases in productivity, which in turn are likely to increase consumption and income of farmers. Opportunities to replicate this success are limited by the prevailing procurement practices of most processors in the region. However, SAFE should leverage these opportunities where possible. Recommendations SAFE Recommendations  SAFE should continue with its current commercialization approach to financing. There are, however, ways it can possibly strengthen it. For example, SAFE could continue to provide, or link its clients to, business development training and counseling on improved record keeping, which is still a high priority according to TNS staff. CHASE Bank in Nairobi cited the lack of adequate record keeping and the Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 39 financial skills of general managers to know if their business is profitable. SAFE is well aware of the problems and is working to identify and develop linkages between their clients and lending institutions.  Since financing is not a core area of SAFE support to the SGBs, developing closer linkages to other donor programs, progressive commercial lenders and government initiatives for SME financing is another possible approach to address financial access.  SWTson financial planning for agricultural enterprises and internal and external sources of credit are recommended, along with linkages to business development services. SAFE could co-host in collaboration with other business service providers, donors, lending institutions and program implementers to take advantage of existing resources such as the USAID DCA loan guarantee facility.  The interactive tool being developed by the SAFE Kenya office to help SGBs navigate the financial landscape better has potential for being a useful tool for the food processing sector in that country. Extending this initiative to the other SAFE country programs would be worthwhile if it is proven that the interactive tool being offered to SAFE’s Kenya clients is helpful and the enterprises are better able to address the credit constraints that is keeping them from making necessary upgrades and purchase inputs that are needed to expand and move forward with SAFE.  With regards to increasing its impact on SHFs, SAFE should prioritize and target assistance to SGBs that employ out-grower and contract farming schemes in order to increase the impact of assistance on smallholders in terms of increased market linkages and improved income and nutrition of farmers. SAFE should further consider providing complimentary assistance to SGBs on procurement models, or partner with projects that specialize in this area in order to enhance linkage between their assistance and impact on producers. Finally, SAFE should consider adding TA capacity in supplier support to the types of TA it provides, or partnering with similar program that implement this type of support. General Recommendations  Consistency in philosophy and approach across a program’s diverse set of activities is important and should be adhered to in all (or nearly all) cases. In SAFE’s case, its decision to take a commercialization approach regarding access to finance is fully consistent with its broader organizational principles and market-oriented approach to building the capacity of food processors in the three countries studied. While such an approach may not always maximize certain outcomes, it enhances the program’s credibility with stakeholders and helps ensure that the program is sending a consistent set of messages both to external stakeholders and internal staff.  Commercial linkages are much less like to emerge organically and thus require an active facilitation effort by the program based on mutual advantage to all parties. A particular application of this principle in SAFE’s case is the program’s goal to generate downstream benefits to smallholder farmers supplying supported SGBs. As emphasized in both the baseline and endline, this goal cannot be assumed to occur organically but requires a specific strategy to achieve it, which includes at a minimum targeting SGBs who have existing commercial relationships with SHFs via some kind of out-grower or contract farming scheme. As we imagine many other programs similarly seek to create downstream benefits to SHFs, SAFE’s experience, and the lesson learned, has broad applicability. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 40 EQ5. HAVE SAFE INTERVENTIONS INCREASED THE AVAILABILITY OF NUTRITIOUS FOODS, PARTICULARLY IN MARKETS CHARACTERIZED BY HIGHER LEVELS OF FOOD INSECURITY AMONG THE CONSUMER POPULATION? Any increased availability (supply) of nutritious foods resulting from SAFE assistance has not been matched by an increase in access to (and consumption of) nutritious foods among food-insecure consumers. By and large, SAFE-assisted processors neither are targeting food-insecure (or generally low-income) consumers nor selling/distributing their products via outlets traditionally patronized by food-insecure consumers. Food processors trying to sell to this market face a variety of obstacles that make it a much different market segment than the ones they are used to serving.” Nonetheless, based on the feedback from the baseline evaluation, and specifically the baseline household survey, TNS has begun considering how SAFE might better support SGBs to market to food-insecure populations. Toward this end, TNS has submitted a proposal to USAID to develop products and marketing strategies for supported SGBs to target insecure populations. USAID has yet to approve the proposal, so no action has been taken on this to date. In lieu of this, TNS has recently hired two consultants to undertake a study of the ‘informal markets’ in Kenya. The informal market is the market where lower income people living in urban settlement areas and marginal rural areas shop for food. It is characterized by numerous small shops (e.g., kiosks, dukas) selling food to low-income consumers who shop daily for their needs in small packets, often repackaged, for small amounts of cash, although often also on credit. In Kenya, this market is referred to as the ‘kadogo’ (small) economy. (Zambia and Malawi have their own forms of the kadogo economy, which operate in many ways similarly to that in Kenya; however, the informal market research was limited to Kenya.) At this time, few if any of the SAFE-supported SGBS are actively selling in the informal economy, which requires a specialized marketing and distribution strategy, but tend instead to sell to sell to large retail chains. The advantage to selling to the large retail chains is the lower marketing and distribution costs associated with selling in large volumes through established distribution networks; the downside is that the retail chains also take longer to pay—up to 30 days—which can cause serious cash flow or working capital problems for the SGBs. The advantage of selling within the informal economy is that payment is made in cash; the downside is that it requires a quite different marketing strategy, selling in smaller volumes and in smaller packages, and working through more fragmented distribution networks. TNS recognizes that most of the food processors it works with are not set up to sell in the informal economy, and would require a good amount of technical assistance to get them there, or alternatively it could target SGBs that are already serving this market. Thus the purpose of the information market study for TNS is to try to understand the process by which foods are distributed down to the kiosks, neighborhood shops and open air markets where most of the lower income residents of urban settlement areas and marginal rural areas shop. Recognizing that (1) selling to poor, food-insecure people is enshrined in the SAFE ToC, (2) SAFE￾supported SGBs are not, by and large, selling within the informal markets where these people shop, and (3) there is a desire among some SAFE stakeholders to do a better job targeting/reaching these informal markets, the ET looked at these issues as part of the SAFE endline evaluation. Their interest here was to understand how processed food products reach the informal market, what the characteristics of this market are, and what the factors are that drive purchasing decisions by low-income consumers who shop within this market. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 41 To conduct this research, the ETmade the following changes to the in-depth case study method used in the baseline evaluation: 1. Added questions about shopping preferences and behaviors to the household survey, with a focus on where respondents shop and why and what they buy there and in what form. 2. Revised the consumer FGD guide to focus on questions related to where respondents shop and why and what they buy there, and in what form. 3. For the market actors KIIs, selected retailers and wholesalers operating in the informal markets, and revised the market actor KII guide to focus on question related to how foods are distributed within the informal market, by whom, in what form, and under what terms. The primary findings from this research relevant for answering EQ5 are summarized below. A more detailed presentation of the in-depth case study findings is found in Annex 7: In-Depth Case Study Findings. Relatively few of the persons living in the case study communities shop at the supermarkets where the processed foods produced by SAFE-supported SGBs tend to be sold. Instead, they most frequently shop at, and purchase the large share of their food, from kiosks, neighborhood shops, and open-air markets. Kiosks are usually semi-permanent structures, close to home and sell all the basic foodstuffs. The bulk of perishables (bread, milk, vegetables) are bought here. Because the shoppers do not have fridges or storage space, they buy goods as they need them, visiting such outlets once or twice a day. These outlets also sell vegetables and some will even chop these vegetables to order. Neighborhood shops are permanent structures but otherwise similar in function to kiosks. They sell a wide range of items in small quantities. Customers expect to find the same range of items as in kiosks but in addition these shops will also sell cleaning and personal hygiene products. Generally customers do not enter the store but are served through a window. In Kenya, kiosks are the most important retail outlet for food purchases among the case study population, while in Zambia open-air markets are the most important outlet for food purchases followed by kiosks. In contrast, supermarkets account for only 7% of all food shopping visits and 25% of all food shopping expenditures in the Kenyan case study communities. In the rural communities covered by the Zambia case study, shoppers are so cash strapped that they rarely venture into supermarkets, thus making supermarkets a negligible share of food shopping visits and food expenditures. Supermarkets are commended for stocking a wide variety of products, selling branded portion packs, and selling at affordable prices. Kiosks and neighborhood shops, on the other hand, sell at higher per-unit price, but they are attractive to the case study populations because they (1) are located in convenient, easy-to-reach locations, (2) provide credit facilities, and (3) sell small, affordable packages (or amounts). In the latter case, kiosks and neighborhood shops will typically break up larger packs and sell portions from those packs. For example, instead of having to buy the full 2kg pack of maize meal, consumers can purchase the amount they require for their next meal. This means the shopper can afford the range of products required to make one meal (charcoal, cooking oil, a tomato, some greens, and some maize meal) even with only a very small amount of cash. Otherwise, if the shopper had to buy a standard retail pack, he or she would get only the maize meal. Offering food products in smaller packs in this way is critical, as the case study populations lack the money to purchase food items in bulk, or in larger packets; even if they could purchase larger packets, most lack proper storage for them. Indeed, cost remains the single most important factor influencing food purchasing decisions among the case study populations in both countries. Brand identity is one of the least important factors influencing food purchasing decisions with taste and quality falling somewhere in-between. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 42 Fresh food is generally purchased on a day-to-day basis, and the quantities purchased per shopping occasion are very small, while non-perishable items are bought either weekly or monthly. For consumers on irregular incomes, kiosks and neighborhood shops are often the only option. Thus what many households in the case study communities eat on a given day is determined by how much money they have on hand and what other demands they have on these funds. They do not have cash reserves, and thus buy just enough to make each meal. Hence, they are unable to buy packaged products from formal retail outlets and rely on retailers who are willing to sell them small quantities, such as a cup of maize meal, a half kwacha of salt, or one kwacha of sugar. At the wholesale level, wholesalers have limited options as to where they can source goods. In Kenya wholesalers are generally supplied either directly from the producer or through a manufacturer’s appointed distributor, while in Zambia wholesalers are operating in more remote rural areas and thus their options for sourcing goods are even more limited owing to the high cost to transport goods. Goods are generally delivered to the wholesaler for cash on delivery. While credit is not routinely offered to the wholesaler, the seller may offer limited credit terms for trusted customers. As for the retailers, their main suppliers are the wholesalers. In Kenya, wholesalers pay the cost of transport, while in the less competitive environment in Zambia; retailers pay the cost of transport. For retailers, desired qualities of suppliers include (1) reliability and quick delivery of products, (2) competitive prices, (3) ability to supply in small quantities (small packages), (4) accepts payment on delivery, and (5) ability to provide fresh produce. Cash flow is very tight for the retailers, and their business model depends on rapid turnover of inventory They are not able to keep large stocks on hand (owing to lack of space and refrigeration); therefore, close proximity to their source of supply, and responsiveness by their wholesaler, are important to ensure quick and easy delivery. Both retailers and wholesalers strive to build a relationship with their customers, and believe that their customers value the security of knowing that the product they want will be readily available when they want them. Because the retailers’ customer base is extremely price sensitive, retailers need to keep their prices low. Thus they are very keen to find suppliers who offer savings, not only in terms of cash prices but also in terms of delivery costs. The willingness of wholesalers to break down packages into smaller quantities is also critical, although a less common practice in Zambia than in Kenya. Not only do the retailers repackage their products to suit the purchasing needs and capacity of their customers, so do the wholesalers. Conclusions Overall, SAFE-supported SGBs are not reaching vulnerable households with nutritious food products. This goal has not been prioritized in SAFE’s design or SOW and, not surprisingly, has not been incorporated into SAFE TA to supported SGBs. Nor, consistent with its mandate, has SAFE made any effort to target support for SGBs that are serving this market. Interestingly, when asked in the processor survey whether their products were reaching vulnerable households using a 4-point scale in which 1 equals ‘not at all’ and 4 equals ‘a lot,’ respondent processors in all three countries said on average that they are reaching them ‘a good amount’ (see Table 15). Based on the evidence collected in the baseline and endline, it appears that the assisted SGBs have erroneous perceptions about just how far down the socio-economic ladder their products are reaching. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 43 Table15: SGB Perceptions of the Extent to which Their Products Reach Vulnerable Households Kenya Zambia Malawi Total B E B E B E B E 2.5 3.1 2.9 3.1 3.0 3.1 2.7 3.1 As best you can tell, to what extent does the food you produce for which you receive SAFE assistance reach people vulnerable to food insecurity or malnutrition: 1=Not at all; 2=Not very much; 3=A good amount; 4=A lot Case study subjects in both countries make the large majority of their food purchases, as measured by frequency of purchases and volume of purchases, at informal market retail outlets. In Kenya, kiosks are the primary retail outlet of choice, followed by open-air markets and supermarkets. In Zambia, open-air markets are the primary retail outlets of choice followed by kiosks and neighborhood shops. The relative preference for open-air markets in Zambia reflects the rural setting of the case study location. On average, shoppers visit the kiosks several times a week to purchase their daily and other short-term food and dry goods needs. Study subjects in Zambia do not shop at supermarkets. Kiosks and neighborhood shops offer a few key advantages that make them the outlet of choice for so many low-income shoppers. First, they repackage goods in small packs, which the low-income shoppers can afford. Although goods packaged this way typically carry a higher per-unit cost than similar products purchased in larger packs, the overall lower cost per pack is affordable for low-income shoppers with irregular incomes and who often have only pennies to spend at a time. The importance of this cannot be overstated. Among the factors low-income shoppers consider in making purchasing decisions, price (as in the baseline) is the overriding consideration. Second, they often offer credit, which enables low-income shoppers to purchase when cash flow is lower and repay when cash flower is higher. Third, they offer convenience, often being located near the shopper’s place of residence. Fourth, they offer familiarity with kiosk and shop owners often knowing their customers name and going to lengths to stock a wide variety of brans and pack sizes according to their familiarity with their customers’ needs. The market characteristics listed above are also reflected among food wholesalers and retailers in which low prices, flexibility, responsiveness, and selling in small packets—in other words, offering food products demanded by low-income consumers how, when, where, and in the quantity and at the price demanded— are all important contributors to successful operations. Thus any food processor seeking to sell to low income populations in large volumes will need to adapt its business model so as to distribute its products through the kiosks and neighborhood shops, which dominate the informal food retail market. Moreover, to do so will require, at a minimum, packaging and pricing strategies targeted to this market. This begins, moreover, at the wholesale level in which wholesalers often also expect goods to be packaged and priced so as to permit quick and easy distribution down to the retail level. Another requirement for working with in the informal markets is a flexible distribution model, which is capable of getting goods to buyers quickly, at a low cost and in the correct amounts. Recommendations SAFE Recommendations  In light of SAFE’s goal to improve the nutritional outcomes in poor and vulnerable households, which requires in turn that nutritious foods produced with SAFE assistance actually reach them, TNS’ decision to study the informal market is both well considered and overdue. It is clear by now that the SAFE model in its current form will not accomplish this goal. While this outcome is an artifact of the program design and SAFE’s SOW (and thus not attributable to SAFE in any way), the ETrecommends Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 44 that BFS actively take an interest in and support this research, as it has important implications for any SAFE extension as well as for future BFS programming decisions.  BFS should give serious consideration to revising the SOW for any SAFE extension to include a provision requiring, or encouraging, the follow-on program to support food processors to increase their presence in and penetration of the informal wholesale and retail food markets for processed foods, subject to the findings of the SAFE informal market research. This could be accomplished either by identifying and supporting food processors already serving this market and/or by supporting food processors to diversity into this market.  As demonstrated in thecase study research, informal food markets in different countries share many characteristics, but also involve some important differences. Given that SAFE’s current informal market research is limited to Kenya, the ET recommends that BFS support additional research on informal food markets in the other countries served by SAFE or by any SAFE follow-on program. This research may take the form of primary or secondary research, or some combination of the two. General Recommendations  Despite SAFE’s high-level goal to improve the nutritional status of poor and vulnerable populations, there is nothing built into the program design that would achieve this goal, other than (it appears) an assumption that such a result will occur organically as an indirect result of program interventions. The relevant goal also reflects a misunderstanding of the process by which processed foods reach poor and vulnerable consumers, and the supply and demand characteristics of the markets in which poor and vulnerable consumers shop for food. That SAFE was not tasked with achieving results related to this goal meant that no work was done early on in the program to determine the consistency of the goal and the program design or to think critically about the process by which program interventions would actually contribute to this goal. Thus the general recommendation is that program design needs to be consistently checked against the program’s higher-level objectives and goals, even if the program is not strictly tasked with achieving those objectives or goals, so as to determine consistency between the design and the stated objective or goal. Engaging in this activity is likely in many cases to uncover hidden assumptions in the objective or goal that can then be checked against what is known about the existing program environment, and/or it can trigger investigations into the validity of key assumptions.  Generally speaking, poor and vulnerable households are likely to perceive and live a very different reality than program designers. To the extent a program aims to generate benefits for these households, whether directly or indirectly, it is critical to take the effort to learn what this alternative reality is and to check program assumptions against it. EQ6. TO WHAT EXTENT HAS ANY INCREASE IN THE AVAILABILITY OF NUTRITIOUS FOODS BEEN MATCHED BY AN INCREASE IN ACCESS TO (AND CONSUMPTION OF) NUTRITIOUS FOODS, PARTICULARLY IN MARKETS CHARACTERIZED BY HIGHER LEVELS OF FOOD INSECURITY AMONG THE CONSUMER POPULATION? In EQ5 the ET concluded that there does not appear to have been any increase in availability of nutritious foods to vulnerable households among the SAFE-supported SGBs. It follows, therefore, that neither has there been any increase in the access to nutritious foods among vulnerable households. There has thus been no change from the baseline to the endline with regards to either EQ5 or EQ6, for reasons explained above. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 45 In its response to the baseline findings regarding EQ5 and EQ6, TNS suggested that there were two possible channels by which products produced by its supported SGBs might reach vulnerable households. One channel was via the informal markets; however, as demonstrated above, there appears to be little probability that the supported SGBs’ products are reaching this market to a significant degree, given the requirements for selling within this market. (There is a possibility that some wholesalers or retailers are purchasing supported SGBs’ products in the supermarket, repackaging them and reselling them in the informal market, but it was not possible with the existing evaluation methodology to research this possibility. The evidence gathered by the evaluation team, however, suggests that to the degree this is happening, it is more likely that it is happening at the margin rather than on any large-scale basis.) The second channel is via assisted food processors working specifically in low income areas and reached indirectly via institutional food buyers (the processor sells to the institutional food buyer who then distributes the food to the vulnerable households) or directly by selling within the low income communities. In support of this second channel, SAFE collaborated with the WFP under its Purchase for Progress (P4P) Program to deliver five short-term trainings for 19 small food processors during September 2015. Two of the trainings took place in the Kakuma refugee camp in Northwestern Kenya. Participants in the trainings included, among others, community-based or cooperative food processors selling directly to the local population or to the WFP. (See Table 16 for a summary of the SAFE-WFP short-term trainings.) During the endline fieldwork, the evaluation conducted KIIs with five of the SGBs participating in these short-term trainings and two of the WFP officers overseeing the trainings. The results of these interviews are summarized below. Table16: WFP Short-Term Trainings Name Date Location Number of SGBs Number of Participants Male Female Total Batch fortification for micro￾millers September 14- 15, 2015 Kakuma Refugee Camp 2 11 11 22 Hammer milling September 16- 17, 2015 Kakuma Refugee Camp 3 6 20 26 Hammer milling September 18- 19, 2015 Lodwar 3 9 6 15 Hammer milling September 21- 22, 2015 Eldoret 7 12 13 25 Hammer milling September 23- 24, 2015 Kilgoris 4 6 12 18 Total 19 33 51 84 Under the P4P, the WFP operates a milling project whereby it buys grain from local farmers and then mills the grains to supply flour for the school feeding program in Kakuma and elsewhere. (The Kenyan government and United Nations High Commissioner for Refugees Kakuma manage the Kakuma refugee camp, which has capacity for 150,000 refugees.) The food processors participating in the short-term trainings were, for the most part, small community-based organizations or cooperatives. While most participants were heavily dependent on WFP purchases, most also sold to other institutional buyers— including Mary’s Meals, National Cereal and Produce Board, schools—or to traders or elsewhere within the marketplace. Most were also new to milling, so they needed basic training in just about all aspects of milling.  “They said for us to work, we had to be trained first, so that we know the proper ways to work and operate the machines. So they had to educate us on how to conduct all operations.” Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 46 Given that a large majority of mill operators, moreover, were illiterate and spoke a variety of languages, SAFE delivered the training relying heavily on pictorials and diagrams:  “Sometimes they would draw diagrams and so on. They are very clear because our members are farmers, and what they were drawing are the things that we deal with. Some of the topics that we hadn’t been doing or weren’t familiar with were explained using diagrams, and everybody understood.” The short-term training participants expressed overwhelmingly positive opinions of the training and felt that it had a significant impact on their business, particularly in the areas of hygiene, business management, networking, and marketing.  “Hygiene, it has really helped us because if not adhered to, we can’t really grow. When we learnt about hygiene; we were able to know what to do to maintain cleanliness, for example keeping sick people away from food, wearing protective gear, and how to handle food. Before we would have sick workers because of the way they are handling the food. Now we don’t have such issues.”  “Record keeping has been very helpful in maintaining all the records in our business and keeping account of everything. This has been very helpful.”  “We have interacted with different groups from other places. We are able to understand what other farmers for example from Trans-Mara and Kitale are doing. This program has therefore given us the chance to interact with different farmers and learn from them.”  “Before the training, we didn’t even have a market; we used to sell on retail here. Once we got the training and got the stage of supplying maize to them, we are proud to have a dedicated market for our products.” Training participants further reported making a number of changes to how they run their businesses as a result of the training:  “If you take the year 2010, we started with a supply of 500 bags to WFP - they are 90 Kg. But we have reduced the bag quantity to 50 Kg. In 2011, we improved to 2,220 bags. We sold for over three million. In the year 2012, we sold 2,956 bags and 623 bags of beans. Right now they buy about 1600 bags from us. We have really progressed, the group revenue has increased.”  “The biggest change that has been realized is having a ready market. We have been linked to the millers as a market, and we can sit down without worrying about the quality and quantity we offer. We are confident that we can go to any market and survive because of our better quality.” Another benefit cited by the short-term training participants (most of whom are also farmers) was improved on-farm knowledge, practice and outcomes, in addition to improved household income as a result.  “After getting the training, we are now able to supply high quality cereals because we know what to do after the harvest. At the same time, we know the storage procedures.”  “We are very thankful because WFP has really developed us. We were not like this before but right now we know that as a farmer, it is not just about growing food for your household, but it is also about growing extra to sell for an extra coin. You can be able to educate your children just from what you get from selling your produce. Right now, we don’t focus on just growing food, but we look at it as a business.”  “Now we have more food than before for our families and we are able to sell to the market. So you can say that we have eradicated the shortage of food.” For its part, the WFP was also satisfied with the outcome of the training: Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 47  “For what we agreed on, I am satisfied with what they’ve given us. I’m happy with the training and the training material they left us. It went well.” Conclusions While the evaluation methodology used does not allow us to reach conclusions about the specific nature and extent of the benefits received by participants in the short-term miller training, by all indications the short-term milling training delivered by SAFE in collaboration with the WFP was a success. The training was well received by all participants interviewed, who universally claimed that it had a significant impact either on their business, their on-farm production, or their household wellbeing. The short-term milling training accomplished several outcomes consistent with SAFE’s objectives. First, similar to the SWTs, they proved to be cost-effective mechanism for delivering useful technical assistance to multiple SGBs at a single time. Second, they reached a cohort of small relatively disadvantaged SGBs for whom such technical assistance would normally be out of their research. Third, the assisted SGBs were, for the most part, selling directly or indirectly (via institutional buyers) into markets characterized by high levels of vulnerable households. Fourth, because the assisted SGBs were membership-based community-based organizations or cooperatives made up largely of smallholder farmers, the trainings also directly or indirectly benefitted these farmers in terms of increased on-farm production and household income, and possibly (although not mentioned by KII respondents) even household nutritional status. Recommendations SAFE Recommendations  The last two outcomes mentioned in the above paragraph represent outcomes that have proven elusive with the other SGBs to whom SAFE has provided customized TA. If BFS is committed to increasing outreach to marginal communities and vulnerable households via TA to small food processors, the type of short-term miller trainings delivered by SAFE in conjunction with the WFP provide a potentially effective option for pursing these objectives, and should therefore be considered for expansion under the current SAFE program and for continuation under any SAFE follow-up program. General Recommendations  A single training is rarely sufficient to induce long-term, sustainable changes in production practices or outcomes (or most any behavior or outcomes for that matter). In the case of SAFE, this conclusion was noted by a number of the WFP training participants, as the following quotes demonstrate: o “There was only one aspect that they didn’t fulfill from my perspective. They were to assess the financial viability of the groups we are working with, so that hasn’t happened.” o “I would recommend a follow-up training just to ensure that it was actually effective and to address any gaps that were still there.” o “There was one aspect we did not expound on, the fortification. They were very instrumental in linking us with the fortification equipment and the fortificants. If we get funding to expand the milling project, then we will try to expand the fortification aspect.” o “I would say there is scope for more trainings. The initial discussions we had with SAFE they had talked about the training on aflatoxin control. That would involve more training for the farmer group.” Thus a general recommendation is to pair up short-term technical trainings with follow-on activities to reinforce past training topics and/or give training participants an opportunity to implement what they learned. There exist any number of options for follow-up activities, for example, produce and Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 48 distribute a newsletter with, among other things, examples of trainees who have successfully implemented practices or technologies covered in the training; offering a help-desk for trainees to ask follow-up questions or receive virtual one-on-one mentoring; facilitating cross-organizational visits by training participants; holding contests among trainees to recognize those who have done the best jobs of implementing training concepts; visiting trainees at their organizations to provide follow-up mentoring; or conducting post-training assessments to determine their effectiveness in changing trainees’ attitudes and behaviors. The ET acknowledges that implementing post-training activities requires staff time and budget, but spending money on activities that lack sustainable results is itself costly, and thus an incremental expenditure to improve the effectiveness of short-term trainings can well be worth it in terms of increased returns. At the very least, it may be worthwhile to commission a study in one or more BFS￾funded program to assess the effectiveness of short-term trainings and the relative effectiveness of different follow-up activities. EQ7. IS PARTICIPATION IN SAFE ASSOCIATED WITH INCREASED LEADERSHIP/MANAGEMENT OPPORTUNITIES FOR WOMEN IN SAFE￾ASSISTED FOOD-PROCESSING FIRMS? In the baseline evaluation, the evidence gathered from nine FGDs of women employees and casual laborers did not support the existence of any direct link between SAFE client assistance and increased leadership opportunities and advancement for women. Low literacy, especially among female part-time workers, was noted as a significant constraint to advancement. SAFE program management responded to this finding and suggested recommendations in the baseline evaluation that … “behavioral change at management is largely beyond the scope of the current program agreement . . . [however] there may be some scope to catalyze a shift in attitudes and allow for more opportunities for women.”22 In the endline KIIs of the Kenya and Zambia SAFE processors, SGBs were asked, “Has SAFE worked with you to help address issues related to female employees?” This is a broader question than asked in the baseline, but did allow for other positive changes initiated through SAFE to be captured. Below are key findings and selected responses from the KIIs with the processors: SGBs were more likely to support the notion of gender parity than they were that of ‘women’s empowerment:”  “I don’t think we’ve had where it has been specific that it’s female employees.”  ”Do I try to create opportunities for women? No, I try to create opportunities for everybody who has something to offer.”  “In that regard I think people realize that they can’t just hire men. That is one of my priorities that we know about. There are particular opportunities for women but not anymore than there are for men.”  ”Do I try to create opportunities for women? No, I try to create opportunities for everybody who has something to offer” 22 SAFE Evaluation-Management Response, undated, page 9. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 49 Most SGBs, moreover, made little to no connection between SAFE and gender issues. They were much more likely to recall being asked about the number of female employees they had, but they did not understand this to be part of any particular gender initiative:  “They first assessed the number of employees we have, and they took the number of women, that is what they did. And they were impressed on the number of women we have employed.”  “No they didn’t. They did talk about gender, everybody is talking about gender now, and when we were writing those reports, they were asking how many female employees and youth are there.”  “I don’t necessarily recall emphasis on gender...promotion of women. I do think there were probably some conditions but I can’t quite remember.” A small number of SGBs have specifically tried to improve the work opportunities for women. In limited instances, company polities and work conditions have improved, and women have taken the initiative for their own professional development as a result of SAFE initiatives and interactions:  “TechnoServe mostly sent female staff that interacted with and motivated my female employees and improved their confidence.”  “My accountant is learning (to be a) CPA. She wasn’t learning it before so you see she is advancing.”  “They have affected positively like right now we have a dress code for women where they cover their head, and we give them aprons so that when they are dealing with the foods there is hygiene.”  “They (SAFE) have helped us so that we give them maternity leave, and when they are on leave, they should be paid.”  “Yes and also maybe you know they have families so we give them sick leave whenever they are having issues with their families, like when the kids are sick.” Although not directly attributed to SAFE, some adjustments to SGB’s thinking in terms of hiring women were noted:  “We should get more women because they have softer fingers in (that kind of) operation. Even now, my thinking is that as we progress, we will target to get more of these women. There is also the passion to just assist since some of them have no husbands.”  “We have done our experiments by hiring them (women), and they are doing well so we would like to increase the number. Once the operation is doing better, we will get more of them since we get more reliability.” In one case, however, an SGB did begin taking steps to actively improve opportunities for women as a result of SAFE:  “There is a gender assessment evaluation that we did and that one was geared towards encouraging management to focus on giving women an opportunity to be part of our dairy. In that gender assessment, we did look at ways that we can promote women to participate. So far in our terms of service, we do offer maternity leave to women for three months. That is 90 working days to encourage them to be here. We also give a compassionate leave so that they are able to attend to some family commitments, and management has always expressed willingness to support women. We do accept membership of women to our society.” Greater focus on the identification and recruitment of women-owned businesses is the most significant development in the SAFE gender inclusion and women’s empowerment strategy. This emphasis further ensures that SAFE is meeting the program performance targets, including targets set for women-owned businesses and women participants in SWTs. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 50 The change in the recruiting strategy began in early 2015. Although SAFE continued to use newspaper adverts on upcoming SWTs (this was the avenue by which several new women-owned SGBs were recruited into the program), it became more proactive in identifying women-owned food processing enterprises through professional networks, notably development partners, such as USAID’s FtF, GAIN, women’s business associations, and financial institutions. The second component to the recruiting strategy was to broaden the net geographically (e.g., Buffalo Mills in Zambia) and place a greater focus on food processing activities, such as milling. Once promising candidates were identified, SAFE went on-site to the processing facility. Since the baseline evaluation, 16 new women-owned SGBs have entered into a formal agreement with the SAFE program, including eight Kenya, four in Zambia, and four in Malawi. Of these, four in Kenya were fully women-owned, and three in Zambia were fully women-owned; the rest were partially owned by women. Overall, 28 current SGB clients are either fully women-owned (12) or partially women-owned (16). The relative breakdowns of fully-owned vs. partially owned in the three countries are six versus eight in Kenya, five versus three in Zambia, and one versus five in Malawi. Conclusions Overall, SAFE has not been associated with leadership and advancement of women within its client SGBs. Its primary contribution has been in TA and product development. Socially progressive enterprises that were women-owned or -managed from the outset and already employed a significant number of women workers (e.g., Project Peanut Butter, GAEA and Classic Foods) were ready and able to advance women as their businesses expanded with the technical other enterprise assistance from SAFE. The workplace environment of the socially progressive enterprises was already supportive of women, and most notably the communications lines between management and unskilled and administrative workers were open and transparent. Using networks rather than relying heavily on mass media outlets to identify and recruit women-owned food-processing enterprises has achieved one of the program goals—increasing the number and ratio of women within the SAFE thee country client portfolio. However, most of the women-owned enterprises are micro or small, and some are not yet formally registered in their local district. This outcome, a programmatic constraint, is further addressed under the gender strategy and access to financing. The greatest impact on women-owned enterprises from the revised strategy has been in its advisement on management practices around women’s issues and work place environment, such as maternity leave and changing rooms. The enterprises have also expressed their appreciation of one-on-one interaction with SAFE staff, especially female staff members who have made visits to the enterprise and continue to support, and in a sense, mentor the women business owners. Since the baseline evaluation, SAFE has internally adopted TechnoServe’s Good Project Standards assessment tool. Enhancements include occasional internal trainings of TNS program staff on gender inclusion and economic empowerment, and assigning a Nairobi-based SAFE staff member as the gender focal person for the overall SAFE program. The gender focal person provides about 25% of her level of effort on gender issues and tools, such as the BAT that now incorporates human resources and gender issues in the onboarding of new SGBs. The gender focal person also represents SAFE at the TechnoServe quarterly Gender Practice Group Forums, which is conducted remotely with gender focal persons or full￾time advisors from other TNS programs in East Africa. SAFE has shared its lessons on reaching out to women in business and advising on management practices, such as women’s changing rooms, maternity leave, policies for promotion and equal pay for equal work for employees and casual workers who are mostly women. In 2016 a measure of gender responsiveness will be added to all relevant SAFE staff performance reviews. SAFE’s overall approach is to incorporate gender issues from the client’s perspective and needs. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 51 Recommendations SAFE Recommendations  SAFE should continue efforts to recruit women-owned food processing enterprises in the final year of the program, since women owned enterprises, and those that are socially progressive, are most likely to be advocates to women’s leadership and advancement opportunities. A separate initiative could be to hold a sector-wide forum in the three countries on women’s leadership. The purpose would be to motivate and network women in the food processing sector and identify resources for leadership trainings and development. Opportunities for mentoring and role modeling could be presented. In Ethiopia, USAID has supported the development of the Women in Agribusiness Leadership Network to support the advancement of women. The USAID 2017 solicitation for continuation of SAFE could specify that women’s leadership/management advancement be specifically addressed and initiatives developed.  These smaller women-owned enterprises need a full suite of business and technical capacity building, especially in business management and financing. If SAFE does not have the staffing resources needed for these labor-intensive inputs, it can strengthen linkages with existing BDS providers that offer affordable business management services.  To the extent possible continue site visits and one-on-one follow-up communications with the women business owners. Make introductions to and facilitate linkages with financial institutions and other sources of credit to help mitigate the financial constraints that most of the enterprises are facing. The gender focal person for the three countries covered in this evaluation should continue to share best practices from the TNS quarterly Gender Practice Group forums, especially on reaching out to women in gender sensitive business management and financing.  A gender strategy and focus for women’s inclusion and empowerment must be in the initial program design. This is recommended for the 2017 solicitation. Based on the evaluation research, women are usually overburdened with paid and unpaid work and have limited free time for group memberships and trainings off hours. This is a constraint to capacity building for leadership and management roles. Low literacy is another significant constraint. The development of supports, such as mentors, at the enterprise is one approach. Literacy sessions could be offered at the lunch break. For women to advance, opportunities and constraints need to be recognized and planned out at the initial design phase. General Recommendations  As with other direct or indirect program benefits, gender impacts are far more likely to occur as a result of as a set of planned activities than they are to occur organically. To the extent BFS desires to produce gender impacts in similar programs, it requires a concerted and consistently implemented strategy, something that has not occurred to date at SAFE.  In the context of business support programs such as SAFE, the gender strategy for women’s economic empowerment should address increasing women’s agency in business management, decision making, negotiating and access to markets, while also addressing the socio-cultural context in which women operate. Understanding the socio-cultural context will provide insights into changing behaviors with positive socio-economic outcomes and in taking up new practices. Access to information and skills development for enterprise management and line workers needs to be envisioned and Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 52 articulated. Resources for developing women’s leadership in civic and business environments need to be facilitated.  BFS programming should reflect adequate budgetary resources, and encourage new or expanded programs, for recruiting technical and market development professionals who already possess experience in gender inclusiveness in program design and implementation. Gender focal persons are generally stretched too thinly across a broad spectrum of activities or a large geographic area. An example would be to recruit women intervention managers with a technical or business background who have previous experience working with women entrepreneurs or other female beneficiaries. EQ8. TO WHAT EXTENT HAS SAFE LED TO INCREASED INCOME GENERATION AMONG WOMEN EMPLOYEES OF SAFE-ASSISTED FOOD-PROCESSING FIRMS? Like the baseline evaluation, the endline evaluation did not demonstrate or provide new evidence that SAFE led to increased income generation among women employees. SAFE management’s comment in the baseline, with respect to this evaluation question, was that it was not a surprise that there was no linkage of SAFE support to increase in wages. “Management decision-making on allocation of business resources to salaries is beyond out sphere of influence.”23 Further, they state, they have no deliverables (outcomes) on incremental increases in incomes due to SAFE support. In the view of the evaluator, getting good measurements on the incremental increases in incomes of SGB employees (women or men) would be very difficult to collect and verify and would entail additional reporting on the SAFE clients, many of whom say they are already overburdened. Conclusions SAFE has not been directly associated with increases in income of female workers within its client enterprises. Its primary contribution has been in technical and product development. TNS has rightly stated that it is beyond their sphere of influence. Recommendations SAFE Recommendations  Even given that increasing the income of female workers at SAFE-supported SGBs is beyond SAFE’s sphere of influence, this should not deter SAFE from encouraging, supporting and providing a platform for capacity building of SGB staff (women and men) that, if well managed, can lead to greater productivity, growth and profitability in the enterprise and thus indirectly to the financial remuneration of its employees. SAFE can, moreover, contribute to advise its clients on promotion and advancement criteria, policies and benchmarks as part of the human resource policies inventoried at the initial stages of client engagement with SAFE. General Recommendations  While increased employee income is a laudable objective in technical support programs like SAFE (in which the technical support is provided at the level of the firm), it is more likely to be a second-to- 23 SAFE Evaluation-Management Response, undated, page 10. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 53 third-order result making it challenging to measure and even more challenging to achieve and even harder to attribute to program support. Unless such a technical support program has specific strategies/ initiatives to promote increased income among the supported firms’ employees, we see little purpose to making this an objective of BFS funding and/or holding the implementing agency accountable for it. EQ9. WAS THE SAFE PROGRAM IMPLEMENTED AS DESIGNED? Both TNS and PFS implemented a number of new initiatives in the interim from the baseline to the endline. Some of these initiatives had begun, or were in the works prior to or at the time of the baseline’ others were launched since the baseline. These new initiatives reflect a broader conclusion about SAFE: SAFE is an evolving program. The challenges inherent in the SAFE model (distance TA by international technical experts, coupled with on-site assistance by local program staff) mean that the program must continually find new and better ways to confront these challenges in order to remain effective. Fortunately for SAFE, both TNS and PFS have demonstrated a willingness and ability to be proactive and creative in addressing these challenges. This process is abetted, moreover, by what appears by all accounts to be a productive and (for the most part) harmonious collaborative relationship between the two. The result is that SAFE, and the SAFE model, is evolving. This evolution includes the SAFE model itself, how this model is administered by its implementing partners, including the SGB identification and selection (onboarding) process, the division of labor between TNS and PFS, the types of services being provided to SAFE clients and to the broader industry, SAFE’s performance monitoring approach, and new strategic thinking about how to achieve SAFE’s higher-level objectives. Specific initiatives referred to above that have been undertaken, or are being continued, by SAFE since the baseline include the following:  Improved SGB onboarding process  Quick wins  Ask an expert  Standardized charters  Industry-wide learning manuals  Impact assessment methodology  Increased focus on reaching low-income, food insecure consumers  Increased full-time support staff at PFS Each of these initiatives is discussed below. Improved SGB Onboarding Process At the time of the baseline, SAFE had recently begun to implement a new SGB onboarding process. The primary challenge SAFE faces here is to identify the ‘right’ candidate for SAFE assistance. According to TNS, the right candidate is one who is dynamic and entrepreneurial and who has a vision for his/her business. As the success of the SAFE model relies heavily on ‘pull’ from the supported SGBs (where pull refers to proactive engagement by the SGB), the recruitment and onboarding process is a critical step by which SAFE screens in those clients who are most likely to demonstrate the kind of pull necessary to make the model work. The previous onboarding process relied heavily on mass media (primarily newspaper) recruitment methods. The idea was to cast a wide net and haul in the best catch, as it were. While this method was effective in soliciting good candidates for program assistance, it also brought in a large number of unqualified or opportunistic applicants, many of which were looking for handouts of subsidized assistance. With such a large number of applicants, moreover, it hampered SAFE’s ability to undertake the type of in- Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 54 depth vetting process required to identify the candidates with the right profile. The previous onboarding process also relied heavily on one-on-one interactions between local TNS staff and the SGB, which created inefficiencies, stressed the local TNS staff, and missed opportunities for group socialization. Beginning with the 2015 recruitment wave,24 SAFE implemented a new onboarding process, which it had specifically developed to address the deficiencies in the previous onboarding process. Over the course of its operations, SAFE has developed a reasonably extensive network of food processors and other sector stakeholders in each of its program countries. Thus SAFE decided to leverage this network to identify candidates for program assistance. While SAFE continued to advertise via mass media outlets, it put a much greater emphasis on working through its network of contacts (e.g., SWT participants, Kenya Association of Manufacturers, Zambia Development Agency, Bureau of Standards, etc.) to identify and recruit potential candidates. (SGBs interviewed for the endline came into contact with SAFE via a variety of means, including newspaper ads, pre-existing relationships with TNS, or attendance at sector-wide trainings such that no single method dominated among respondents). Once candidates have been identified, the local TNS staff undertakes a thorough vetting process that includes the following:  The local team meets to review each candidate and his/her qualifications.  The local team conducts two sites visits, one to see the business and meet with the management team, and another to visit the factory and conduct a factory assessment. During the two site visits, the TNS team scores the SGB using a formal scoring sheet.  After the initial site visits, the SGB submits a Concept Note, which explains who it is, what is doing, what its challenges are, and what it wants to accomplish with SAFE assistance.  After the submission of the Concept Note, TNS culls the candidate list further and shortlists a group of SGBs to proceed further.  For those shortlisted SGBs, TNS staff conducts two additional site visits to administer a Quality Audit and a Business Assessment.  Finally, after the completion of the Quality Audit and Business Assessment, TNS makes its final selection of SGBs to be onboarded into the program. Before the SGBs have been officially onboarded, however, SAFE convenes a ‘familiarization meeting,’ attended by the new recruits, by representatives from TNS and PFS, and by current or past SAFE￾supported SGBs. At this meeting, SAFE, among other things describes the program, defines the roles and responsibilities of participating parties, explains the type of client information wanted by the program and when and answers questions from the new recruits. SAFE also gives time to the current or past SAFE SGBs to talk about their experience with the program and what the new recruits can expect. From SAFE’s perspective, the familiarization meeting serves as an effective icebreaker and important socialization tool for the new recruits, plus it offers a more efficient way to communicate critical information about the program to the new SGBs than the one-on-one methods relied on in the past. At the conclusion of the familiarization meeting, the recruits and SAFE sign a client agreement. The recruits are then given two weeks to review the client agreement and make a final decision about joining the program. 24 SAFE undertakes a single, large recruitment wave each year during January-March. Most SGBs that join the program in any given year are recruited during this recruitment wave. However, SAFE will undertake small recruitment efforts during the course of the year to replace SGBs who have dropped out of the program. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 55  “At the meeting, they were telling us about reporting, what reports we will each be submitting, information we were supposed to provide and that it would all be in confidence, monitoring the progress of the program and such things.”  “We were told that we had been successful and then we were called for a meeting with the other applicants where we were then briefed on what would be the outcomes, how we were supposed to report, what the SAFE was and TechnoServe all about, and after we signed some agreement. And then the program started.” With the new process in place, TNS estimates that of 51 applicants in Kenya during the last recruitment round in 2015, 17 were shortlisted. By comparison, in Zambia, of 28 applicants, 7 were shortlisted. During interviews with the supported SGB, evaluators asked them to comment on the quality of information provided them by SAFE during the onboarding process. Overall, the SGBs responded that the purpose and scope of program was well explained and that TNS did a good job explaining the program and answering questions and that the communication process was, for the most part, clear and open.  “They were very open and they were available for any inquiries where we were not clear. And they gave us an opportunity to decline if we didn’t like anything about the program.”  “First of all, before you deal with anybody in terms of what we are doing with TechnoServe we wanted to know who are these people because the information we were sharing with them was about our company and some of it is confidential. So first we wanted to know the details of TechnoServe and they actually gave us their history and told us how they have worked with so many companies like ours within this region, so all I can say is that they gave us enough information about themselves.”  “They confirmed out expectations that if they came in, they would assist us on plan design, acquiring the right equipment for processing, accessing and setting up procedures to use in various areas when it comes to good processing, and assist our technical staff with more training.” SGB respondents further noted that the onboarding process was in-depth, detailed and time consuming, leading a few to complain about its length, although this was not a widely shared sentiment.  “The process was rather slow in that everything is done by telephone and discussion and how they enrol us. It is so difficult; it’s just a slow process where as if you had a consultant who would come and work with us maybe two months it’s done fast they say the results quickly.”  “What we didn’t like about the process are the phone calls and the response, they used to take a lot of time in responding to issues … ok not even issues because you know it takes a lot of your time when doing that so the process was very tedious … because they gave us a very big form to fill online so I can say it was a bit tedious.” As a means of corroborating the comments in the process KIIs, the food processor survey asked those SGBs who participated in the new onboarding process to rate the process—including the recruitment process, information provided about SAFE, familiarization meeting, and initial business assessment—on a scale of 1-4 where 1 equals ‘very dissatisfied’ and 4 equals ‘very satisfied.’ As seen in Table 17, the respondents in all three countries consistently said that they were ‘satisfied’ with each aspect of the process. When asked to rate the overall usefulness of the initial SAFE assessment (Quality Audit and Business Assessment) on a similar 4-point scale, respondents in Kenya and Zambia (no response was available from Malawi) gave even higher ratings approaching a 4 or ‘very useful’ in both countries (Table 18). Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 56 Table 17: SGB Satisfaction with Revised SAFE Onboarding Process Step in Onboarding Process Kenya Zambia Malawi Total Recruitment process 3.1 3.7 3.1 3.2 Information provided about SAFE 3.3 3.5 3.3 3.3 Familiarization meeting 3.2 3.3 3.1 3.2 Initial business assessment 3.0 3.3 3.0 3.1 Process to identify quick win projects 2.8 3.0 3.3 3.0 Selection of quick win projects 2.9 3.3 3.4 3.1 TNS assistance on quick win projects 3.2 3.3 3.1 3.2 Outcomes of quick win projects 3.0 3.0 3.0 3.0 How satisfied were you with: 1=Very dissatisfied; 2=Not satisfied; 3=Satisfied; 4=Very satisfied Table 18: Usefulness of Quality Audit and Business Assessment Kenya Zambia Malawi Total 3.5 3.6 N/A 3.5 How useful was the initial SAFE assessment of your company: 1=Not at all useful; 2=Not very useful; 3=Somewhat useful; 4=Very useful Expanded SGB Client Services Since the baseline evaluation, SAFE has either launched or is in the process of ramping up three new SGB clients services: quick wins, ask-an-expert, and standardized charters. Quick Wins Perhaps the most important adaptation to the SAFE model to have occurred since its inception is the introduction of the quick win charter with the new crop of recruits onboarded during 2015. Quick win charters are smaller, simpler projects at client SGBs, which are implemented, or begun, by the local TNS staff with the client SGB during the interim between the SGB onboarding and placement of the PFS volunteer. The primary motivation for the quick win projects was the long lag time—up to 90 days—to finalize charters and recruit and onboard the PFS volunteers. During this time, TNS observed that the project momentum often stalled and client SGBs became demotivated thus putting stress on SAFE’s relationship with the client. The idea behind the quick win charter is to get the SAFE-client relationship off to a running start by quickly identifying and addressing issues—related to both production/operational processes and administrative/managerial processes—at the client SGB that can be addressed in the immediate to short￾term using local TNS staff. Drawing on the findings of the Quality and Business Assessments conducted during the onboarding process, local TNS staff work with the SGB to identify and charter projects addressing technical or business gaps that fall under the experience/knowledge of the local TNS staff and that lend themselves to relatively quick resolution. Quick win project cover a wide range of areas, including physical plant, manufacturing processes, financial management, human resources, signage, hygiene practices, or process documentation. The following quotes demonstrate the types and range of quick wins implemented. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 57  “Good manufacturing practices, for example the chemical substances you have got to know that there is food grade soap for using to clean the utensils. Introduction of certain records like the logbook where people have to enter the main food production area. In the past people would just go in.”  “Just to close up some gaps, like the windows were open, there was no protection from theft in and out, and then also the door in most cases it would be open. We needed to put something to protect at the door. And then also the sign writing for processes on the walls was not very adequate and we had to put it there.”  “Documentation of processes, and when I say documentation, if you’re making yoghurt, how much yoghurt have you made, you know, basically everything that happens in the factory should be documented because it enables you to know eve when you have a challenge you know where the problem is coming in from, formulation of processes and giving them to the employees so that each and every person knows what they are supposed to do.”  “They advised us on what to do and what to check so that we collect the right milk. They also assisted us in the tests that we did in the lab. They also gave us lab procedures we need to be using when we are doing the milk testing. They also gave us plan designs and regulations like we are not supposed to leave the place open to reduce the pests coming in. They advised us on the drainage of the plant and how they should be separated, the milk reception area and the processing area and how they should be separated.” Some quick wins have a more marginal impact on the SGB’s operations and outcomes, while others, despite their relatively small size, can have a significant impact on operations and outcomes.  “The impact on the business was quite substantial and being in the rural set up, there are certain things which you start doing that people don’t understand. Like when you tell them you cannot shake hands anytime, you cannot go out and do anything that you want anytime. When you come in with clothes from home, you have to take them off, and you have to wear this, or maybe wear the protective clothing and your work suits, you protect your hair like that. I think that was some of the knowledge that helped because when it comes from outside, staff will listen, other than just you.”  “The quick wins have helped very much. I mean am now able to know like ok this is how the flow of production is supposed to be this is how the sales and the records and record keeping should be.”  “We have improved in areas of getting good quality milk from farmers because when you perform the right tests at the reception and make the follow ups at the lab, we have been able to get good quality milk.”  “It has improved on production because you see once time is not wasted; the production takes the shortest time possible.” The processor survey asked those SGBs who had participated in quick wins to rate their satisfaction with different aspects of the quick win process—identification of quick wins, selection of quick wins, TNS assistance with quick wins and outcomes of quick wins—using a 4-point scale in which 1 equals ‘very dissatisfied’ and 4 equals ‘very satisfied.’ As seen in Table 19, respondents consistently said they were ‘satisfied’ with the different aspects of the quick wins process. Table 19: SGB Satisfaction with Quick Win Projects Aspects of Quick Win Process Kenya Zambia Malawi Total Process to identify quick win projects 2.8 3.0 3.3 3.0 Selection of quick win projects 2.9 3.3 3.4 3.1 TNS assistance on quick win projects 3.2 3.3 3.1 3.2 Outcomes of quick win projects 3.0 3.0 3.0 3.0 How satisfied were you with: 1=Very dissatisfied; 2=Not satisfied; 3=Satisfied; 4=Very satisfied Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 58 The same respondents were next asked to rate the impact of the quick win projects on their business in terms of labour costs, production efficiency, product quality, compliance with standards, sales, and income, this time using a 3-point scale in which 1 equals ‘no impact’ and 3 equals ‘significant impact’ (see Table 20). Overall, the responses indicate ‘some impact’ albeit with greater variation across both areas and countries. Respondents rated the impact highest in terms of product quality and compliance with regulations and standards and the lowest in terms of labor costs. Within the countries, respondents from Kenya rated the impact consistently close to ‘some impact’ across all categories. In contrast, respondents from Zambia rated the impact closer to ‘significant impact’ across the board, with the exception of labor costs, while in Malawi respondents rated the impact on product quality and compliance close to ‘significant impact’ and rated the impact on the other categories somewhere between ‘no impact’ and ‘some impact.’ Table 20: Estimated Impact of Quick Win Projects Impact on Business Kenya Zambia Malawi Total Labor costs 2.1 2.0 1.3 1.8 Production efficiency 2.3 3.0 1.7 2.2 Product quality 2.3 2.7 2.6 2.5 Compliance with regulations and standards 2.2 3.0 2.9 2.6 Sales 2.1 2.7 1.5 2.0 Income 2.3 2.7 1.5 2.1 What impact did the quick wins projects have on: 1=No impact; 2=Some impact; 3=Significant impact Quick wins represent an important adaptation to the PFS model given its initial, and nearly exclusive, focus on delivering distance-based technical assistance by international experts with local staff acting as intermediaries between the client and the PFS volunteer. This focus made operational sense, since PFS was, at the time (and this is changing, see below) predominantly volunteer organization, and its primary asset was, and remains, the volunteers coming from its corporate partners. At the end of the day, if PFS is not fulfilling the demand among its corporate partners to place volunteers in distance-based TA roles, it is not fulfilling its mission. The quick win charters thus represent a significant shift in strategy at PFS (and SAFE), which has required PFS to rethink and adapt its basic operational model. To PFS’ credit, it had the flexibility to recognize the purpose and value-added of the quick win charters and to make the necessary adjustments to its model. The adoption of the quick win charters at SAFE, and the factors that led to their adoption, demonstrate a fundamental tension that exists in the PFS model: distance-based versus local assistance to address constraints within African food enterprises. The distance-based TA model is one of many potential approaches to address these constraints. To be sure, one that offers certain comparative advantages (e.g., access to international experts and expertise not available locally) but that also has certain comparative weaknesses (e.g., challenges related to distance, time lags and communications). Another approach is to provide local-based TA using local (including regional) technical experts. Yet a third approach is to use some combination of the previous two. To an extent, SAFE began as a pre-existing solution (distance-based TA) applied to address capacity shortcomings among African food processors. SAFE was, in essence, a large-scale test as to the effectiveness of the PFS distance-based TA model. Over time, however, experience in implementing this model has revealed a number of inherent challenges both in the model itself and in implementing the model. Rather than continuing to insist on implementing the model as is, both PFS and TNS have recognized the need to adapt the model, which they have done, and which they continue to do. The Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 59 adoption of the quick win charter is, as mentioned above, perhaps the most significant adaptation made to date. Having now adopted the quick win charter, however, a more fundamental question facing TNS and PFS is to what extent they should continue to adapt the model to allow for local solutions to capacity issues where they are both appropriate and available? At the end of the day, the primary objective is to build the capacity of local food processors to produce and market nutritious foods. If this can be done using local experts and local solutions (assuming greater efficiency and lower cost), then is this not a better solution than superimposing a distance-based approach where it may not be required? The following quotes by key informants at PFS aptly convey this sentiment:  “In the beginning, we felt like most problems could be turned into charters, but we found early on that what SGBs need is answers to specific questions. We want to provide value early on to build credibility and meet SGB needs. So, if TNS can address a problem, then it should do so. If it can’t, then bring on PFS. We believe that there will be many projects that are too complex for a quick win, but not every project needs to be chartered. We want to find different ways to deliver expertise as soon as we can and with high quality.”  “To me, it always made the most send in the world. Initially, had the default paradigm that volunteers provide technical support, while TNS acted as intermediary. We realized, however, that TNS staff has skills of their own. We don’t need to involve volunteers to give basic information. The simple insight is that it can be a bit flexible. I’m not sure whether there was an unwritten rule or perception over time that TNS had liberty to do these things on their own; this was a barrier to delivering value to clients, and we eventually perceived that. So in hindsight, it’s a no brainer.” Key informants at TNS see potential for further scaling-up of the quick win projects so that they comprise up to 60% of all projects implemented by SAFE. When SAFE food technologists and business advisors were asked how many of the current PFS charters they felt could be addressed locally, either by TNS staff alone or by local experts, they gave varying answers, ranging from 75% in Kenya to 40% in Zambia. In the former case, the TNS staff felt that they could themselves address most of the current charters, and those they could not address, they felt there was sufficient expertise within Kenya to address the remainder (up to the 75%). In contrast, the TNS staff in Zambia felt like they could only address about 10% of the current charters on their own, while local experts could address the remainder (up to the 40%). Clearly, then, the capacity to provide local solutions to food processor capacity constraints depends on the availability of local expertise within the country. Nonetheless, the local TNS staff also emphasized that there are certain charters that naturally lend themselves to the distance-based TA that PFS offers due to the lack of local expertise. These include charters related to different areas of marketing—such as marketing strategy, branding, labelling, and packaging—product formulation, research and development, fortification, software design, canning, baby food formulation, selecting equipment, and, in Zambia, plant design. The dairy sector in Kenya is, moreover, well advanced and TNS feels that there is ample local expertise in Kenya to address constraints related to dairy processors. To address this issue, TNS proposes a pyramid approach that postulates three types of SGBs needing three types of technical assistance. At the bottom of the pyramid are smaller, less advanced SGBs that need mostly quick win projects or standardized charters, which require mostly local TA. Most SGBs/projects would fall here. In the middle of the pyramid are larger, more sophisticated SGBs that need a combination of quick wins and more advanced charters requiring both local and international expertise. Finally, at the top of the pyramid (representing the fewest number of projects) are the largest, most Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 60 sophisticated SGBs that have complex problems requiring complex solutions, which can best be provided via distance-based TA with international experts (see Figure 2). Figure 2. Project Pyramid The logic of the SAFE evolutionary process would seem to be pushing it toward a model that is something like the pyramid proposed by TNS offering a combination of local and international solutions according to the needs of the supported SGBs. If this is true, it implies an increasing tension between the organizational imperatives of PFS (e.g., recruit and place its corporate volunteers) and the specific needs of the supported food processors, which may or may not require the expertise of a General Mills, Carghill, or DSM, as the following quotes from PFS informants illustrate:  “Most important is the number of volunteers getting engaged. If we can come up with ways that volunteers come in, even on the basis of foundational work done by TNS, it can shorter the time to work on projects. But what is important is that volunteers are being used.”  “I would be open to the idea of TNS doing charters. However, the purpose of PFS is to employ volunteers remotely. I think if we have TNS do more and more work, it is defeating the purpose of technology transfer from volunteers to clients.” - Largest, most sophistacted SGBs - Advanced Charters - Larger, more sophisticated SGBs - Quick wins and avdanced charters - Smaller, less advanced SGBs - Quick wins and standardized charters Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 61 There is also the risk that, if more autonomy is handed over to TNS (or whoever PFS’ local implementing partner is), that their comparative lack of technological sophistication (where it occurs) may produce sub￾optimal results for the SGB clients and, importantly from PFS’ perspectives, a bad experience for its volunteers, which has downstream implications for the viability of the PFS model.  “I do think that we will have to be mindful going forward. If TNS frames the project in a way that’s not workable, but PFS later finds out that the client is not ready for this, then this ends up being a bad experience for the volunteers. This is the risk from PFS giving more latitude, but this is better than wasting 90 days because we didn’t give TNS latitude to make decisions on its own.” For the same reason, it also implies natural limits to the extent that SAFE (and particularly PFS) can be expected to adapt the PFS model. Managing this tension, while continuing to find ways to adapt within the general parameters of the PFS model, stand out as one of the greatest challenges for SAFE moving forward, both under the current agreement and under any SAFE extension. It also has important implications for future BFS programming, as this tension needs to be addressed upfront in any future program design. Ask an Expert SAFE offers three primary types of TA to its clients: quick wins, PFS charters and ask-an-expert. Ask-an￾expert is the least formal method of TA delivered by SAFE and involves one-one-one communication between supported SGBs and PFS volunteers to address issues that do not require a charter and which can be resolved over the space of a few phones calls or emails. Ask-an-expert has existed for some time now, but until recently, only operated on an informal basis. PFS found that many of the SAFE SGBs were contacting volunteers on the side to ask follow-up questions about ongoing or completed charters or to ask about issues not covered by their charters, which could not be addressed by the local TNS staff. Rather than allow this practice to continue informally, TNS and PFS saw the value in formalizing and launching the service as a featured form of TA offered to clients. To formalize ask-an-expert, PFS recruited a couple volunteers dedicated to its operation, has created a formal operational procedures for it to document the issues addressed and how they were addressed and has provided training to local TNS staff on when and how to use the service. Ask-an-expert is offered to both current and graduated SGBs. Standardized Charters SAFE was originally designed, in the words of a PFS informant, as an “all you can eat buffet,” meaning that TA was customized to the specific needs of each SGB. The presumption was that for each client, SAFE would have an approximately two-year relationship and with whom it might do 3-5 customized projects, each with its own project teams. As a result, SAFE tended to try to set up project teams and launch the project in a similar fashion regardless of the type of project. After operating this way during the early stages of the program, it became apparent that there were certain commonalities among supported SGBs, and particularly the smaller, less sophisticated ones, many of which needed basic assistance in areas such as quality or financial management. In such cases, SAFE has come to come to conclude that a better approach may be a “menu approach” in which SGBs can choose from a set of common ‘solutions’ to basic problems. According on a PFS key informant, “in the old days, we would have had teams client by client and bring together teams of volunteers to work simultaneously with multiple clients with the same basic needs, and with little customization.” In contrast, under a menu approach, SAFE offers a standardized approach to resolve a common problem and, where appropriate, a team of local TNS staff and/or volunteers can work with a set of SGBs to address a the problem. An example of this menu approach in practice occurred recently in Malawi in which SAFE worked with several of the recently onboarded SGBs to obtain MS21 certification. In order to sell their products, Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 62 Malawian food processors must be certified by the Malawi Bureau of Standards (MBS) that they meet a minimum quality standard (MS21). MS21 certification covers basic plan set-up and hygiene required for a food processor to operate. Without the certification, MBS will seize the processors’ products from sales sites and destroy them. Obtaining MS21 is a necessary condition to facilitate the growth of a food processing business. According to a TNS staff member in Malawi,  “We work directly with processors to address ‘smaller’ constraints that do not require PFS volunteer assistance. These are the quick wins that let us engage the processors earlier than previously. We developed strategy to support processors to obtain Malawi Bureau of Standards (MBS) certification (MS21). All the new projects are quick wins and finance and management. These are always implemented by local SAFE staff. This always starts with an audit. We developed this as the quick wins strategy starting up compacts with PFS is slow, and we felt we needed to move quicker. Obtaining the MS21 certification facilitates growth of the businesses.” The TNS team in Malawi is concentrating its quick win strategy on working with processors to obtain the MS21 certification. (The TNS food technologist in Malawi, Lawrence Muhamba, is a former MBS employee.) Of the six new SGBs SAFE is working with in Malawi, SAFE is providing five of them MS21 support. Two of the assisted SGBs have now obtained MS21 due to SAFE support, while a third is expected to obtain MS21 certification once it receives the necessary machinery. TNS in Malawi has developed a MS21 audit template, which can be used going forward, as this is expected to become a recurring issue to the extent SAFE continues to onboard new SGBs. PFS is in the process of thinking through which types of projects might be included under the standard charter umbrella, according to one PSF informant, “For standardization, we don’t have an official list yet. There will be a meeting with TNS staff in the next month to talk about this; for sure, quality management and financial management, but probably also business planning. There could also be others.” Industry-Wide Learning Manuals Information on the industry-wide learning manuals is presented along with the findings related to EQ3. Impact Assessment Information on SAFE’s new impact assessment methodology to estimate incremental volume produced and incremental net profit earned by supported SGBs is presented along with the findings related to EQ1. Increased Focus on Reaching Low-Income, Food Insecure Consumers Information on the increased program focus on reaching low-income, food insecure consumers is presented along with the findings related to EQ5 and EQ6. Increased Full-Time Support Staff at PFS Since the baseline, PFS has added several new full-time staff and is in the process of adding yet more, either to take on roles not previously staffed within the organization or to replace existing staff that performed the role on a voluntary or part-time basis. Important new full-time staff positions include those described below. Director of Field Programs and Impact: This person is responsible for overseeing the quality and implementation of the PFS service portfolio to its client SGBs (e.g., reviewing charters and making sure they are actionable, defining client needs so as to match them with suitable volunteers, and making sure charters have SMART indicators and measureable objectives) in addition to developing methods and Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 63 systems for measuring PFS results and overseeing their accurate and timely capture. To this point, PFS has relied heavily on the performance data collected and reported by TNS—of the 17 indicators currently tracked by PFS, 15 of them are TNS indicators—but with the hiring of the Director of Field Programs and Impact, PFS is signaling its intention to invest greater time and resources in measuring its results without relying so much on TNS. According to one PFS informant,  “There is clearly a desire from the Board to move beyond output to outcome and impact. These conversations keep coming up, and we are very keen to develop a framework with field partners and internally about how best to address this issue. We are becoming less satisfied with the ‘number of SGBs’? We would be better served to focus on quality and not quantity, but what is quality? We are just beginning to discuss it, so I don’t know how it will go.” PFS faces significant challenges in coming up with credible outcome and impact measures, not only because of the technical challenges in defining such measures but also because of its lean on-the-ground presence, which makes data collection challenging as well: “We are limited in size in that we do not have a ground presence. We’d like to do our own measurement, but we are a long way from that.” Toward this end, the Director of Field Programs and Impact is leading an initiative to, among other things, understand more about basic client information, to test SAFE assumptions about the characteristics of successful charters and develop a typology of successful SGBs, develop systems to gather more information about SGBs, validate performance data and do more in-depth analysis of that data and to standardize performance information across PFS corporate partners.25 This initiative, however, remains in its infancy and, as PFS concedes (echoing the point made repeatedly by PFS informants, as captured in the above quotes), “While we are in a better position to be part of a dialogue about what route to take, internally we’ve started these conversations, there’s a long way to go.” Training and Technology Specialist: This person is responsible for developing and delivering training to the PFS volunteers so that they understand their roles as volunteers and the expectations placed on them so that they are prepared to move as seamlessly as possible into their volunteer role. PFS has created separate training modules for volunteers, SGB leads, country leads, and field partners (TNS in this case), each covering multiple topics. (The PFS training curriculum for each of these four groups is presented in Annex 10: PFS Training Curriculum). Training is typically delivered virtually to small groups, or when required, on a one-on-one basis. At the moment, volunteers are not required to participate in the training, although participation is strongly recommended. PFS is currently considering, but has not decided, whether to make the volunteer training mandatory. In addition to formal training courses, the Training and Technology Specialist also delivers one-on-one coaching to volunteers and volunteer teams. PFS is further developing complementary training tools or applications, which both volunteers and SGB clients can access. For example, to address the cultural divide that volunteers inevitably encounter working with their African counterparts; PFS has created a program called ‘GlobeSmart,’ which allows the volunteers to log in and read about the business culture in other countries. PFS is also developing a library to document its work on as a resource for both volunteers and clients so they can see what has been done to address similar issues/problems that they are facing. For SGB clients, PFS is working on developing free online and/or smartphone applications, which clients can access to receive instruction on common problems, such as good manufacturing practices (GMP), 25 For example, one method PFS is investigating for standardizing performance information across its corporate partners is the IRIS Metrics (https://iris.thegiin.org/metrics). Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 64 finance or aflatoxin. PFS expects to rollout the first apps related to finance and GMP for beta testing in about a month and to finalize them in about six months’ time. Volunteer Profile and Salesforce Administrator: This person is responsible for overseeing the volunteer life cycle, which includes connecting volunteers to projects and ensuring that they have a satisfying experience and managing the Salesforce database, which is a repository of information about volunteers, projects and SGB clients. Program Associate: This person is responsible for creating for mapping internal PFS processes—so as to standardize processes (e.g., chartering, graduating clients) across PFS corporate partners and identifying problem areas—and for helping to manage the monthly project cycle process. PFS has hired a second Program Associate who will assume duties shortly. In addition to the above full-time staff positions, PFS has also hired a full-time Staff Assistant and is recruiting for a full-time Marketing and Communications Manager and a Volunteer Services Manager. The latter hire will significantly upgrade PFS’ volunteer management capacity, as it will allow one full-time staff member to focus on managing the ‘field-facing’ side of volunteer operations (e.g., volunteer technical service delivery) and another full-time staff member to focus on managing the ‘corporate-facing’ side (e.g., volunteer recruitment) of volunteer operations. Finally, PFS has also recently added a new volunteer position called the ‘Senior Consultant’ who is a senior technical expert with significant volunteer experience. The Senior Consultant is not assigned to any particular country or SGB but rather provides ad hoc guidance to the PFS team and its volunteers at different stages through the chartering and TA delivery process. An important motivation for increasing the full-time PFS staff is to relieve the administrative burden on volunteers so that they can focus on their core purpose: providing customized technical assistance to PFS client SGBs. This relieves the administrative burden in two ways. First, it frees volunteers from taking on the administrative duties related to managing PFS operations and its volunteer force, much of which was previously done by volunteers or by private contractors. Second, it reduces the time volunteers must spend to comply with PFS administrative requirements related to such things as documentation, data collection, form completion, and so forth. (For example, the Program Associate now has primary responsibility for entering and updating information into the Salesforce database, setting up meetings, preparing meeting materials, etc. thereby relieving volunteers from responsibility for these tasks.) The following quotes from PFS informants illustrate this point:  “We have been making many adaptations and changes. Now we have more staff; for example, a training specialist where we had a contractor before. The Director of Field Programs role is brand new, we are adding more junior staff. In the past, we relied mostly on volunteers to manage PFS. The opportunity cost was that we were having volunteers doing administrative work rather than working with clients. So we’ve added some new positions to restructure the head office to have more support resources to do project management so volunteers can work with clients.”  “When we started with SAFE, the person charged with volunteer recruitment was a retiree, a private contractor. But now we have two full-time staff managing the volunteer pipeline to help recruit and place volunteers faster. Too often the projects were slow to get started, to get volunteers in place. One this is done, the work goes faster. We want to recruit and place volunteers faster; we want volunteers to contribute time to clients rather than documentation and administrative work.” More generally, PFS’ purpose for beefing up its full-time staff is to transform PFS from a volunteer-led organization to an increasingly professional one, which, although it maintains a volunteer ethos, is Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 65 increasingly incorporating the private sector principles of efficiency and continuous improvement of its corporate partners. Processors’ Overall Perceptions of TNS and PFS There were no significant changes from the baseline to the endline in how the supported SGBs perceived SAFE in general and TNS and PFS in particular. This is seen in Table 21, where the interviewed SGBs were asked to rate local TNS staff and PFS volunteers on different performance dimensions using a 4-point scale. At the endline, the scores were as high or slightly higher in all cases. Overall, respondent SGBs rate TNS and PFS assistance as ‘good.’ Table 21: SGB Perceptions of TNS and PFS Indicator Kenya Zambia Malawi Total B E B E B E B E Knowledge of local TechnoServe staff 3.3 3.3 3.2 3.1 3.0 3.0 3.2 3.2 Helpfulness of local TechnoServe staff 3.5 3.4 2.9 3.0 2.0 3.0 3.1 3.2 Knowledge of PFS volunteers 3.1 3.0 3.1 3.1 4.0 3.7 3.2 3.2 Helpfulness of PFS volunteers 2.9 2.9 2.9 3.0 3.3 3.2 2.9 3.0 Assistance through Skype or other long-distance means 2.3 2.9 3.3 3.0 2.7 3.2 2.7 3.0 Please rate the following aspects of the SAFE program: 1=Very poor; 2=Poor; 3=Good; 4=Very good The processor survey next asked respondents SGBs to rate their satisfaction with different aspects of the service provided by TNS and PFS using a 4-point scale. Their responses are shown in Table 22. Continuing the pattern seen with similar questions, respondents from all three countries consistently said that they were satisfied with the process used to identify and assess projects (although they were least satisfied here), assistance received and outcomes of TNS assistance, and assistance received and outcomes of PFS assistance. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 66 Table 22: SGB Satisfaction with TNS and PFS Assistance Kenya Zambia Malawi Total Process used to identify and assess projects 2.8 2.9 3.0 2.8 Assistance received from TNS staff 3.1 2.9 3.3 3.1 Outcomes from TNS assistance 3.0 3.1 3.1 3.0 Assistance received from PFS volunteers 3.0 3.0 3.5 3.2 Outcomes from PFS assistance 3.0 3.0 3.2 3.1 Overall, how satisfied are you with: 1=Very dissatisfied; 2=Not satisfied; 3=Satisfied; 4=Very satisfied Finally, when asked to rate the overall usefulness of SAFE assistance using a similar 4-point scale (Table 23), the respondent SGBs in Kenya and Zambia consistently gave SAFE a rating midway between ‘useful’ and ‘very useful.’26 Comparing the results in Table 23 with those in Table 22, we find that the overall evaluation scores given the by supported SGBs in Table 23 are consistently higher than the scores found in Table 22. Notably, the SGBs gave very similar satisfaction ratings to TNS and PFS To understand this result, it should first be noted that the questions in the two tables are measuring different things. In Table 22, the questions are asking SGBs their satisfaction with different aspects of service delivery by TNS and PFS and their outcomes, while in Tables 23 the question is asking SGBs to rate the overall usefulness of the assistance received by SAFE in general. With that noted, it does appear that the SGBs rate the whole of their experience with SAFE more favorably than individual aspects of that experience delivered by TNS and PFS. Thus despite some apparent concerns about the assistance received from TNS and PFS and the outcomes of this assistance, overall SGBs found the assistance to be ‘very useful’ (or close to it). Table 23: Overall Usefulness of SAFE Assistance Kenya Zambia Malawi Total Assistance received from SAFE 3.5 3.6 N/A 3.5 Overall, how useful was the assistance you received through the program: 1=Not at all useful; 2=Not very useful; 3=Somewhat useful; 4=Very useful Perceived Benefits Received from TNS and PFS Assistance The largely positive perceptions of SAFE, TNS, and PFS found in Tables 21-23 were echoed in the processor interviews. The following quotes are illustrative of the numerous statements that interviewers heard from supported SGBs over the course of the endline interviews describing the benefits they perceived to have received from the assistance they received from both TNS and PFS. TNS  “They helped us understand our firm. They were like a sounding board. We had a business plan so they helped us check if our business plan was too ambitious in the processing line.” 26 Due to a miscommunication, an older version of the processor survey was used in Malawi, which did not include this question. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 67  “TechnoServe facilitated very well, they shared the information, they were clear, and they supported us if we needed to structure it but also in pushing us to get it done. So that facilitation by TechnoServe was good.”  “They were hands on, they came, looked at the plant, the equipment, they took samples, and all that. So really we can’t ask for anything to be better than that.”  “Understanding what we need to succeed in a retail market; it was well achieved, well-articulated, and well-explained to us.”  “They also gave us a lot of strategies on which direction we can go, because I had brought up the same story on marketing that we cannot advertise but they brought up ideas like you can do one to one, you can send or make a display of jams and give like the first lady or the minister of commerce and that kind of marketing of one to one does not cost much. So we were left with a very nice template with ideas on how we can take on marketing.”  “They are there to assess and also give us ideas on what we should do, so they are giving us a technical assessment point and technical support in terms of how we handle our maize from the store to the finished product…”  TechnoServe basically does all the follow-up. They basically act as an in-between, they actually do more of the follow-up with PFS, but we are also there to provide all our input and all our ideas to them.”  “The layout of the processing plant; they taught us how to position the raw materials; where to place the finished products. Generally, the flow of the operation went from point of receiving the raw materials to the packaging of end products and its storage or readiness for dispatch.” PFS  “The PFS guys know what we have done, at least they know the process of what we have done. They have pictures on what we have done and also they assess the strategic plan because we do it with TechnoServe direct and then we send it to them then they assess and then they correct how we should have done it better and how we should move on to audit.”  ”Its General Mill who searched for a machine in South Africa called a HIPPO. This is a machine that mills flour. They are the ones who searched for it, found the company that sells it, found out how much it costs and how much it would cost to bring it to Mombasa. That helped in determining exactly how much we would need to get that machine”  “They are the principle developers of the business plan, and we have been doing it online in that we were meeting online. We discussed and gave them our site; what is going on here; what we are doing and they have also been advising us and assuring us that what we have done will work. So in terms of providing us information and in terms of being able to analyze a situation and also being able to give recommendations, they have been sharing their experiences with us.”  “They suggested that maybe we are making them too fine so the solution is to change the sieve that is used. I found this very helpful because, without them, I would have cracked my head for long wondering what could be wrong.”  “They helped us a lot because we get quarterly inspections from the Zambian Bureau of Standards and because of the interventions from PFS we always pass. There are few complaints from the Bureau of Standards when they inspect because of what we learnt from the program.”  “We are mostly very satisfied with SAFE assistance. The PFS (Cargill) assistance on soymilk plant layout was very useful. It is their specialty.”  “We work by Skype with PFS volunteers (GM and Cargill, based in Belgium) on plant layout and expansion and quality systems, and on adapting accounting manuals.”  “Skype-based assistance was very useful. Skype was an opportunity to ask questions that were not as easy to ask by email.” Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 68  “We implemented quite a few projects with SAFE. Initially, General Mills helped with packaging. It seems that white carton was not as strong as the brown one. They (GM) helped with sourcing cheaper milk from Europe. They also helped on a feasibility study and a study of quality [GM identified the source of a contaminant]. They’re a large company, so the collaboration was a good match.” Issues with TNS and PFS Service Delivery Although the SGBs’ perceptions of TNS and PFS were largely positive, not all of the SGBs interviewed in the endline perceived to have benefitted from TNS or PFS assistance.  “It’s mixed I think, some have been good, some have not been as good.”  “No we didn’t implement. I am telling you we found out that if we implemented, we would direct our customers to one level and not what our customers want today so we didn’t want to implement that … yes it would limit us and also it would make us lose business.”  “To be honest, that business plan didn't work for me, but I feel it was on my part like the document didn't look practical for me.”  “I was trying to get a cooling equipment to cool my products even though we finally didn't really like get but we tried. That’s the one I made noise about; they took me to some place in Industrial Area. It didn't finally work, but we tried.” More commonly, however, SGBs interviewed in the endline raised same issues identified in the baseline related to the lag times, communication difficulties, time differences, locally inappropriate solutions (e.g., PFS volunteer insufficiently understands the local context), and disruptions in service (e.g. PFS volunteer reduces engagement for personal reasons or due to full-time work duties). The following quotes are illustrative of these concerns:  “Because the voluntary workers work when they have time, and that makes that an issue because it takes more time for projects to be concluded.”  “The time they are taking having phone calls with us and the people in the US is costly, so I would wish we would do one phone call instead of having a lot of them. Maybe we can all report to one person instead of having so many.”  “The only thing is the time. As I told you, the time they take to finish a project is too long.”  “The time they take to implement some of these projects because at times you find that it is taking a lot of time in their response, so they should improve on their time lines and their presence should at least increase. Even if they set aside one month where they come here, and we finish the whole thing and we move to the next thing, rather than having a whole six months doing one project, which you would have done in one month you end up doing in 6 months.”  “With PFS, we don’t see them, they don’t come. That’s just part of the agreement that they will not come, you interact with them through conference calls, so it’s a bit difficult, but we’ve managed.”  “Our situation here is sort of rough, and they wanted the type of information that if I had it I probably could have solved it myself. So we just let it go. We spent too much effort and time to educate the experts of our situation, so it wasn’t really worth it, and we just let it go. We were operating at two extremes; one a society where everything is fine and everything is documented and another situation where you don’t know what is on the ground.”  “The expert is asking questions whose answers would take a lot of time and effort for me to find out.”  “There is a limit to what a volunteer can do to a busy company, and you recognize that so you analyze whether you will get further or not.  “When we are in business, we have no time to waste. Sometimes when you talk about one thing more than three times, you feel like you are not moving anywhere.” Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 69  “They need to visit us more often rather than just on calls. We miss some of the information, like we get some phone calls from America and miss some of the information. Like the last call I didn’t hear a word, just the introductions. They have never visited so they never see the final product.”  “I wouldn’t encourage them to be doing more of remote assistance, they can establish maybe the challenges remotely, but they need to physically come and appreciate themselves even when it comes to interventions even when they are far, it becomes easier. We would be able to speak the same language, and you would be able to be assisted from your point of need.”  “I think, it would have been better to discuss directly with them, if we had that opportunity with us—as a bakery we were given an expert and it would have been better for us to interact directly”  “When they talk to you through TechnoServe, maybe TechnoServe would filter certain information before it gets to us, maybe because they thought it wasn’t relevant; this is information that we maybe would have considered useful being the people on the ground.” The above quotes illustrate that the SAFE model continues to have inherent challenges that will require further innovation to resolve. Both TNS and PFS, moreover, readily concede these challenges, as the following quotes illustrate:  “Communications challenges have gotten better but have not been solved. It is the hardy perennial weed coming up in the garden.”  It is a challenge to communicate across PFS corporate partners, each with its own systems, security systems, and firewalls. “  “Distance and communication always a challenge and takes time. Keeping clients and volunteers engaged is a challenge.”  “PFS model of remote assistance is challenging in general. It is like a telephone tag game, sometimes things get lost in translation.”  “It’s always a challenge working with volunteers. PFS says work and family come first and PFS volunteering is extra. So whether on weekends or after work, we try to emphasize to volunteers, don’t believe in volunteer guilt. If it’s not working out, then they need to be open and we will try to accommodate them. PFS is addressing this by giving good training and creating a support network; there are avenues they can reach out to and express concerns.” While the interviewed SGBs had no problems identifying challenges with the PFS volunteer-based distance model, they also acknowledged that, for the most part, it worked. Following up on the discussion above about the evolving division of responsibility between TNS and PFS (e.g., local solutions vs. distance solutions), evaluators asked supported SGBs about the capacity of local TNS staff to address the different issues at their firm in lieu of distance assistance from PFS volunteers. The SGBs agreed that local TNS staff was capable of addressing many of the issues they faced and had, in fact, done a good job doing so. However, they still saw an important role for PFS, as they recognized that the many of skills provided through PFS were not available locally.  “General Mill for example has been operating for many years and has seen many challenges and has acquired a lot of experience, and their team of experts are willing in their hearts to help. The ones in Nairobi may have the ability and may be willing to do it but they don’t have as much experience or may be missing some technology. So I can’t say Nairobi can’t do what PFS can, and I also can’t say that PFS shouldn’t come to Kenya, we need the knowledge they have.”  “No, those people in USA like General Mills are much advanced in terms of knowledge, in terms of experience, in terms of exposure so you know definitely they have contributed much more than the local staff, there’s no doubt on that. It doesn’t mean that local staff is bad, they are also good, but the point is you could get to something more because of experience, because of exposure from the USA people.” Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 70  “I think all of them are necessary. Definitely the systems up there are more improved, and you know we get the technology from up there, so it’s still necessary to have them in the chain.”  “I don’t know whether there are people who are qualified (locally) for potato industry just yet. I don’t know whether they would be able to give this kind of advice and information consultancy.”  “It would be good if some of these things could be done locally. Mainly that we would be dealing with the person directly to advise us through technologies locally unfortunately specially on dried products knowledge is zero in Zambia.”  “Not in Zambia, no there are not many people who know about drying in Zambia”  “Locally, no, because those were the only ones using cassava in sauce. We don’t have the experts locally.”  “I don’t think so, what it is like I believe every person’s experience and knowledge is unique even if you are in the same class being taught by the same professor you come out different and unique. I believe in networking. I believe in getting knowledge from different angles. I still believe that Partners for Food Solutions is different from what I would get from the locals.” Communication Breakdowns about Reasons for Dropping SGBs from the Program Interviews with SGBS that had been dropped from the program found that TNS does not always communicate well the reasons why the SGB has been dropped. Five dropped SGBs interviewed by the evaluation team expressed genuine confusion as to why they had been dropped from the program. The common refrain among these SGBS was that they only found out they had been dropped after abruptly receiving a letter from TNS stating as much, as reflected in the following quotes:  “They just wrote to us and said they have stopped working with us, they didn’t explain.”  “We received a graduation letter, which surprised us.”  “We suddenly got a ‘Graduation Letter.’ Oh, we graduated! Yeah, we were surprised.” This finding suggests a breakdown in communication between TNS and the SGBs where it comes to informing them that they have been dropped and the reasons for it. While we do not view this communication breakdown to be as serious an issue with regards to SAFE service delivery as those identified both in the baseline and above, it is nonetheless an issue that needs to be investigated and addressed. Conclusions The answer to EQ9 depends on what perspective one chooses to take. From one perspective, TNS and PFS did implement the program as it was designed, broadly speaking. Both adhered closely to the basic approach of selecting SGBs and then providing them distance-based technical assistance from international experts at PFS corporate partners, while TNS provided on-the-ground support and acted as an intermediary/translator for the PFS volunteers, who did most of the technical advising heavy lifting. By most accounts, moreover, both TNS and PFS did an overall good job of implementing this model (subject to the inherent constraints of the model), which appears also to have produced a number of positive benefits for participating SGBs, including increasing the availability of nutritious foods within the countries and markets served by the supported SGBs. From another perspective, however, TNS and PFS did not implement the program as designed. This conclusion, however, is not intended as a criticism of SAFE but rather the opposite. Rather than take the model as is and attempt to impose it within the program countries, both TNS and PFS have been active since the beginning of the program, but increasingly so since the baseline, in innovating the SAFE service delivery model to improve its efficiency and effectiveness. Notable innovations to the model include an improved SGB onboarding process; the expansion of SAFE client services via the quick win projects, Ask Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 71 An Expert, and industry-wide learning manuals; standardized charters; a more methodologically robust performance measurement approach via its new impact assessment methodology; an increased focus on vulnerable households; and increased full-time support staff at PFS. Of these, the quick win projects and standardized charters represent the most significant innovations to the SAFE service delivery model to date. Key informants at both PFS and TNS are well aware of the constraints to the SAFE model of distance￾based service delivery using volunteer experts (e.g., lag times, communication difficulties, time differences, locally inappropriate solutions, and disruptions in service) and are realistic about its limitations. This relatively clear-eyed approach has abetted what appears to be a culture of learning and innovation within both organizations. In fact, their willingness to acknowledge and confront these limitations to the model directly motivates the innovations to model implemented by both organizations over the course of the evaluation. As key informants at PFS concede that certain limitations to the model are “hardy perennial weeds” that will be difficult to eradicate completely, it is possible to continue to address them at the margin so as to improve TA service delivery, TA outcomes, and (vitally important to PFS) the volunteer experience. In summary, SAFE is an evolving program, and, as best determined by the evaluation team, the evolution is moving progressively toward creating a better program and a better service delivery model, albeit one with persistent limitations. As noted above, however, this evolution carries with it an increasing tension, particularly related to the relative roles of local TNS staff and PFS volunteers, as they continue to work out the best mix of providing locally-based technical solutions provided by local or regional experts (including TNS staff) and distance-based solutions provided by international volunteer experts. How PFS and TNS manage this tension will be one of the most defining characteristic of the model’s evolution over time. Recommendations SAFE Recommendations  PFS and TNS should be encouraged, and given ample space by BFS, to continue to innovate the service delivery model, even to the point where certain innovations fail or fare less well than others. It is clear over the two evaluation rounds that the inherent limitations to the model require ongoing innovation to address, and one can expect that solutions to these limitations are more likely to be addressed incrementally over time via a process of ongoing innovation and improvement as opposed to a single, point-in-time solution that solves the problem.  In particular, BFS should encourage PFS and TNS to redefine over time and through practice the relative roles of local TNS staff and local/regional experts vs. the PFS volunteer experts, as represented to date by the advent and evolving role played by the quick win projects and standardized charters. General Recommendations  Implementing a program ‘as planned’ should not necessarily be assumed to be a good thing. In SAFE’s case, for example, ‘as planned’ involved an inflexible approach that was ill suited to the diversity of SGB needs. There are often very good reasons for deviating program implementation from planned implementation. In cases where program deviations occur, it is important to understand the reasons for why the deviations occurred. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 72  While SAFE is a specific example, its experience highlights the general recommendation that programs often need the space and flexibility to innovate and learn by trial and error and adapt program design and implementation as appropriate. Unfortunately, the current USAID accountability system (e.g., MEP) imposes a rigid set of performance expectations on programs that inhibits this type of innovation and trial and error learning. This recommendation is particularly true for programs like SAFE that implement a pre-packaged, or off-the-shelf, methodology (in this case distance-based TA by international experts). EQ10. WHAT MIGHT SAFE HAVE DONE DIFFERENTLY TO IMPROVE PROGRAM IMPLEMENTATION? This section draws on the findings and conclusions from above to reach conclusions, along with a corresponding set of recommendations, about what SAFE might have been done differently to this point, and what it might do moving forward, to improve program implementation and results. The evaluation identified a number of areas, and a set of associated recommendations, where SAFE might do things differently so as to improve program implementation. These areas included the following: volunteer selection, inappropriate business or technical solutions (owing to limited local environmental knowledge), communications, processor selection, availability of TNS staff, face-to-face assistance, performance monitoring, sector-wide trainings, scaling up program impact, smallholder farmers, marketing assistance, gender sensitization, and training and access to finance. Progress achieved in these areas and new issues for consideration, along with associated recommendations, are presented below. Recommendations presented below apply both to the remaining life of the program and any follow-up program to SAFE. Volunteer Selection As recommended in the baseline, PFS continues to take actions to refine the volunteer selection and management process. These actions include increasing full-time staff at PFS headquarters with responsibilities to interact with volunteers, from both field-facing and corporate-facing perspectives; creation of the Senior Consultant position to work with and support volunteers in addition to the full￾time PFS staff; general reduction in the administrative duties tasked to volunteers; ongoing refinement and expansion of the Salesforce/PFS Connect database; and formalization and expansion of volunteer training. Recommendations  In light of PFS’ past growth and future growth plans, the operational systems that support and manage its volunteer expert force should be increasingly and, eventually, fully formalized. PFS has already taken important steps in this direction by increasing its full-time paid staff and phasing out the use of volunteers or part-time staff. This process should be continued up to the point where PFS is fully staffed with full-time, paid staff carrying out all important operational/administrative duties. Inappropriate Business or Technical Solutions/Limited Local Environmental Knowledge This is one of those “hardy perennial weeds” that are inherent to the SAFE service delivery model. There is no avoiding the fact that cultural/technological/professional gaps will exist between the PFS volunteers and the SGBs they are assisting. As recommended in the baseline, PFS is addressing this issue by formalizing and improving the training it provides to volunteers, which includes a separate module on “cultural exposure,’ and by beefing up its full-time volunteer support staff. The creation of the Senior Consultant position should also help reduce the occurrence of culturally or contextually inappropriate business Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 73 solutions. PFS has also created the online resource GlobeSmart, which allows volunteers to log in and read about the business culture in different countries. Recommendations  No recommendation is offered here. PFS has initiated an internal process to strengthen the training and support services it provides to its expert volunteers, both through the hiring of additional full￾time staff and through the development and implementation of training materials and regimens. PFS is expected to continue with this process. Communications This is yet another “hardy perennial weed” inherent to the SAFE service delivery model. While a variety of communications technologies are available to improve the efficiency and effectiveness of communications between PFS volunteers and client SGBs, to an inevitable extent the quality of the communication between the two will be limited by the state of communication technologies in the program countries, both related to internet availability and bandwidth and to mobile phone network coverage and reliability. PFS has addressed this issue in part by standardizing the BlueJeans tele-video conferencing technology across all PFS corporate partners. (It is one tele-video conferencing technology that is able to be integrated across all five corporate partner communications and security platforms.) PFS is currently trying to add more local access points for BlueJeans in Africa, and is also in the process of looking for a large international conference call provider that has local access numbers for people in PFS program countries, and is preparing to roll out the first wave of internet-based apps for client SGBs that can be accessed via computer or smart phone. Recommendations  No recommendation is offered here. PFS is actively looking for and experimenting with new approaches or technologies to improve the quality of the communication between its expert volunteers and its client SGBs. PFS is expected to continue with this process. SGB Selection The baseline evaluation recommended that TNS improve the SGB selection by increasing reliance on word-of-mouth and networking over mass recruitment efforts. TNS has, in fact, adopted this recommendation, which was underway at the baseline. It has also strengthened the entire SGB onboarding process through greater enforcement of SGB screening criteria (nutritious product, staple food, farmer/economic impact, maturity, female ownership, and coherence with FtF geographies); strengthening pre-selection assessment and vetting (e.g., Quality Audit and Business Assessment); and by introducing the familiarization meeting with all short-listed SGBs. Overall, the ET concludes that TNS has materially improved its SGB selection and onboarding process. Recommendations  TNS should refine the SGB selection and onboarding process further so as to bring on more SGBS who (1) are already selling in informal food markets or are looking to sell in informal food markets and (2) have existing, strong commercial linkages with SHFS (e.g., via an out-grower scheme) or are looking to establishing such commercial linkages with SHFs. To date, relatively few SAFE-supported SGBs are processing and selling products that reach vulnerable populations in the places where they tend to shop, and relatively few have established the types of relationships with their SHF suppliers (similar to, for example, COMACO) that would lead to the conclusion that they were having much Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 74 of an on-farm or economic impact on their supplier SHFs. While targeting either of these two types of SGBs is not mandated in SAFE’s SOW, they both appear fruitful areas for expanding SAFE’s impact as part of any follow-on program. Availability of TNS Staff The baseline evaluation recommended that TNS hire more local staff, or develop solutions allowing local TNS staff to follow-up more frequently with client SGBs. TNS has adopted the first of these recommendations, increasing the number of local staff since the baseline from 1 to 2 in both Zambia and Malawi. Both countries now have a local TNS Business Advisor and Food Technologist. The ET is not aware, however, if TNS has developed solutions that would free up time for local staff to conduct more frequent follow-up visits to supported SGBs. With that said, the ET learned that PFS has advocated to TNS to increasing the number of SGBs served by each of the local SAFE Food Technologists (FTs) and Business Advisors (BAs) as a means to increase program operational efficiencies. However, the ET is concerned that any increase in efficiency achieved through this policy will be more than compensated for in terms of reduced service quality, given an already significant work load shouldered by SAFE FTs and BAs. This will be particularly the case to the extent SAFE FTs and BAs are tasked with taking on greater responsibility for providing local solutions to client SGBs, whether as part of the quick win projects and as part of the standardized charters. Recommendations  SAFE should avoid adding significantly to the number of SGBs served by SAFE FTs and BAs until the additional work requirements involved with providing locally based TA (e.g., via quick wins and standardized charters) are clarified. Performance Monitoring The baseline evaluation concluded that the quarterly data collection of SGB performance by local TNS staff was excessive and created a good deal of resentment among the client SGBs. The baseline recommended a switch to semi-annual data collection and even annual data collection for certain MEP indicators. TNS has implemented a significant revision to its data collection process, although it took a different direction than recommended. In lieu of quarterly collection of data on the incremental volume of food processes and incremental net profits, TNS pilot tested and adopted a new impact assessment methodology that uses a more intensive, rigorous, and methodologically sound method to estimate the incremental volume and profits at client SGBs, which will, henceforth, replace the old system of quarterly data collection. This methodology, while imperfect, is a far better methodology than what was used in the past, which significantly distorted SAFE’s contribution to changes in incremental volumes and profits, whether by significantly overstating or understating them. Although TNS will no longer collect this incremental data quarterly, it has raised the possibility of doing it at least semi-annually, or some other frequency exceeding once per year. At the same time, TNS has raised the possibility of further reducing the frequency of data collection for the other PMP indicators, but it has not taken action in this regard. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 75 Recommendations  TNS should move forward to implement and institutionalize its new impact assessment methodology in lieu of the old method of measuring incremental production volumes and profits. The ET recommends, however, that TNS limit this to one measurement activity per year. Despite TNS’ optimism that SGBs appreciate this activity and would be willing to do it more than once a year, and comments by some SGBs supporting this conclusion, the impact assessment methodology is an intensive time consuming process that would eat up a significant portion of time for both TNS staff and client SGBs. Support for increasing the frequency of measurement in the abstract would, we believe, and based on SGBs comments in the baseline regarding intrusive data collection activities, come into conflict with the reality of time and resources that this would require done on a repeated basis each year.  The ET further recommends that over the next few months, TNS implements the impact assessment with all other SGBs missed during the initial September 2015 measurement activity, and for whom impacts may have realistically occurred, so as to create a complete program baseline and to ensure that the interim between the baseline and follow-up measurements allow sufficient time for measurable change to have occurred. Scaling-Up Program Impacts The baseline evaluation concluded that SWTs had been fairly effective in facilitating learning among participants, were highly valued by participants, and appeared to be the SAFE activity that has the greatest potential for achieving truly ‘significant’ scale within the program countries, albeit at a lower per-unit cost of TA delivery. Short-term trainings targeting specific types of SGBs—such as the short-term miller trainings offered by SAFE in conjunction with the WFP—are not SWTs, per se, but they serve a similar function as SWTs in terms of expanding program outreach and impact, at a relatively low cost per SGB served. These findings all suggest the conclusion that SWTs are an under-exploited tool. The baseline evaluation further concluded opportunities to scale-up program impacts existed by providing more ‘standardized’ solutions to common problems that face multiple processors. SAFE has taken an important step in this direction with the development of the standardized charter, although this innovation remains under development and has yet to be rolled-out on a wide scale. A further possible innovation for the standardized charter is to offer it to groups of SGBs outside the context of a formal chartering process, for example in conjunction with a SWT. Another option is to create a different classification of SGBs who participate in a less stringent and more short-term chartering process focused on working with multiple SGBs at once on standardized charters. Yet another options (out of possibly many) is to develop a learning resource similar to the sector-wide learning resources discussed in this report (e.g., manual, e-book, or course curriculum) covering standardized charter topics at the end of which participants receive some form of certification, either through by the course itself or by the relevant government ministry or industry association. The development of sector-wide learning resources—e.g., new product development manual and e-books on business strategy and fortification—is yet another potentially effective mechanism for scaling-up program outreach and impact, which also contributes to program sustainability. During the ET’s discussions with the TNS management team, the latter suggested that one way to improve the sustainability of program outcomes, and help build local capacity, is to involve local technical experts as observers or participants in the charters so that they too can benefit from the PFS volunteer experts. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 76 The end objective here would be to develop a cadre of local experts capable of providing technical assistance to those food processors not brought under the SAFE umbrella and who can continue to provide locally-based TA to food processors after SAFE’s departure. The ET agrees this is an idea worth pursuing further. Recommendations  Conduct routine follow-ups with SWT participants, especially those who appear to be good candidates for working directly under SAFE. This includes not only the follow-up currently being planned by TNS to assess the trainings’ longer-term benefits but also follow-up activities to reinforce aspects of the training and to provide additional assistance as needed. This might take the form of follow-up phone calls, workshops, visits to production facilities, linkage meetings (such as was done in Zambia with lending institutions), participation in Ask an Expert, and so forth. SAFE may also consider implementing a voucher scheme for follow-up, whereby SWT attendees can redeem vouchers for TA from pre￾identified trainers at a later date.  BFS should make additional alignments to the budget, if feasible, to fund SWTs during the remaining life of the program and that funding for SWTs be given a high priority in any follow-on program to SAFE.  Increase the number, scope, and outreach of short-term focused trainings, along with follow-up activities.  Pursue strategies for scaling-up the standardized charters, both as part of the formal chartering process and outside of the formal chartering process as discussed above.  Develop and disseminate additional sector-wide learning resources, making sure to allocate funding to market them once they are completed.  Investigate the feasibility and process for partnering with local technical experts with the selected charter teams as a strategy for increasing SAFE’s impact on the local food processing sector and for increasing SAFE’s long-term sustainability. Local technical experts in this case include local or regionally-based experts or business development service providers looking to develop their technical capacity to deliver similar technical assistance on a fee-for-service basis. Smallholder Farmers The baseline concluded that the program, as currently designed, was unlikely to have much of an impact on the on-farm production or wellbeing of the SHFs supplying SAFE client SGBs. In the interim since the baseline, SAFE has not embarked on any initiative specifically intended to address this issue or change this outcome (although again SAFE is not required within its SOW to do so) on any significant scale. The one SAFE activity that did indirectly address this issue was the series of short-term miller trainings done in Kenya with small community-based or cooperative SGBs whose membership consisted of local SHFs. As learned in the KIIs with these members, the short-term miller trainings also touched on issues related to on-farm production or post-production, which the interviewees claimed improved both their on-farm performance and their household wellbeing. While it was not possible to verify the extent to which this actually happened, this finding does suggest that targeting such short-term trainings to these type of community-based SGBs is a potentially effective strategy for extending the benefits of SAFE participation down to the farm level. It further suggests that Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 77 targeting such SGBs with more focused TA, whether as part of quick wins, standardized charters, or full￾blown PFS charters, is likewise a potentially effective strategy for pursing this objective. Recommendations  Prioritize the selection of SGBs that have formal contractual or out-grower relationships with their SHF suppliers, similar to COMACO. Consider, moreover, providing complimentary assistance to SGBs on procurement models, or partner with projects that specialize in this area in order to enhance linkages between their assistance and impact on producers. Finally, increase the number, scope and outreach of short-term focused trainings for community-based SGBs with SHF membership. Marketing Assistance The baseline evaluation concluded that client SGBs required greater assistance in marketing covering the 4P’s of marketing: product, pricing, packaging, promotion, and placement (distribution). The need to provide greater assistance in marketing practices takes on even greater importance should SAFE choose to work with client SGBS to improve their capacity sell within the informal markets. Selling successfully in informal markets requires a very different marketing strategy, again covering each of the 4 Ps of marketing, than selling to major retail chains or to other established formal market buyers. Recommendations  If SAFE chooses to work with its client SGBs to sell in informal markets, it will need to emphasize providing marketing assistance to SGS covering each of the 4P’s of marketing that reflects a clear and in-depth knowledge of the requirements to sell within informal markets. Gender Sensitization and Training The baseline evaluation concluded that SAFE assistance had not had any generalized impact on gender outcomes among supported SGBs, in large part due to inconsistent and weak implementation of gender activities within the SGBs. As SAFE did not implement any further gender initiatives after the baseline, there is no reason to change the conclusion reached in the baseline. Thus the recommendation offered in the baseline applies as well to the endline. Recommendations  Potential program improvements to improve gender sensitization and training include co-sponsoring on-site life skills development and other learning activities for lower and mid-level workers, facilitating gender sensitization workshops or trainings for processor staff, integrating the gender audit into the BAT, and following up with all processors with whom it has done gender audits to share the results, discuss the implications, and plot the way forward. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 78 ANNEXES Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 79 ANNEX 1: BIBLIOGRAPHY Jere, Paul. (2012). “Improving Land Sector Governance in Malawi.” World Bank. Lentz, Erin C. and Christopher B. Barrett. (2013). “The Economics and Nutritional Impacts of Food Assistance Policies and Programs.” ESA Working Paper No. 13-04.FAO. Pérez-Expósito, Ana B. and Barbara P. Klein.(2009). “Impact of Fortified Blended Food Aid Products on Nutritional Status of Infants and Young Children in Developing Countries.” Nutrition Reviews, 67(12): 706-18 Place, Frank, JemimahNjuki, Festus Murithi, and FridahMugo. (2009). “Agricultural Enterprise and Land Management in the Highlands of Kenya.” International Food Policy research Institute. Quisumbing. Agnes R. (2003). “Food AID and Child Nutrition in Rural Ethiopia.” FCND Discussion Paper No. 150, International Food Policy Research Institute “Measuring Progress Toward Empowerment: Women’s Empowerment in Agriculture Index Baseline Report”, USAID Feed the Future Program, International Food Policy Research Institute, 2014. Technoserve. (2010). “Industry Mapping Exercise: Zambia.” AAIFP. Technoserve. (2011). “Industry Mapping Exorcise: Kenya.” AAIFP. Technoserve. (2011). “Industry Mapping Exorcise: Malawi.” AAIFP USAID. (2011). “Property Rights and Resource Governance: Zambia” USAID Country Profile. World Bank. (2013). “Doing Business Reports: Kenya, Malawi, Zambia.” Kenya Demographic and Health Survey 2008-09 Malawi Demographic and Health Survey 2010 Zambia Demographic and Health Survey 2007 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 80 ANNEX 2: STAKEHOLDERS INTERVIEWED Name (Last, First) Organization Title in Organization TECHNOSERVE Kenya Johnson Kiragu Technoserve Program Manager/Kenya Jonathan Thomas Technoserve COP SAFE Esther Kamau Technoserve Regional Deputy Program Director/DCOP SAFE David Galaty Technoserve Regional M&E Manager Florah Mukabana Technoserve M&E Specialist Amnda Satterly Technoserve Director, Gender Initiatives Daisy Kambandu Technoserve Commercialization Manager Antony Etiang Technoserve Food Technologist Jane Njeri Technoserve Business Advisor Margaret Ngetha Technoserve Business Advisor, SAFE East Africa Gender Focal Person Zambia Jane Lungu Technoserve Program Manager Thandie Hamaimbo Technoserve Food Technologist Malawi Lawrence Muhamba Technoserve Food Technologist Amos Mkezwa Technoserve Program Manager PFS David Dayhoff PFS Vice President, Programs & Operations Meredith Kos PFS Director of Field Programs & Impact Kim Lapante PFS Training and Technology Specialist Ginger Anthony PFS Volunteer Profile & Salesforce Administrator Clare Healy PFS Program Associate Jim Olive General Mills SGB Lead-Gaea Foods Erika Smith General Mills Country Lead-Kenya FOOD PROCESSORS Kenya Martilda Wanjiku Fresh n’ Crunchy Pauline Njeri SOPA Supplies Managing Director Aura Omamo Marenyo Multipurpose Corporate Society Chairperson Buluku Benjamin Kakamega Dairy Acting Manager Isaac Ruto Plant Manager Amos Ombaso Borabu Famers Cooperative Society General Manager Michael Njoroge Kariuki Classic Foods General Manager Stella Kimemia Classic Foods Managing Director Frederick Mureithi Mukurweni Wakulima Dairy CEO Jame Mereithi Muiti Nyala Dairy Director Brian Cuthbert Doinyo Lessos Creameries Director Rosemary Meru Greens Horticulture Co-Director Mary Cherop Cherubet Company Ltd. Managing Director Tedlin Murila Mace Foods Production Manager Sudir Vaidya Njoro Canning Managing Director Catherine Amusugut Tanga Commercial Village Manager Value Addition Roseanne Wanjiku Gaea Foods Managing Director Florence Musumba Star of the Lake Enterprises Director Winston Odhiambo Edom Nutritional Solutions CEO Johnson Gachuhi Mwailu Enterprises CEO Eric Muthomi Stawi Foods and Fruits CEO Lawrence Buffalo Millers Director Finance and Operations Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 81 Reuben Kenyua Kirinyaga Flour Mills Director Evelyn Kenyua Kirinyaga Flour Mills Accountant John Ero Loyaal Millers Manager Erick Bashizi Jenga Maisha Zambia Jackson Kankoyo Bakewell Bakery Co-owner Sylvia Banda Sylva Foods Managing Director Dorothy Ericson Chankwakwa Company Managing Director David Chisulo Zelo Food CEO Daniel Ball Forest Fruits Managing Director Farayi Martin Muzofa Java Foods Production Manager Eddie Kasongo Khan SHARE Africa Director Precious Mwaka Mwinga Kane Limited Managing Director Chabala Chisunka Kane Limited Accountant Peter Makoleka Kane Limited Food Technologist Mohammed Salim Dawoodjee High Protein Foods Ltd. Managing Director Joshua Shonga COMACO Regional Manager David Sakala COMACO Quality Manager Daka Whiteson COMACO Extension Manager Malawi Alima Amini Project Peanut Butter Executive Director Chandrin Dissanaike Rab Processors General Manager Gaffar Jakhura Rab Processors Chairman Ahmed Sunka Rab Processors Deputy Managing Director Stanley Howse House Oil Mill Enterprise Managing Director Garnet Howse House Oil Mill Enterprise Accountant Lawrence Mwale Bvumbewe Dairy Cooperative Accountant Rhoda Gadama-Misomali Tafika Milling Chairperson Mark Chimimba Phiri Tayamba Soya Milk Cooperative Society Executive Director Maness Nkhata Lakeshore Agro Processing General Manager Manoj Kumar Vats Sunseed Oil Ltd. General Manager FINANCIAL INSTITUTIONS Kenya Agnes Manthi Root Capital Business Development Officer Peter Onguika Root Capital Regional Risk Manager Beatrice Githinji Chase Agriculture Manager Knowledge Risk Monitoring & Fund Management Gathuo Kairo Umati Capital Manager Financial Services Zambia Bennet Mwale Madison Finance Company Ltd. Head MSME Business OTHER STAKEHOLDERS Kenya Dr. Mary Mugo University of Kenya, Nairobi Dean, Faculty of Business and Law, Multimedia Dr. Robinson Ocharo University of Nairobi Chairman, Department of Sociology Zambia Rob Munro MUSIKA Director of Strategy Jacob Mwale Zambia Agricultural Commodity Exchange Managing Director Ward Siamusantu Thrive Project Technical Advisor Ralf Wiwiti Scaling Up Nutrition Business Network Country Network Manager NEW PRODUCT DEVELOPMENT MANUAL Regina Gikiru Kiambu Institute Professor Rueben Langat Kenya Industrial Property Institute Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 82 Mbugua Samuel University of Nairobi Professor FORTIFICIATION E-BOOK Felistus GAIN Coordinator of Food Fortification Peter Wathigo DSM Walther von Reading Buhler AG Food Engineer BUSINESS STRATEGY E-BOOK Victor Otieno Viffo Consult Managing Director Gachara Ng’an’a Synergy Financial Advisory Managing Parter Eunice Muthoni Strathmore Business School Professor Amos Njuguna United States International University￾Africa Associate Professor of Finance WFP TRAINING Samuel Serem Schemers CBO Manager Eliud Mololo Malakusi Mali Shambani Chairman Rose Atieno Oduor Pap Nyadiel Secretary INSTITUTIONAL BUYERS Kenya Lillian Mwangi, World Food Programme Program Policy Officer Zippy Mbati World Food Programme Program Policy Officer Malawi Chris Maclullich Mary’s Meals Regional Representative Martin Swinchatt Mary’s Meals Country Representative Phillip Hovmand World Food Programme Procurement Officer RETAILERS & WHOLESALERS Kenya Margaret Karimi Egg Depot Kibera Retailer Jackeline Ndunge Mam Chinku Shop Retailer Simon Jamii Jamii Corner Shop Retailer Agnes Wanjiku By Grace Shop Retailer Kate Njeri Mware Retail Shop Retailer Asaneth Kurgat Urafiki Wholesaler Wholesaler Teddy Otieno Dapo Wholesaler Wholesaler Justin Gilgal Gilgal Stores Wholesaler Zambia Pearson Phiri Pear Retailer Cephas Chikonde Elilz General Dealers Retailer Justina Banda Kanjele Investments Retailer Peter Phiri Uncle Petty General Dealers Wholesaler Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 83 ANNEX 3: CONSUMER FOCUS GROUP DISCUSSIONS Number Participants Sex Location Kenya 8 F Nairobi-Kayole Area 8 F Nairobi-Kayole Area 8 F Nairobi-Kayole Area 8 F Nairobi-Kayole Area Zambia 8 F Chipata 8 F Chipata 8 F Chipata 8 F Chipata Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 84 ANNEX 4: HOUSEHOLD SURVEY INSTRUMENT CASE STUDY QUESTIONNAIRE - KENYA Serial Number:___________ Latitude: ___________ Longitude: ___________ INTERVIEWER: ASK TO SPEAK WITH THE PERSON RESPONSIBE FOR MAKING FOOD PURCHASE DECISIONS Hello, my name is ........... from Infinite Insight, a market research agency based in Nairobi. We are conducting a survey in your area about food purchasing decisions and nutritious foods. The survey will take no more than 30 minutes. Respondent Name:______________________ Phone Number: __________________ Respondent Address:____________________ Interviewer:______________________ Province:______________________________ Interviewer ID:____________________ County:_______________________________ Supervisor:_______________________ Town/Village: Supervisor ID:_____________________ Juja 1 Thika 2 Ruiru 3 Kibera 4 Kayole 5 Kitengela 6 Tassia 7 Mwiki 8 Huruma 9 Athi River 10 Date of Interview:_______________________ Household Type: Sample Point Number:___________________ - Male and Female adult: 1 - Female only: 2 - Male only: 3 Respondent Name:______________________ Phone Number: __________________ Respondent Address:____________________ Interviewer:______________________ Province:______________________________ Interviewer ID:____________________ County:_______________________________ Supervisor:_______________________ Town/Village: Supervisor ID:_____________________ Juja 1 Thika 2 Ruiru 3 Kibera 4 Kayole 5 Kitengela 6 Tassia 7 Mwiki 8 Huruma 9 Athi River 10 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 85 QF1 Please describe the foods (meals and snacks) that members of your household consumed yesterday during the day and night, not including foods purchased and eaten outside the home. INTERVIEWER: READ OUT FOOD GROUPS AND EXAMPLES # Food Group Examples Yes No A Cereals Bread, noodles, ugali, chapattis, biscuits, cookies or any other foods made from millet, sorghum, maize, rice, wheat 1 2 B Vitamin A rich vegetables and tubers Pumpkin, carrots, squash or sweet potatoes that are orange inside; capsicums 1 2 C White tubers and roots White potatoes, white yams, cassava, white sweet potatoes 1 2 D Dark green leafy vegetables Dark green leafy vegetables, including wild ones: kale/sukuma, cassava leaves, spinach, 1 2 E Other vegetables Other vegetables (e.g., tomato, onion, eggplant), including wild vegetables 1 2 F Vitamin A rich fruits Ripe mangoes, + other locally available vitamin-A rich fruits 1 2 G Other fruits Other fruits, including wild fruits 1 2 H Organ meat (iron rich) Liver, kidney, heart or other organ meats or blood-based foods 1 2 I Flesh meats Beef, pork, lamb, goat, rabbit, bush meat, chicken, duck or other birds 1 2 J Eggs 1 2 K Fish Fresh or dried fish or shellfish 1 2 L Legumes, nuts and seeds Beans, peas, lentils, nuts, seeds or foods made from these 1 2 M Insects Insect larvae, lake fly, ants 1 2 N Milk and milk products Milk, cheese, yogurt or other milk products 1 2 O Oils and fats Oils, fats or butter added to food or used for cooking 1 2 P Sweets Sugar, honey, sweetened soda or sugary foods, such as chocolates, sweets or candies 1 2 Q Spices, condiments, beverages Spices (black pepper, salt), condiments (soy sauce, hot sauce), coffee, tea, alcoholic beverages OR local examples 1 2 QF2 Now I would like to ask a few questions on the difficulties some families experience when providing food for their families. Please tell me whether you have experienced the following problems. # Response Options Code A In the past [4 weeks/30 days], was there ever no food to eat of any kind in your house because of lack of resources to get food? Yes – 1 No – 2 (SKIP TO QF2C) B How often did this happen in the past [4 weeks/30 days]? Rarely (1-2 times) 1 Sometimes (3-10 times) 2 Often (more than 10 times) 3 C In the past [4 weeks/30 days], did you or any household member go to sleep at night hungry because there was not enough food? Yes – 1 No – 2 (SKIP TO QF2E) D How often did this happen in the past [4 weeks/30 days]? Rarely (1-2 times) 1 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 86 Sometimes (3-10 times) 2 Often (more than 10 times) 3 E In the past [4 weeks/30 days], did you or any household member go a whole day and night without eating anything at all because there was not enough food? Yes – 1 No – 2 (SKIP TO QF3) F How often did this happen in the past [4 weeks/30 days]? Rarely (1-2 times) 1 Sometimes (3-10 times) 2 Often (more than 10 times) 3 QF3 How familiar are you with nutritious foods? By nutritious foods we mean foods that provide essential nutrients and energy to sustain growth, health and life while satiating hunger. Not at all familiar 1 Not very familiar 2 Pretty familiar 3 Very familiar 4 D/K 9 QF4a Now I would like to talk to you about different places, where you shop for food. Which of the following outlet types do you consider the most important when buying food? HAND SHOW CARD QF4b And which is the second-most important? QF4c And the third-most important? First Second Third Supermarkets 1 1 1 Mini-Markets 2 2 2 Convenience Stores 3 3 3 Open Markets 4 4 4 Kiosks 5 5 5 Neighbourhood Stores 6 6 6 Butchers 7 7 7 Others (specify) 98 98 98 QF4d Now thinking about the past 4 weeks, which of these outlets have you actually visited to buy food for you and your family? FOR EACH OUTLET MENTIONED IN QF4d, ASK QF4e – QF4g QF4e Again thinking about the past 4 weeks, how often have you bought food items at … Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 87 QF4f And, on average, how much money did you spend per occasion at this outlet? QF4g And which of the food items on this list did you buy at this outlet? HAND SHOW CARD; MULTIPLE CODES Cereals 1 Vitamin A rich vegetables and tubers 2 White tubers and roots 3 Dark green leafy vegetables 4 Other vegetables 5 Vitamin A rich fruits 6 Other fruits 7 Organ meats 8 Flesh Meats 9 Eggs 10 Fish 11 Legumes, nuts and seeds 12 Milk and milk products 13 Oils and fats 14 Sweets 15 Spices, condiments, beverages 16 F4d F4e (Number of Visits) F4f (Avg. Amount Spent per Visit) F4g (Food Items Bought) Supermarkets 1 …. KES Mini-Markets 2 …. KES Convenience Stores 3 …. KES Open Markets 4 …. KES Kiosks 5 …. KES Neighbourhood Stores 6 …. KES Butchers 7 …. KES QF5a I will now read some statements that other people have made about the various types of outlets. For each statement, please tell me to which outlet type the statement applies. HAND SHOW CARD Super Markets Mini Markets Convenien ce Stores Open Markets Kiosks Neighbour￾hood Stores Butchers None of Thes e Wide choice of products 1 2 3 4 5 6 7 9 They sell at affordable prices 1 2 3 4 5 6 7 9 Convenient, easy to reach location 1 2 3 4 5 6 7 9 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 88 I can get credit when I am short of cash 1 2 3 4 5 6 7 9 They sell branded portion packs 1 2 3 4 5 6 7 9 They break up larger packs and sell portions from those packs 1 2 3 4 5 6 7 9 Their food is always fresh 1 2 3 4 5 6 7 9 It is difficult to know whether their food has already expired 1 2 3 4 5 6 7 9 An outlet for people like me 1 2 3 4 5 6 7 9 QF5b Do you have a place where you grow your own food? Yes, close to where I live 1 Yes, up-country 2 No 3 IF CODE 1 OR 2 IN QF5b, ASK QF5c What proportion of the food consumed in your household is bought from shops or stores? And what proportion do you grow yourself? Bought in shops ____ Grown by myself ____ TOTAL 100% QF6 Now I would like to talk to you how you make food-purchasing decisions. Please rate each item on how important it is to you when making food-purchasing decisions. Please use the same scale from 1 to 7, where 1 means that the item is not important at all and 7 means that it very important. # Questions Ratings D/K A Cost 1 2 3 4 5 6 7 9 B Quality 1 2 3 4 5 6 7 9 C Taste 1 2 3 4 5 6 7 9 D Ease of preparation 1 2 3 4 5 6 7 9 E Availability (ease of finding) 1 2 3 4 5 6 7 9 F Packaging 1 2 3 4 5 6 7 9 G Brand name 1 2 3 4 5 6 7 9 H Nutrition value 1 2 3 4 5 6 7 9 I Familiarity / tradition 1 2 3 4 5 6 7 9 J Requested by spouse or adult family members 1 2 3 4 5 6 7 9 K Requested by children 1 2 3 4 5 6 7 9 L Other (specify) 1 2 3 4 5 6 7 9 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 89 INTERVIEWER: HAND BRAND LIST TO RESPONDENT QB1 Which of these brands have you seen or heard about? FOR BRANDS MENTIONED IN QB1, ASK QB2 Which of these brands have you ever tried? FOR BRANDS MENTIONED IN QB2, ASK QB3 Which of these brands are you using nowadays, even if only infrequently? FOR BRANDS MENTIONED IN QB3, ASK QB4, QB5, AND QB6 QB4 On average, how often do you buy .... (INSERT BRANDS FROM QB3) INSERT CODE IN GRID BELOW Every day 1 Several times a week 2 Once a week 3 Several times a month 4 Once a month 5 Less often 6 Don’t know 9 QB5 Using a scale from 1 to 7, where 1 means completely dissatisfied and 7 means completely satisfied, how satisfied are you with.... INSERT RATING IN GRID BELOW QB6 And where did you learn about Brand.... INSERT CODES IN GRID BELOW; MULTIPLE CODES TV 1 Radio 2 Magazine 3 Newspaper 4 Internet 5 Sign/poster/billboard 6 Friend/relative/associate 7 Road show 8 Other (specify)_________________ 99 It’s Wild: Yummy Soy It’s Wild: Peanut Butter Seba Foods: Corn-Soy Blend Chankwak wa Corn￾Soy Blend Freshpikt Peanut Butter Jungle Beats Peanut Butter None of these QB1 (Awareness) 1 2 3 4 5 6 9  QB2 (Trial) 1 2 3 4 5 6 9  QB3 (Repertoire) 1 2 3 4 5 6 9  QB4 (Frequency) QB5 (Satisfaction) QB6 (Source of awareness) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 90 INTERVIEWER: THE FOLLOWING QUESTIONS ARE ASKED ONLY FOR It’s Wild: Yummy Soy AND/OR It’s Wild: Peanut Butter IF RESPONDENT IS UNAWARE OF BOTH BRANDS, GO TO DEMOGRAPHICS CHECK QB4 – IF ONCE A MONTH OR LESS OFTEN (CODE 5 OR 6), ASK QN1 Why do you not buy ... or buy it more often? DO NOT READ; MULTIPLE CODES It’s Wild: Yummy Soy It’s Wild: Peanut Butter Too expensive 1 1 Not sold in market 2 2 Too hard to find 3 3 Do not like taste 4 4 Not part of traditional diet 5 5 Too hard to prepare 6 6 Not packaged appropriately (e.g. packs are too big and we don’t need that much or cost too much) 7 7 Not appropriate food for people like us (e.g., for sick or old people) 8 8 Other adults in the family do not like it or do not want me to buy it 9 9 Children in the family do not like it or do not want me to buy it 10 10 It’s not a food for people like me 11 11 Other (specify) 98 98 ASK ALL AWARE OF BRAND 1, Brand 2 AND/OR BRAND 3 IN QB1 QN2 How do you rate the quality of.... It’s Wild: Yummy Soy It’s Wild: Peanut Butter Poor 1 1 Fair 2 2 Good 3 3 Very good 4 4 QN3 And how would you rate the cost of ... It’s Wild: Yummy Soy It’s Wild: Peanut Butter Inexpensive 1 1 Not very expensive 2 2 Expensive 3 3 Very expensive 4 4 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 91 FOR BRAND(S) USED IN QB3, ASK QN4 AND QN5 QN4 What pack size to you usually buy? QN5 And how much do you pay for this pack size? It’s Wild: Yummy Soy It’s Wild: Peanut Butter QN4 PACK SIZE 1kg 1 Other (specify) 98 2kg 2 Other (specify) 3 QN5 PRICE .......ZMK .......ZMK ASK ALL WHO HAVE TRIED BRAND 1 AND/OR BRAND 2 IN QB2 QN6 How would you rate the nutritional value of....? It’s Wild: Yummy Soy It’s Wild: Peanut Butter Not nutritious 1 1 Not very nutritious 2 2 Moderately nutritious 3 3 Very nutritious 4 4 QN7 And how would you rate the taste of...? It’s Wild: Yummy Soy It’s Wild: Peanut Butter Not at all tasty 1 1 Not very tasty 2 2 Moderately tasty 3 3 Very tasty 4 4 QN8 a On a scale from 1 to 7, where 1 means dislike very much and 7 means like very much, how much do adults in your household like ....? QN8b And how much do children in your household like....? It’s Wild: Yummy Soy It’s Wild: Peanut Butter QN8a (Adults) QN8b (Children) QN9 Would you recommend ... to your friends or associates? Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 92 It’s Wild: Yummy Soy It’s Wild: Peanut Butter Definitely not 1 1 Probably not 2 2 Not sure 3 3 Probably 4 4 Definitely 5 5 QN10 How likely are you to buy ... in the future? It’s Wild: Yummy Soy It’s Wild: Peanut Butter Very unlikely 1 1 Not very likely 2 2 Not sure 3 3 Somewhat likely 4 4 Very likely 5 5 IF CODE 1 OR 2 in QN10, ASK QN11 Why would you not buy... in the future? SPONTANEOUS; MULTIPLE CODES It’s Wild: Yummy Soy It’s Wild: Peanut Butter Too expensive 1 1 Not sold in market 2 2 Too hard to find 3 3 Do not like taste 4 4 Not part of traditional diet 5 5 Too hard to prepare 6 6 Not packaged appropriately (e.g. packs are too big and we don’t need that much or cost too much) 7 7 Not appropriate food for people like us (e.g., for sick or old people) 8 8 Other adults in the family do not like it or do not want me to buy it 9 9 Children in the family do not like it or do not want me to buy it 10 10 Other people will consider you a failure, if you are seen eating this product 11 11 Other (specify) 98 98 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 93 DEMOGRAPHICS Respondent sex Male……..1 Female…..2 Household head sex Male……..1 Female…..2 Respondent age Respondent marital status Currently married-monogamous…….1 Currently married-polygamous……...2 Widowed…………………………….3 Divorced……………………………..4 Single……………………….………..5 Separated…………………………….6 Household head can read and write Yes……..1 No….…..2 Respondent can read and write Yes……..1 No….…..2 Level of education of household head Standard 1-4……….1 Standard 5-8……….2 Formal 1-2…………3 Formal 3-4…………4 Post secondary……..5 Adult literacy………6 None……………….7 Level of education respondent Standard 1-4……….1 Standard 5-8……….2 Formal 1-2…………3 Formal 3-4…………4 Post secondary……..5 Adult literacy………6 None……………….7 Total number of household members Total number of people above 65 years Total number of people 15-64 years Total number of children 5-14 years Total number of children under 5 years Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 94 Response Response Codes ENUMERATOR OBSERVE (DO NOT ASK) Roof top material (outer covering) Tile. . 1 Wood. . 2 Corrugated metal. . 3 Plastic sheeting. . 4 Thatched/vegetable matter/sticks. . 5 Mud/cow dung. . 6 Other. . 7 ENUMERATOR OBSERVE (DO NOT ASK) Floor material Earth/mud. . 1 Concrete/flag stone/cement. . 2 Tile/bricks. . 3 Wood. . 4 Other. . 5 ENUMERATOR OBSERVE (DO NOT ASK) Exterior walls Earth/mud. . 1 Concrete/flag stone/cement. . 2 Tile/bricks. . 3 Wood. . 4 Other. . 5 What is the main type of toilets your household uses? Flush, shared. . 1 Flush, private. . 2 Ventilated improved pit latrine. . 3 Pit latrine. . 4 Community toilet. . 5 Pan/bucket. . 6 No toilet. . 7 Other. . 8 What is the main source of drinking water for your household? Piped into dwelling. . 1 Piped into plot/yard. . 2 Public tap (someone else’s private tap). 3 Tubewell/borehole. . 4 Protected dug well. . 5 Protected spring. . 6 Rain water collection. . 7 Unprotected dug well/springs. . 8 River/ponds/streams. . 9 Tankers-truck/vendor. . 10 Bottled water. . 11 Other (specify). . 12 Does this house have electricity? Yes = 1 No = 2 What is the main source of cooking fuel for your household? Electricity. . 1 Piped or liquid propane gas (biogas). . 2 Kerosene. . 3 Charcoal. . 4 Firewood. . 5 Animal dung. . 6 Agricultural crop residue. . 7 Other. . 8 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 95 CASE STUDY QUESTIONNAIRE - ZAMBIA Serial Number:___________ Latitude: __________ Longitude: ___________ Respondent Name:______________________ Phone Number: __________________ Respondent Address:____________________ Interviewer:______________________ Province:______________________________ Interviewer ID:____________________ County:_______________________________ Supervisor:_______________________ Town/Village: Supervisor ID:_____________________ Chipata 1 Katete 2 Date of Interview:_______________________ Household Type: Sample Point Number:___________________ - Male and Female adult: 1 - Female only: 2 - Male only: 3 Respondent Name:______________________ Phone Number: __________________ Respondent Address:____________________ Interviewer:______________________ Province:______________________________ Interviewer ID:____________________ County:_______________________________ Supervisor:_______________________ Town/Village: Supervisor ID:_____________________ Chipata 1 Katete 2 Date of Interview:_______________________ Household Type: Sample Point Number:___________________ - Male and Female adult: 1 - Female only: 2 - Male only: 3 Respondent Name:______________________Phone Number: __________________ Respondent Address:____________________ Interviewer:______________________ Province:______________________________ Interviewer ID:____________________ County:_______________________________ Supervisor:_______________________ Town/Village: Supervisor ID:_____________________ Chipata 1 Katete 2 Date of Interview:_______________________ Household Type: Sample Point Number:___________________ - Male and Female adult: 1 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 96 QF1 Please describe the foods (meals and snacks) that members of your household consumed yesterday during the day and night, not including foods purchased and eaten outside the home. INTERVIEWER: READ OUT FOOD GROUPS AND EXAMPLES # Food Group Examples Yes No A Cereals Bread, noodles, ugali, chapattis, biscuits, cookies or any other foods made from millet, sorghum, maize, rice, wheat 1 2 B Vitamin A rich vegetables and tubers Pumpkin, carrots, squash or sweet potatoes that are orange inside; capsicums 1 2 C White tubers and roots White potatoes, white yams, cassava, white sweet potatoes 1 2 D Dark green leafy vegetables Dark green leafy vegetables, including wild ones: kale/sukuma, cassava leaves, spinach, 1 2 E Other vegetables Other vegetables (e.g., tomato, onion, eggplant), including wild vegetables 1 2 F Vitamin A rich fruits Ripe mangoes, + other locally available vitamin-A rich fruits 1 2 G Other fruits Other fruits, including wild fruits 1 2 H Organ meat (iron rich) Liver, kidney, heart or other organ meats or blood-based foods 1 2 I Flesh meats Beef, pork, lamb, goat, rabbit, bush meat, chicken, duck or other birds 1 2 J Eggs 1 2 K Fish Fresh or dried fish or shellfish 1 2 L Legumes, nuts and seeds Beans, peas, lentils, nuts, seeds or foods made from these 1 2 M Insects Insect larvae, lake fly, ants 1 2 N Milk and milk products Milk, cheese, yogurt or other milk products 1 2 O Oils and fats Oils, fats or butter added to food or used for cooking 1 2 P Sweets Sugar, honey, sweetened soda or sugary foods, such as chocolates, sweets or candies 1 2 Q Spices, condiments, beverages Spices (black pepper, salt), condiments (soy sauce, hot sauce), coffee, tea, alcoholic beverages OR local examples 1 2 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 97 QF2 Now I would like to ask a few questions on the difficulties some families experience when providing food for their families. Please tell me whether you have experienced the following problems. # Response Options Code A In the past [4 weeks/30 days], was there ever no food to eat of any kind in your house because of lack of resources to get food? Yes – 1 No – 2 (SKIP TO QF2C) B How often did this happen in the past [4 weeks/30 days]? Rarely (1-2 times) 1 Sometimes (3-10 times) 2 Often (more than 10 times) 3 C In the past [4 weeks/30 days], did you or any household member go to sleep at night hungry because there was not enough food? Yes – 1 No – 2 (SKIP TO QF2E) D How often did this happen in the past [4 weeks/30 days]? Rarely (1-2 times) 1 Sometimes (3-10 times) 2 Often (more than 10 times) 3 E In the past [4 weeks/30 days], did you or any household member go a whole day and night without eating anything at all because there was not enough food? Yes – 1 No – 2 (SKIP TO QF3) F How often did this happen in the past [4 weeks/30 days]? Rarely (1-2 times) 1 Sometimes (3-10 times) 2 Often (more than 10 times) 3 QF3 How familiar are you with nutritious foods? By nutritious foods we mean foods that provide essential nutrients and energy to sustain growth, health and life while satiating hunger. Not at all familiar 1 Not very familiar 2 Pretty familiar 3 Very familiar 4 D/K 9 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 98 QF4a Now I would like to talk to you about different places, where you shop for food. Which of the following outlet types do you consider the most important when buying food? HAND SHOW CARD QF4b And which is the second-most important? QF4c And the third-most important? First Second Third Supermarkets 1 1 1 Mini-Markets 2 2 2 Convenience Stores 3 3 3 Open Markets 4 4 4 Kiosks 5 5 5 Neighbourhood Stores 6 6 6 Butchers 7 7 7 Others (specify) 98 98 98 QF4d Now thinking about the past 4 weeks, which of these outlets have you actually visited to buy food for you and your family? FOR EACH OUTLET MENTIONED IN QF4d, ASK QF4e – QF4g QF4e Again thinking about the past 4 weeks, how often have you bought food items at … QF4f And, on average, how much money did you spend per occasion at this outlet? QF4g And which of the food items on this list did you buy at this outlet? HAND SHOW CARD; MULTIPLE CODES Cereals 1 Vitamin A rich vegetables and tubers 2 White tubers and roots 3 Dark green leafy vegetables 4 Other vegetables 5 Vitamin A rich fruits 6 Other fruits 7 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 99 Organ meats 8 Flesh Meats 9 Eggs 10 Fish 11 Legumes, nuts and seeds 12 Milk and milk products 13 Oils and fats 14 Sweets 15 Spices, condiments, beverages 16 F4d F4e (Number of Visits) F4f (Avg. Amount Spent per Visit) F4g (Food Items Bought) Supermarkets 1 …. KES Mini-Markets 2 …. KES Convenience Stores 3 …. KES Open Markets 4 …. KES Kiosks 5 …. KES Neighbourhood Stores 6 …. KES Butchers 7 …. KES QF5a I will now read some statements that other people have made about the various types of outlets. For each statement, please tell me to which outlet type the statement applies. HAND SHOW CARD Super Markets Mini Markets Convenience Stores Open Markets Kiosks Neighbour￾hood Stores Butchers None of These Wide choice of products 1 2 3 4 5 6 7 9 They sell at affordable prices 1 2 3 4 5 6 7 9 Convenient, easy to reach location 1 2 3 4 5 6 7 9 I can get credit when I am short of cash 1 2 3 4 5 6 7 9 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 100 They sell branded portion packs 1 2 3 4 5 6 7 9 They break up larger packs and sell portions from those packs 1 2 3 4 5 6 7 9 Their food is always fresh 1 2 3 4 5 6 7 9 It is difficult to know whether their food has already expired 1 2 3 4 5 6 7 9 An outlet for people like me 1 2 3 4 5 6 7 9 QF5b Do you have a place where you grow your own food? Yes, close to where I live 1 Yes, up-country 2 No 3 IF CODE 1 OR 2 IN QF5b, ASK QF5c What proportion of the food consumed in your household is bought from shops or stores? And what proportion do you grow yourself? Bought in shops ____ Grown by myself ____ TOTAL 100% QF6 Now I would like to talk to you how you make food-purchasing decisions. Please rate each item on how important it is to you when making food-purchasing decisions. Please use the same scale from 1 to 7, where 1 means that the item is not important at all and 7 means that it very important. # Questions Ratings D/K A Cost 1 2 3 4 5 6 7 9 B Quality 1 2 3 4 5 6 7 9 C Taste 1 2 3 4 5 6 7 9 D Ease of preparation 1 2 3 4 5 6 7 9 E Availability (ease of finding) 1 2 3 4 5 6 7 9 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 101 F Packaging 1 2 3 4 5 6 7 9 G Brand name 1 2 3 4 5 6 7 9 H Nutrition value 1 2 3 4 5 6 7 9 I Familiarity / tradition 1 2 3 4 5 6 7 9 j Requested by spouse or adult family members 1 2 3 4 5 6 7 9 K Requested by children 1 2 3 4 5 6 7 9 L Other (specify) 1 2 3 4 5 6 7 9 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 102 INTERVIEWER: HAND BRAND LIST TO RESPONDENT QB1 Which of these brands have you seen or heard about? FOR BRANDS MENTIONED IN QB1, ASK QB2 Which of these brands have you ever tried? FOR BRANDS MENTIONED IN QB2, ASK QB3 Which of these brands are you using nowadays, even if only infrequently? FOR BRANDS MENTIONED IN QB3, ASK QB4, QB5, AND QB6 QB4 On average, how often do you buy .... (INSERT BRANDS FROM QB3) INSERT CODE IN GRID BELOW Every day 1 Several times a week 2 Once a week 3 Several times a month 4 Once a month 5 Less often 6 Don’t know 9 QB5 Using a scale from 1 to 7, where 1 means completely dissatisfied and 7 means completely satisfied, how satisfied are you with.... INSERT RATING IN GRID BELOW QB6 And where did you learn about Brand.... INSERT CODES IN GRID BELOW; MULTIPLE CODES TV 1 Radio 2 Magazine 3 Newspaper 4 Internet 5 Sign/poster/billboard 6 Friend/relative/associate 7 Road show 8 Other (specify)_________________ 99 It’s Wild: Yumm y Soy It’s Wild: Peanut Butter Seba Foods: Corn-Soy Blend Chankwak wa Corn￾Soy Blend Freshpikt Peanut Butter Jungle Beats Peanut Butter None of these QB1 (Awareness) 1 2 3 4 5 6 9  QB2 (Trial) 1 2 3 4 5 6 9  QB3 (Repertoire) 1 2 3 4 5 6 9  QB4 (Frequency) QB5 (Satisfaction) QB6 (Source of awareness) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 103 INTERVIEWER: THE FOLLOWING QUESTIONS ARE ASKED ONLY FOR It’s Wild: Yummy Soy AND/OR It’s Wild: Peanut Butter IF RESPONDENT IS UNAWARE OF BOTH BRANDS, GO TO DEMOGRAPHICS CHECK QB4 – IF ONCE A MONTH OR LESS OFTEN (CODE 5 OR 6), ASK QN1 Why do you not buy ... or buy it more often? DO NOT READ; MULTIPLE CODES It’s Wild: Yummy Soy It’s Wild: Peanut Butter Too expensive 1 1 Not sold in market 2 2 Too hard to find 3 3 Do not like taste 4 4 Not part of traditional diet 5 5 Too hard to prepare 6 6 Not packaged appropriately (e.g. packs are too big and we don’t need that much or cost too much) 7 7 Not appropriate food for people like us (e.g., for sick or old people) 8 8 Other adults in the family do not like it or do not want me to buy it 9 9 Children in the family do not like it or do not want me to buy it 10 10 It’s not a food for people like me 11 11 Other (specify) 98 98 ASK ALL AWARE OF BRAND 1, Brand 2 AND/OR BRAND 3 IN QB1 QN2 How do you rate the quality of.... It’s Wild: Yummy Soy It’s Wild: Peanut Butter Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 104 Poor 1 1 Fair 2 2 Good 3 3 Very good 4 4 QN3 And how would you rate the cost of ... It’s Wild: Yummy Soy It’s Wild: Peanut Butter Inexpensive 1 1 Not very expensive 2 2 Expensive 3 3 Very expensive 4 4 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 105 FOR BRAND(S) USED IN QB3, ASK QN4 AND QN5 QN4 What pack size to you usually buy? QN5 And how much do you pay for this pack size? It’s Wild: Yummy Soy It’s Wild: Peanut Butter QN4 PACK SIZE 1kg 1 Other (specify) 98 2kg 2 Other (specify) 3 QN5 PRICE .......ZMK .......ZMK ASK ALL WHO HAVE TRIED BRAND 1 AND/OR BRAND 2 IN QB2 QN6 How would you rate the nutritional value of....? It’s Wild: Yummy Soy It’s Wild: Peanut Butter Not nutritious 1 1 Not very nutritious 2 2 Moderately nutritious 3 3 Very nutritious 4 4 QN7 And how would you rate the taste of...? It’s Wild: Yummy Soy It’s Wild: Peanut Butter Not at all tasty 1 1 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 106 Not very tasty 2 2 Moderately tasty 3 3 Very tasty 4 4 QN8 a On a scale from 1 to 7, where 1 means dislike very much and 7 means like very much, how much do adults in your household like ....? QN8b And how much do children in your household like....? It’s Wild: Yummy Soy It’s Wild: Peanut Butter QN8a (Adults) QN8b (Children) QN9 Would you recommend ... to your friends or associates? It’s Wild: Yummy Soy It’s Wild: Peanut Butter Definitely not 1 1 Probably not 2 2 Not sure 3 3 Probably 4 4 Definitely 5 5 QN10 How likely are you to buy ... in the future? It’s Wild: Yummy Soy It’s Wild: Peanut Butter Very unlikely 1 1 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 107 Not very likely 2 2 Not sure 3 3 Somewhat likely 4 4 Very likely 5 5 IF CODE 1 OR 2 in QN10, ASK QN11 Why would you not buy... in the future? SPONTANEOUS; MULTIPLE CODES It’s Wild: Yummy Soy It’s Wild: Peanut Butter Too expensive 1 1 Not sold in market 2 2 Too hard to find 3 3 Do not like taste 4 4 Not part of traditional diet 5 5 Too hard to prepare 6 6 Not packaged appropriately (e.g. packs are too big and we don’t need that much or cost too much) 7 7 Not appropriate food for people like us (e.g., for sick or old people) 8 8 Other adults in the family do not like it or do not want me to buy it 9 9 Children in the family do not like it or do not want me to buy it 10 10 Other people will consider you a failure, if you are seen eating this product 11 11 Other (specify) 98 98 DEMOGRAPHICS Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 108 Respondent sex Male……..1 Female…..2 Household head sex Male……..1 Female…..2 Respondent age Respondent marital status Currently married-monogamous…….1 Currently married-polygamous……...2 Widowed…………………………….3 Divorced……………………………..4 Single……………………….………..5 Separated…………………………….6 Household head can read and write Yes……..1 No….…..2 Respondent can read and write Yes……..1 No….…..2 Level of education of household head Standard 1-4……….1 Standard 5-8……….2 Formal 1-2…………3 Formal 3-4…………4 Post-secondary……..5 Adult literacy………6 None……………….7 Level of education respondent Standard 1-4……….1 Standard 5-8……….2 Formal 1-2…………3 Formal 3-4…………4 Post-secondary……..5 Adult literacy………6 None……………….7 Total number of household members Total number of people above 65 years Total number of people 15-64 years Total number of children 5-14 years Total number of children under 5 years Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 109 Response Response Codes ENUMERATOR OBSERVE (DO NOT ASK) Roof top material (outer covering) Tile. . 1 Wood. . 2 Corrugated metal. . 3 Plastic sheeting. . 4 Thatched/vegetable matter/sticks. . 5 Mud/cow dung. . 6 Other. . 7 ENUMERATOR OBSERVE (DO NOT ASK) Floor material Earth/mud. . 1 Concrete/flag stone/cement. . 2 Tile/bricks. . 3 Wood. . 4 Other. . 5 ENUMERATOR OBSERVE (DO NOT ASK) Exterior walls Earth/mud. . 1 Concrete/flag stone/cement. . 2 Tile/bricks. . 3 Wood. . 4 Other. . 5 What is the main type of toilets your household uses? Flush, shared. . 1 Flush, private. . 2 Ventilated improved pit latrine. . 3 Pit latrine. . 4 Community toilet. . 5 Pan/bucket. . 6 No toilet. . 7 Other. . 8 What is the main source of drinking water for your household? Piped into dwelling. . 1 Piped into plot/yard. . 2 Public tap (someone else’s private tap). 3 Tubewell/borehole. . 4 Protected dug well. . 5 Protected spring. . 6 Rain water collection. . 7 Unprotected dug well/springs. . 8 River/ponds/streams. . 9 Tankers-truck/vendor. . 10 Bottled water. . 11 Other (specify). . 12 Does this house have electricity? Yes = 1 No = 2 What is the main source of cooking fuel for your household? Electricity. . 1 Piped or liquid propane gas (biogas). . 2 Kerosene. . 3 Charcoal. . 4 Firewood. . 5 Animal dung. . 6 Agricultural crop residue. . 7 Other. . 8 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 110 ANNEX 5: DISCUSSION GUIDES Processor Discussion Guide Introduction Processor Background Recruitment Early win projects DISCUSSION GUIDE Introduce self, the purpose of the study and the equipment Reassure respondent of confidentiality Ask respondent for their business card Please tell me your name and position within…. STATE THE NAME OF THE PROCESSOR YOU ARE INTERVIEWING How long have you worked here in this role? Please tell us about your firm, including: o Products produced and associated brand names o Number of total employees, male employees, and female employees at supervisory and non-supervisory levels o Date you officially became a SAFE client ASK ONLY IF ON-BOARDED SINCE MAY 2015 How did you first learn about SAFE? Why did you decide to work with SAFE? (PROBE TO DETERMINE WHAT OBJECTIVE RESPONDENT WANTED TO ACCOMPLISH What was the process SAFE used to recruit and enroll you in the program? In answering this question, please comment on the amount and usefulness of information provided to you about the program, what its purpose is, how it works, what the expectations are for both you and SAFE. As part of this enrolment process, did you attend an event with other food processors like yourself? If ‘Yes,” how useful was this event? Why? What specific benefits did you get from it? As part of this enrolment process, did SAFE (or local TechnoServe) staff perform an assessment of your business? If ‘Yes,’ how useful was this? Why? What specific benefits did you get from it? Were there any conditions, related to women’s involvement or economic empowerment, that SAFE requested or required as part of your participation in the program? If ‘Yes,’ what were they? What aspects of the recruitment and enrolment process did you like the best or find most useful? What aspects of the recruitment and enrolment process did you like less or find less useful? ASK ONLY IF FEMALE OWNED OR MANAGED FIRM How did the fact that you are a woman-owned or managed firm affect your firm’s decision or selection to work with SAFE? ASK ONLY IF PROCESSOR PARTICIPATED IN EARLY WIN PROJECTS We would now like to talk with you about the early win projects you have participated in, or are participating in, with SAFE. EARLY WIN PROJECTS ARE: THE USE OF LOCAL TECHNOSERVE STAFF TO ADDRESS ISSUES AT THE ASSISTED PROCESSOR BEFORE AGREEING ON AND BEGINNING PROJECTS WITH PFS. FOR EARLY WIN PROJECTS THAT ARE STILL IN PROCESS, FRAME THE QUESTIONS TO REFER TO EXPECTED OUTCOMES AND IMPACTS ON THE BUSINESS. Please describe the early win projects you have implemented, or are implementing, with SAFE. What assistance did TechnoServe provide you on these early win projects? What key recommendations did the local TechnoServe staff make related to the early win projects? Did you follow them? Why or why not? Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 111 Effectiveness and Satisfaction Women’s Economic Empowerment Access to Finance SAFE Impact Assessment How successful were these early win projects? What impact did it have in your business? Why? (PROBE TO LEARN ABOUT BOTH SUCCESSFUL AND UNSUCCESSFUL PROJECTS AND THE REASONS WHY THEY WERE SUCCESSFUL OR UNSUCCESSFUL) What recommendations would you make to improve: - The process used to identify and select early win projects? Why? - The assistance from TechnoServe staff on your early win projects. Why? - The outcomes of early win projects? Why? We would now like to talk with you about the projects you have participated in, or are participating in, with SAFE. CLARIFY THAT THE FOLLOWING QUESTIONS REFER TO THE PROJECTS FOR WHICH THEY HAVE RECEIVED BOTH TECHNOSERVE AND PFS ASSISTANCE. FOR PROJECTS THAT ARE STILL IN PROCESS, FRAME THE QUESTIONS TO REFER TO EXPECTED OUTCOMES AND IMPACTS ON THE BUSINESS. Please describe the projects you have implemented, or are implementing, with SAFE i.e. TNS & PFS What assistance do/did TechnoServe and PFS provide you on these projects? (PROBE TO DISTINGUISH THE ASSISTANCE PROVIDED BY LOCAL TECHNOSERVE STAFF AND PFS VOLUNTEERS) What key recommendations did the local TechnoServe staff and PFS volunteers make related to these projects? Did you follow them? Why or why not? (PROBE TO DISTINGUISH THE RECOMMENDATIONS MADE BY LOCAL TECHNOSERVE STAFF AND BY PFS VOLUNTEERS) How successful were these projects in impacting your business? Why? (PROBE TO LEARN ABOUT BOTH SUCCESSFUL AND UNSUCCESSFUL PROJECTS AND THE REASONS WHY THEY WERE SUCCESSFUL OR UNSUCCESSFUL) Do you think the assistance received from PFS could be provided locally i.e. by the local TNS staff or any other local expert? What impact did these projects have on your business performance? What recommendations would you make to improve: o The process used to identify and select projects? Why? o The assistance from TechnoServe staff? Why? o The assistance from PFS volunteers? Why? o The outcomes of TechnoServe and PFS assistance? Why? Please describe the opportunities in your business for women in terms of work, personal advancement, professional advancement and management opportunities. Has SAFE worked with your business in any way to help you address issues related to your female employees? If ‘Yes,’ please describe. Has your work with SAFE affected in any way the opportunities in your business for your female employees in terms of work, personal advancement, professional advancement and management opportunities? Did SAFE help you access commercial financing for your business? If ‘Yes,’ can you please tell me about it? (PROBE TO FIND OUT FINANCIAL INSTITUTION, TYPE OF FINANCING, AMOUNT, INTEREST RATE, CONDITIONS AND REPAYMENT TERMS) What role did SAFE play in helping you receive this financing? What can an organization like SAFE do to help businesses like you access commercial finance? ASK ONLY IF PROCESSOR PARTICIPATED IN THE TECHNOSERVE IMPACT ASSESSMENT We understand that TechnoServe conducted an in-depth assessment of its impact on your firm during 2015 during which time representatives from TechnoServe visited your firm to determine how its assistance has impact such things as your sales, income and employment, quality, labour costs and production costs. o Do you consider this type of activity to be more of a time burden or bother or as something useful to you? Why? o What did you learn about your business from this exercise? Anything else? Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 112 Disengagement Wind up and close o Would you recommend repeating this activity? Why? If so, how often (e.g., how many times per year)? o How could this type of activity be improved? Why? ASK ONLY IF PROCESSOR WAS DROPPED FROM SAFE. NOTE: THE RESPONDENT MAY HAVE ALREADY MENTIONED THAT THE FIRM CEASED ITS WORK WITH SAFE, IF SO, ASK THESE QUESTIONS AT THAT POINT. We understand that your relationship with SAFE was ended either at your or SAFE’s request. If this true please tell me how/why it ended? Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 113 Wholesaler Discussion Guide Screening Question Introduction Suppliers Customers DISCUSSION GUIDE Make sure you are talking to the owner or a person who plays a primary role in the business and who can speak knowledgably about it. Introduce self, the purpose of the study and the equipment Reassure respondent of confidentiality Please tell me your name and position within…. STATE THE NAME OF THE BUSINESS YOU ARE INTERVIEWING (NOTE: SOME REPSPONDENTS MAY BE SELF￾EMPLOYED AND NOT HAVE A FORMAL BUSINESS NAME) What is your position in the business? How long have you been here in this role? What types of foods do you distribute/sell? From whom do you buy these foods? GET A FULL PROFILE OF SUPPLIERS FOR EACH FOOD TYPE MENTIONED. PROBE: - From manufactures - From middle men(describe) - From other distributors - From retailers How do you buy these foods? FOR EACH FOOD TYPE GET THE RESPONDENT TO DESCRIBE THE PURCHASING PROCESS, INCLUDING SUCH THINGS AS TRANSPORT, PACKAGING, PRICING, PAYMENT TERMS (E.G., CREDIT, CASH, TIMING) FOR EACH MAJOR SUPPLIER Why do you purchase from these suppliers? How important are things like the following? PROBE TO FIND OUT HOW IMPORTANT THE FOLLOWING ARE OR ARE AND WHY o Convenience/proximity o Price o Packaging (e.g., products provided in small quantities/packages that can afford and/or are easy to transport) o Credit o Product variety/selection o Knows and trusts seller o Other Who are your customers? GET A PROFILE OF CUSTOMERS (STORE TYPES, SIZES, LOCATIONS, ETC) What products do your customers buy from you? PROBE TO FIND OUT WHAT THE RELATIVE DEMAND IS FOR THE PRODUCTS SOLD BY THE RESPONDENT How often do your customers buy from you? PROBE FOR DUKAS, KIOSKS, OPEN MARKET SELLERS, LARGER CUSTOMERS o Daily? o Monthly? o Weekly? Why do your customers buy from you? How important are things like the following? PROBE TO FIND OUT HOW IMPORTANT THE FOLLOWING ARE AND WHY o Convenience/proximity o Delivery service o Price o Packaging o Payment terms CONFIRM WHETHER THE DELEAR PROVIDES CREDIT AND DISCUSS HOW THIS WORKS o Product variety/selection o Knows and trusts you o Other Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 114 Knowledge and awareness of nutritious foods Wind up and close When someone says ‘nutritious food,’ what do you think of? How do you know how nutritious a particular type of food is? What nutritious foods do you sell/distribute? Why these foods? OR Why do you not sell/distribute nutritious foods? From whom do you buy these nutritious foods? To whom do you sell these nutritious foods? To what extent do you care about the nutrition content of food? Why? To what extent do your customers care about the nutrition content of foods? Why? ASK RESPONDENT IF THEY HAVE ANYTHING FURTHER THEY WOULD LIKE TO SAY THEN THANK RESPONDENT Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 115 World Food Programme Discussion Guide Introduction Institution Background Background and Outcomes of Training Food Processing Sector Satisfaction and Plans for the Future Wind up and close DISCUSSION GUIDE Introduce self, the purpose of the study and the equipment Reassure respondent of confidentiality Ask respondent for their business card Please tell me your name and position within…. STATE THE NAME OF THE ORGANISATION YOU ARE INTERVIEWING How long have you worked here in this role? We understand that you (WFP) have worked with SAFE/TechnoServe to provide training to food processors/millers to build their capacity to supply the WFP. Can you confirm your knowledge of this collaboration? What was the context/background that led you to decide to (a) provide training to these food processors and (b) work with SAFE/TechnoServe to provide training to these processors? Please describe the nature of this collaboration, including the following: o The purpose of the training? o The location and participants in the training? o The type of training provided? o Who developed the training materials and delivered the training? How successful was the training in terms of: o Processors acquiring the capacity to supply the WFP? Why? o Processors successfully supplying the WFP at present? Why? o Processors’ capacity to supply the WFP over the long term? Why? Overall, how successful do you think the training has been? Why? What are the primary challenges facing the food processing sector in Kenya today? o In general? o Specific to WFP? What can be done to help food processors meet these challenges? o In general? o Specific to programs like SAFE? What is the WFP strategy to help food processors meet these challenges? Overall, how satisfied are you with your collaboration with SAFE/TechnoServe? Why? Do you have plans to continue collaborating with SAFE/TechnoServe and/or to continue providing these types of trainings to food processors in the future? Why? If so, how? What would you recommend to improve these types of trainings and/or collaborations in the future? Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 116 World Food Programme Training Discussion Guide Introduction Background Challenges Training content and impact Satisfaction Wind up and close DISCUSSION GUIDE  Introduce self, the purpose of the study and the equipment  Explain that participant that his/her enterprise was selected because it participated in a training offered by the World Food Programme and TechnoServe/SAFE. Please tell me your name and position within…. STATE THE NAME OF THE PROCESSOR YOU ARE INTERVIEWING How long have you worked here in this role? Please tell us about your firm, including: o Products produced and associated brand names o Number of total employees, male employees, and female employees at supervisory and non-supervisory levels  What are the primary challenges that a food processor such as your enterprise has in terms of supply to the World Food Programme? Why?  How were you recruited to participate in the training?  What were the topics covered at the training?  How relevant were the topics covered during the training to the challenges you facing supplying the World Food Programme?  How useful were the instructional materials and content?  How well were the instructional materials and content matched to your level of knowledge and experience and that of other training participants?  How effective was the training in terms of increasing your technical knowledge?  What type of guidance or assistance did the training provide you on how to implement the things discussed during the training?  What changes have you or your organization made, or what actions have you or your organization taken, as a result of the training?  What impacts have these changes or actions had on you organization in terms of your ability to supply the World Food Programme? Why?  Are you currently supply the World Food Programme? What and how much? How important is the World Food Programme to the success of your enterprise?  What challenges do you still face in terms of supply the World Food Programme? What kind of support do you expect to help you address these challenges?  Do you sell to other organizations? Why? If ‘Yes,’ please describe.  What other benefits did you receive from the training? (PROBE TO FIND OUT ABOUT OTHER BENEFITS IN TERMS OF SUCH THINGS AS PRODUCTION, SALES, EMPLOYMENT, INCOME, QUALITY, ETC.)  Overall, how satisfied were you with the training? Why?  What are the strengths and weaknesses of the training?  How can these types of trainings be improved? Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 117 E-Books Discussion Guide Introduction Institution Background Process Use of E-Book Satisfaction and Plans for the Future Wind up and close DISCUSSION GUIDE Introduce self, the purpose of the study and the equipment Reassure respondent of confidentiality Ask respondent for their business card Please tell me your name and position within…. STATE THE NAME OF THE ORGANISATION YOU ARE INTERVIEWING How long have you worked here in this role? We understand that your organization has worked with SAFE/TechnoServe to develop and/or use E-Books on topics related to food processing, fortification or business strategy. Can you please confirm whether this is the case? Please describe from your perspective what the E-book is and what its purpose is. Please describe the process used to develop this E-book, including the following: o Your background and expertise for working on this E-book? o How TechnoServe/Safe recruited you to work on this E-book? o The specific role you played in developing the E-book? o The nature and quality of your collaboration with other experts on this E-book? o The nature and quality of TechnoServe’s support to you and your colleagues working on the E-book, including communications, logistics support, and meetings/workshops. What problems did you encounter during this process and how did you or SAFE/TechnoServe deal with them? How effective is the E-book as a practical teaching tool? Why? (PROBE TO FIND OUT ABOUT SUCH THINGS AS TO WHETHER IT IS WRITTEN AT ANAPPROPRIATE LEVEL FOR THE TARGET MARKET, WHETHER IT COVERS THE APPROPRIATE TOPICS, WHETHER IT IS TRANSLATED INTO THE APPROPRIATE LANGUAGE, ETC.) As far as you know, how is the E-book being marketed to the targeted users? (PROBE TO FIND OUT WHAT MARKETING STRATEGIES ARE BEING USED TO ENSURE THAT THE E-BOOK IS BEING ACCESSED BY THE TARGET USERS AND WHETHER IT IS ULTIATELY ACCESSIBLE TO THESE USERS) Who do you see as the primary users of this E-book in the future? What do you think can be done to make the E-book a more effective teaching tool or to increase its impact? Why? (AMONG OTHER THINGS, PROBE TO FIND OUT WHETHER IT HAS BEEN SUFFICIENTLY VETTED BY EXPERTS, POTENTIAL USERS AND HAS BEEN SUBJECTED TO A PILOT TEST OF USERS.) Are you using, or do you plan to use this E-book yourself in the future? Why? If ‘Yes,’ how are you using it or how will you use it? What is the potential for scaling-up E-books across other organizations in the future? Why? Overall, how successful do you think the collaborations with SAFE/TechnoServe has been in terms developing this E-book? Why? Do you anticipate doing something like this again in the future with SAFE/Technoserve or another organization? Why? What would you recommend to improve the process used to develop this E-book in the future? New Product Development Manual Discussion Guide Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 118 Introduction Institution Background Process Understanding of Process Use of Manual DISCUSSION GUIDE  Introduce self, the purpose of the study and the equipment  Reassure respondent of confidentiality  Ask respondent for their business card  Please tell me your name and position within…. STATE THE NAME OF THE ORGANISATION YOU ARE INTERVIEWING  How long have you worked here in this role?  We understand that your organization has worked with SAFE/TechnoServe to develop and/or use a manual on New Product Development. Can you please confirm whether this is the case and what your specific role has been? ASK THE PROCESS QUESTIONS ONLY IF THE RESPONDENT PARTICPATED IN THE DEVELOPMENT OF THE NEW PRODUCT DEVELOPMENT MANUAL  Please describe from your perspective what the manual is and what its purpose is.  Please describe the process used to develop this manual, including the following: o Your background and expertise for working on this manual? o How TechnoServe/Safe recruited you to work on this manual? o The specific role you played in developing the manual? o The nature and quality of your collaboration with other experts on this manual? o The nature and quality of TechnoServe’s support to you and your colleagues working on the manual, including communications, logistics support, and meetings/workshops.  What problems did you encounter during this process and how did you or SAFE/TechnoServe deal with them?  Overall, how successful do you think the collaborations with SAFE/TechnoServe has been in terms developing this manual? Why?  Do you anticipate doing something like this again in the future with SAFE/Technoserve or another organization? Why?  What would you recommend to improve the process used to develop this manual in the future? ASK THE UNDERSTANDING OF PROCESS QUESTIONS ONLY IF THE RESPONDENT DID NOT PARTICIPATE IN DEVELOPING THE MANUAL,  To the best of your knowledge, please describe the process used to develop the manual.  What would you recommend to improve the process used to develop this manual in the future?  Are you using, or do you plan to use this manual yourself in the future? Why? If ‘Yes,’ how are you using it or how will you use it?  How effective is the manual as a practical teaching tool? Why? (PROBE TO FIND OUT ABOUT SUCH THINGS AS TO WHETHER IT IS WRITTEN AT ANAPPROPRIATE LEVEL FOR THE TARGET MARKET, WHETHER IT COVERS THE APPROPRIATE TOPICS, WHETHER IT IS TRANSLATED INTO THE APPROPRIATE LANGUAGE, ETC.)  As far as you know, how is the manual being marketed to the targeted users? (PROBE TO FIND OUT WHAT MARKETING STRATEGIES ARE BEING USED TO ENSURE THAT THE MANUAL IS BEING ACCESSED BY THE TARGET USERS AND WHETHER IT IS ULTIATELY ACCESSIBLE TO THESE USERS)  Who do you see as the primary users of this manual in the future?  What do you think can be done to make the manual a more effective teaching tool or to increase its impact? Why? (AMONG OTHER THINGS, PROBE TO FIND OUT WHETHER IT HAS BEEN SUFFICIENTLY VETTED BY EXPERTS, Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 119 Wind up and close POTENTIAL USERS AND HAS BEEN SUBJECTED TO A PILOT TEST OF USERS.)  What is the potential for scaling-up manuals across other organizations in the future? Why? Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 120 Financial Institutions Discussion Guide Introduction Institution Background Background and Outcomes of Training Food Processing Sector Satisfaction and Plans for the Future Wind up and close DISCUSSION GUIDE Introduce self, the purpose of the study and the equipment Reassure respondent of confidentiality Ask respondent for their business card Please tell me your name and position within…. STATE THE NAME OF THE ORGANISATION YOU ARE INTERVIEWING How long have you worked here in this role? We understand that your organization has worked with SAFE/TechnoServe to provide information, capacity development and/or financing to SAFE-assisted food processors. Can you confirm whether this is the case? What was the context/background that led you to collaborate with SAFE/Technoserve? Please describe the nature of this collaboration, including the following: o The purpose? o The activities implemented or services provided? o The relative roles played by SAFE/Technoserve and your organization? How successful was the collaboration? Why? (PROBE TO FIND OUT WHAT OUTCOMES RESULTED FROM THE COLLABORATION) Have any SAFE-assisted food processors received financing from your organization? Why? If so, please describe who and the details of the financing. Overall, how successful do you think the collaboration with SAFE/Technoserve has been? Why? What are the primary challenges facing the food processing sector in terms of accessing finance? What can be done to help food processors meet these challenges? o In general? o Specific to your organization? o Specific to programs like SAFE? What is your organization’s strategy to help food processors meet these challenges? Overall, how satisfied are you with your collaboration with SAFE/Technoserve? Why? Do you have plans to continue collaborating with SAFE/Technoserve and/or to continue to work with food processors in the future? Why? If so, how? What would you recommend to improve these types of collaboration in the future? Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 121 Retailer Discussion Guide Screening Question Introduction Suppliers Customers DISCUSSION GUIDE Make sure you are talking to the owner or a person who plays a primary role in the business and who can speak knowledgably about it. Introduce self, the purpose of the study and the equipment Reassure respondent of confidentiality Please tell me your name and position within…. STATE THE NAME OF THE BUSINESS YOU ARE INTERVIEWING (NOTE: SOME REPSPONDENTS MAY BE SELF￾EMPLOYED AND NOT HAVE A FORMAL BUSINESS NAME) What is your position in the business? How long have you been here in this role? What types of foods do you distribute/sell? From whom do you buy these foods? GET A FULL PROFILE OF SUPPLIERS FOR EACH FOOD TYPE MENTIONED. PROBE: - From manufactures - From middle men - From other retailers How do you buy these foods? FOR EACH FOOD TYPE GET THE RESPONDENT TO DESCRIBE THE PURCHASING PROCESS, INCLUDING SUCH THINGS AS TRANSPORT, PACKAGING, PRICING, PAYMENT TERMS (E.G., CREDIT, CASH, TIMING) FOR EACH MAJOR SUPPLIER Why do you purchase from these suppliers? How important are things like the following? PROBE TO FIND OUT HOW IMPORTANT THE FOLLOWING ARE OR ARE AND WHY o Convenience/proximity o Price o Packaging (e.g., products provided in small quantities/packages that can afford and/or are easy to transport) o Credit o Product variety/selection o Knows and trusts seller o Other Who are your customers? GET A PROFILE OF CUSTOMERS (GENDER, AGE, SEC WORKING, FAMILY SIZE) What products do your customers buy from you? PROBE TO FIND OUT WHAT THE RELATIVE DEMAND IS FOR THE PRODUCTS SOLD BY THE RESPONDENT How often do your customers shop with you? FOR DUKAS, KIOSKS, OPEN MARKET SELLERS ONLY o Daily? o Monthly? o Weekly? Why do your customers buy from you? How important are things like the following? PROBE TO FIND OUT HOW IMPORTANT THE FOLLOWING ARE AND WHY o Convenience/proximity o Price o Packaging (e.g., products provided in small quantities/packages that can afford and/or are easy to transport) o Credit CONFIRM WHETHER THE SHOP PROVIDES CREDIT AND DISCUSS HOW THIS WORKS o Product variety/selection o Knows and trusts you o Other When someone says ‘nutritious food,’ what do you think of? Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 122 Knowledge and awareness of nutritious foods Wind up and close How do you know how nutritious a particular type of food is? What nutritious foods do you sell/distribute? Why these foods? OR Why do you not sell/distribute nutritious foods? From whom do you buy these nutritious foods? To whom do you sell these nutritious foods? To what extent do you care about the nutrition content of food? Why? To what extent do your customers care about the nutrition content of foods? Why? ASK RESPONDENT IF THEY HAVE ANYTHING FURTHER THEY WOULD LIKE TO SAY THEN THANK RESPONDENT Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 123 Consumer Discussion Guide Introduction and warm up Shopping behavior Shopping outlets NEIGHBORHOOD SHOPS DISCUSSION GUIDE Introduce self, the purpose of the study and the equipment, reassure respondents of confidentiality Ensure that respondents are relaxed and that they know  No right or wrong answers; want to hear their personal opinions • Be honest; want to know what they really think • We want to hear from everyone – so don’t be shy • No need to raise your hand • One person talks at a time Ask respondents to introduce themselves with their name, what they do and some information about their shopping behavior. As I said, we are here to talk about shopping, please tell me about your shopping behavior. Are there certain items you buy at particular stores or times or frequency? PROBE  Monthly/weekly shop  Day to day shopping  Food shopping Now thinking about the places you shop can you please explain what each of these terms mean to you o Convenience stores o Minimarkets o Open markets o Kiosks o Neighborhood shops o Supermarkets PROBE ANY OTHER TERMS THAT MAY HAVE COME UP ASK FOR EACH • What type of people generally shop at each of these? • What are the advantages/disadvantages of each? • On what occasions do you shop at each? • What are the typical products you purchase at EACH OF THE ABOVE? Now thinking about your monthly shopping, what proportion of your shopping VISITS is done at each of the above (NOTE MUST ADD UP TO 100%) In total over the month, what proportion of your monthly shopping budget is spend at each of the above store types (NOTE MUST ADD UP TO 100%) Of your total monthly shopping budget, how much is spent on food purchases? ASK QUESTIONS BELOW FOR THE TOP 3 STORE TYPES THAT THEY GENERALLY SHOP AT: Now thinking specifically about your visits to neighborhood shops, please tell me • How often do you visit a neighborhood shop? • What kind of shopping do you do at your neighborhood shop? o Daily? o Monthly? o Weekly? • What items do you purchase in your neighborhood shops? o How often do you purchase them? o Why do you purchase them here? Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 124 KIOSKS OPEN AIR MARKETS ALL OTHER STORE TYPES MENTIONED IN TOP 3 • When you purchase food items in your neighborhood shop, what factors do you consider and how important is each one? PROBE TO DETERMINE HOW IMPORTANT EACH OF THE FOLLOWING IS o Convenience/proximity o Price o Packaging (e.g., products provided in small quantities/packages that can afford and/or are easy to transport) o Shop provides credit o Product variety/selection o Knows and trusts proprietor o Other Now thinking specifically about your visits to kiosks, please tell me • How often do you visit kiosks? • What kind of shopping do you do at kiosks? o Daily? o Monthly? o Weekly? • What items do you purchase in kiosks? o How often do you purchase them? o Why do you purchase them here? • When you purchase food items in kiosks, what factors do you consider and how important is each one? PROBE TO DETERMINE HOW IMPORTANT EACH OF THE FOLLOWING IS o Convenience/proximity o Price o Packaging (e.g., products provided in small quantities/packages that can afford and/or are easy to transport) o Shop provides credit o Product variety/selection o Knows and trusts proprietor o Other Now thinking specifically about your visits to open air markets, please tell me • How often do you visit such markets? • What kind of shopping do you do at open air markets? o Daily? o Monthly? o Weekly? • What items do you purchase in open air markets? o How often do you purchase them? o Why do you purchase them here? • When you purchase food items in open air markets, what factors do you consider and how important is each one? PROBE TO DETERMINE HOW IMPORTANT EACH OF THE FOLLOWING IS o Convenience/proximity o Price o Packaging (e.g., products provided in small quantities/packages that can afford and/or are easy to transport) o Shop provides credit o Product variety/selection o Knows and trusts proprietor o Other Now thinking specifically about your visits to ….SPECIFY….., please tell me • How often do you visit this store type? • What kind of shopping do you do there? o Daily? o Monthly? o Weekly? • What items do you purchase there? Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 125 Nutritious foods o How often do you purchase them? o Why do you purchase them here? • When you purchase food items here, what factors do you consider and how important is each one? PROBE TO DETERMINE HOW IMPORTANT EACH OF THE FOLLOWING IS o Convenience/proximity o Price o Packaging (e.g., products provided in small quantities/packages that can afford and/or are easy to transport) o Shop provides credit o Product variety/selection o Knows and trusts proprietor o Other  When someone says ‘nutritious food,’ what do you think of?  How do you know how nutritious a particular type of food is?  Do you try to find out the nutrition value of foods before buying them? Why or why not?  How important is the nutritional content of food to you when you make food￾purchasing decisions? Why?  How easy is it to find nutritious foods in the markets?  If you wanted to purchase nutritious food, where would you go to purchase it? What nutritious foods do you purchase? Wind up and close Does anyone want to share anything else or are there any questions? Do you have suggestions on how we can change our session next time to better fit your expectations? Thank and hand out incentives. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 126 ANNEX 6: PROCESSOR SURVEY PROCESSOR NAME INTERVIEW DATE 1. Please rate the following aspects of the SAFE program Very poor Poor Good Very good n/a a Knowledge of local Technoserve staff b Helpfulness of local Technoserve staff c Knowledge of PFS volunteers d Helpfulness of PFS volunteers e Provision of assistance through Skype or other long distance means 2. How much change have you made in your business as a result of the assistance you received: No change A little change Some change Significant change n/a a Raw materials sourcing b Production c Quality d Management capacity e Technical capacity f Sales g Profits h Investment i Commercial relationships 3. Which of the following areas of your business has grown as a result of this assistance: No increase Some increase Significant increase n/a a Number of product lines b Productivity c Sales d Profits e Product quality f Investment 4. How satisfied were you with Very dissatisfied Not satisfied Satisfied Very satisfied n/a a The recruitment/enrolment process b The initial information provided about the SAFE program c The enrolment event (where you met other SAFE processors) d The initial assessment of your business conducted by SAFE e The process used to identify early win projects f The selection of early win projects g The assistance of Technoserve on early win projects h The outcomes for early win projects Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 127 5. What impact did the early win projects have on: No impact Some impact Significant impact n/a a Labour costs b Production efficiency c Product quality d Compliance with standards/regulations e Sales f Income 6. How many new jobs have you created as a result of the SAFE program overall Male Female a Full time jobs b Part time jobs 7. Overall how satisfied are you with Very dissatisfied Not satisfied Satisfied Very satisfied n/a a The process used to identify and assess projects b The assistance from Technoserve staff c The outcomes from Technoserve assistance d The assistance from PFS volunteers e The outcomes from PFS assistance 8. Not at all useful Not very useful Somewhat useful Very useful n/a Overall how useful was the assistance you received through the program Overall how useful was the initial SAFE assessment of your company Overall how useful was the recent SAFE impact assessment program to your company 9. As best you can tell, to what extent does the food you produce for which you receive SAFE assistance reach people vulnerable to food insecurity or malnutrition? Not at all Not very much A good amount A lot n/a PLEASE TICK APPROPRIATE RESPONSE FOR EACH QUESTION Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 128 ANNEX 7: IN-DEPTH CASE STUDY FINDINGS Findings from the Household Survey Introduction Kenya: The evaluation team conducted an in-depth case study of Classic Foods in Kenya. Classic Foods produces three nutritious food products that it sells in, among other outlets, lower-end retail chains and independent shops in a variety of settlement areas in and around Nairobi. The three Classic Foods products and brands covered by the case study include Classic Porridge, Capital Flour (sifted maize meal) and Classic Fresh Milk. For the case study, the evaluation team conducted the following research activities: (1) four consumer FGDs in Kayole (an area selected because of its proximity to Classic Foods) and Kibera with low-income women responsible for household food purchases, (2) four KIIs in Kayole and four KIIs in Kibera with retail shop/store owners ranging from small general stores to supermarkets and including one wholesaler/retailer of dry goods, and (3) 438 household surveys with randomly selected households in 10 communities served by Classic Foods. Respondents to the household survey included the person in the household responsible for making food-purchasing decisions. The 10 communities included: Juja, Thika, Kibera, Ruiru, Kibera, Kayole, Kitengela, Tassia, Mwiki, Huruma and Athi River. Fieldwork for the in-depth case study commenced on 19th March and ended on 23rd March 2016. Zambia: The evaluation team conducted an in-depth case study of COMACO in Zambia. COMACO produces two nutritious food products that it sells in the Eastern Province of Zambia. The two COMACO products and brands covered by the case study include It’s Wild Peanut Butter and It’s Wild Yummy Soy (a corn soy blend). For the case study, the evaluation team conducted the following research activities in the Eastern Province: (1) four consumer FGDs in Chipata with lower-income women responsible for household food purchases, (2) KIIs with four retailers, three wholesalers and one retailer/wholesaler in Chipata, and (3) 464 household surveys with randomly selected households in two communities served by COMACO: Chipata and Katete. The baseline survey included a third community, Lundazi, but due to budget constraints in the endline, Lundazi was dropped from the survey. Respondents to the household survey included the person in the household responsible for making food-purchasing decisions. Fieldwork for the in-depth case study took place during March 21-26, 2016. Unlike most of the food processors interviewed, COMACO is a large processor producing a wide variety of food products, including peanut butter, rice, honey, corn soy blends (CSB), beans and animal feed. COMACO sells its products under the brand name ‘It’s Wild.’ It’s major customers are the large national retail chains Shoprite, Pick n’ Pay, Spar and Melisa that have outlets throughout the country. COMACO also sells to NGOs for school feeding programs. Owing to its relative size and level of sophistication, COMACO’s interaction with SAFE has been less concerned with the basics of hygiene and food safety and more with improving the product and growing the market. Among COMACO’s product line, Yummy Soy is the only product that is it attempting to sell within food insecure retail markets, although it is also currently looking at different packaging options to make its peanut butter more affordable to lower income groups. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 129 Food Security Kenya: Compared to last year’s 24%, the proportion of food-insecure respondents in the study locations has risen to 53%; 59% have experienced lack of resources for food 3 or more times during the past 4 weeks, while another 23% had to go to bed without food. Of these, 61%, experienced this rarely, 14% had to make do without food for a full 24 hour period, and a third of those experienced this three or more times. The rise in the proportion of food-insecure respondents can be adduced to two factors: (1) during the same period in 2015, government sponsored National Youth Service Projects, which provided employment across the study areas, was suspended, leaving many without jobs, and (2) the cost of living has increased sharply, leading to basic commodities being more expensive, resulting in consumers having to do without. Zambia: Food Security has worsened in the two study locations since 2015, for a few reasons. First, the lack of resources to get food increased from 30% to 46%. Second, going to bed hungry increased from 19% to 40%. Lastly, going without food for a full 24-hour period increased from 13% to 33%. Chipata has suffered worse from this deterioration than Katete. Zambia as a whole has experienced severe drought conditions caused by El Niño. As the population in Eastern Zambia is dependent mainly on subsistence farming, characterized by traditional methods and rain irrigation, this drought has affected food security. Staple foods are now being rationed (e.g. one pack of maize meal per customer). Food Items Consumed Kenya: The general pattern of food items consumed during the previous day has basically remained unchanged; yet, there is increased consumption of dairy products, meats and fish as well as vegetables and fruit. Consumption of sweets, on the other hands, has declined from 78% in 2015 to 68% in the current study. There are no major differences in the diets of men and women; however, food-insecure families are less likely to consume Vitamin A rich vegetables and fruits, eggs and flesh meats. Zambia: While the pattern of food items consumed has remained stable since 2015, a few items, namely oils and fats and dark green leafy vegetables, have reduced across both locations: (1) oils and fats declined from 90% to 75% and (2) dark green leafy vegetables declined from 85% to 69%. However, consumption of other items, such as other vegetables or flesh meats, has increased in Katete, but declined in Chipata. Clearly, consumers in Katete are less affected by the drought and its economic duress than consumers in Chipata. Furthermore, food-insecure consumers, as well as women, have to make do with lower levels of consumption of certain food items than do food-secure respondents or men. Among these items are vegetables, meats, dairy products, legumes and eggs. Familiarity with Nutritious Foods Kenya: Claimed familiarity with the concept of nutritious food has increased marginally against the previous year: the current mean score (on a 4-point scale) stands at 2.95, whereas it came in at 2.83 in 2015. Modest this increase may be; yet, it still is significant at the 95% confidence level. Zambia: Consumers claim greater familiarity with nutritious foods than they did in 2015. The mean score of the familiarity rating has risen from 2.57 (on a 4-point scale) to 2.79, and this is statistically significant at the 99% level of confidence. Importance of Different Types of Retail Outlets Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 130 Kenya: Questions on patronage of retail outlets were newly inserted in the current study; hence, no comparative data exist for 2015. Kiosks were rated the most important retail outlets with 93%, followed by supermarkets (69%) and open markets (64%). There exist important differences in the level of patronage among these three types of retail outlets. Supermarkets are frequented about once a week; they account for just 7% of all shopping occasions. Yet, since consumers shop in bulk, the average expenditure is higher at supermarkets than for any other retail outlet (1,715 KES), which results in a value share of 25%. Kiosks, on the other hand, are visited almost daily (five times per week), Thus, kiosks account for 64% of all shopping occasions, but the expenditure per visit is low (355 KES), which reduces the value share of kiosks to 47%. Zambia: Open markets are the most important type of outlets (first mention: 89%; total mention: 99%). Kiosks and neighborhood stores follow with 79% and 78%, respectively. Open markets are used to procure all the households’ food items; only meats and fish are sourced from butchers. Open markets are frequented three times per week, giving them a 47% share of shopping occasions. On average, ZMK 58 are spent per visit, which results in open markets attracting a value share of 62%. The main competitive advantages of kiosks and, to a lesser extent, neighborhood stores, is the opportunity to get credit facilities (although 23% of consumers claim that no outlet type provides those) and the ability of purchasing parts of branded packs. Consumers shop for the same items at both supermarkets and kiosks: dairy products, cereals, cooking oils and eggs. For fresh vegetables, legumes and fruit, they go to open markets, while meats are procured from butchers. The consumer FGDs asked respondents about four types of retail outlets: supermarkets, kiosks and neighborhood shops, open markets, and mini-markets and convenience stores. Their responses are summarized below. Shopping Habits Kenya: Consumers shop for the same items at both supermarkets and kiosks: dairy products, cereals, cooking oils and eggs. For fresh vegetables, legumes and fruit, they go to open markets. Meats are procured from butchers. Zambia: Consumers shop for the same items at both supermarkets and kiosks: dairy products, cereals, cooking oils and eggs. For fresh vegetables, legumes and fruit, they go to open markets. Meats are procured from butchers. Outlet Evaluation Kenya: Supermarkets are commended for stocking a wide variety of products, selling branded portion packs and selling at affordable prices. Kiosks, on the other hand, are not known for low prices. They are, however, attractive to consumers because they provide credit facilities. Kiosks are also located in convenient, easy to reach locations, whereas supermarkets require trips on public transport. Furthermore, kiosks will break up larger packs and sell portions from those packs. For example, instead of having to buy the full 2kg pack of maize meal, consumers can purchase the amount they require for their next meal. On the downside, consumers have less control over the freshness of products sold in kiosks. Zambia: In line with their expressed preference for open markets, 70% of consumers claim that these are “outlets for people like me.” Open markets are associated with wide choice, freshness, affordability and the sale of branded portion packs; however, 13% claim that none of the outlet types offer branded Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 131 portion packs. This points at a marketing opportunity, as clearly there is a demand for portion packs. Open markets, however, do not make it easy for consumers to determine whether products have expired. Kiosks have the edge over open markets when it comes to opening branded packs and buying parts of it. They are also slightly more likely to offer credit facilities. However, almost a quarter of consumers (23%) complain that none of the outlet types offer credit facilities. Access to Growing Areas Kenya: Eighty-four percent of consumers in the study locations do not have access to places where they might grow some of their own food, 3% have an area close-by, and 12% have a place up-country. Nonetheless, there is no strong relationship between access to a growing area and the proportion of food that is grown on one’s own. Across the locations, 77% of the households’ food is store-bought and 23% is home grown. Even with no access to one’s own growing area, a certain proportion of food is not store￾bought but is provided by family or friends with their own growing areas. Zambia: Overall, 67% of respondents have access to areas where they can grow food (50% nearby; 17% up-country). These growing areas are a significant source of food for households that have them: 42% of their food is home-grown, while 58% are bought in stores. Households that do not have access to their own food growing areas need to procure 97% of their food from open markets or stores. Purchase Decision Factors Kenya: Cost and quality considerations remain the most important factors in decision-making. With the exception that adults’ preferences now counting more than children’s preferences, the ranking of factors remains unchanged from the baseline. However, ratings come in higher across the board than in the baseline (significant at 99%) without, however, affecting consumers’ priorities. Zambia: Across demographic groups and locations, cost is the most important consideration, achieving a mean rating of 6.52 out of a possible 7. Branding is the least important, with a mean score of 4.33. In Chipata, cost and availability have become significantly more important since 2015 (at 99%), while this also holds true in Katete. Taste, quality and nutrition value have also become significantly more important in that location. This points to consumers in Chipata being more strained financially than consumers in Katete. Performance of Study Brands Kenya: The Kenyan case study focused on three brands: Classic Fresh Milk, Classic Porridge and Capital Flour. Of these brands, only Classic Fresh Milk achieves usable readings:  Brand Awareness has increased from 13% to 16%.  Trial levels have remained unchanged at 4%.  Usage, however, has further contracted from 2% in 2015 to 1% in the current study. With just three users of Classic Fresh Milk, further brand-wise analysis was not possible. Classic Porridge and Capital flour recorded no trial or usage at all. Hence, the processor seems to have made no efforts during the past one year to resolve marketing and distribution issues. The processor needs more guidance beyond instructions on how to produce enriched products in the areas of marketing and distribution. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 132 Zambia: Two COMACO brands, It’s Wild: Peanut Butter and It’s Wild: Yummy Soy, were the focus of the case study. Both brands have increased their awareness, trial and usage shares since last year; however, the conversion ratios from awareness to trial and from trial to usage have decreased. That is, while the brands are growing, they have not yet consolidated in terms of inducing trial or motivating adoption into consumers’ repertoires. The most important source of awareness remains word of mouth followed by TV and radio advertisement. Road shows and billboards are negligible sources of awareness in Chipata, while in Katete, both media recorded gains and now are the third and fourth-most important media. Brand Ratings Kenya: The three brands of interest (Classic Fresh Milk, Classic Porridge, Capital Flour) are not just stagnating at last year’s poor levels, but deteriorating even further in their attractiveness to consumers – a mere 3 respondents use Classic Fresh Milk; none use Classic Porridge or Capital Flour. Although the popular brands had lower readings in terms of awareness, trial and usage, their conversion ratios have, on the whole, remained fairly steady. KCC My Choice even improved its conversion ratio from trial to usage, despite deteriorating ability to induce trial. Although the quality of Classic Fresh Milk has improved (3.22 vs. 2.76 in 2015), the cost is rated worse (2.08 vs. 2.45 in 2015). Ratings of nutritional value and taste of Classic Fresh Milk have actually improved slightly: (1) nutrition 3.25 vs. 3.0 and (2) taste .25 vs. 3.06. Yet, the number of users has drastically collapsed to a mere 3 respondents. Hence, further brand-wise analysis is not possible. Zambia: As was the case in 2015, It’s Wild: Peanut Butter is rated significantly better in terms of quality and cost than It’s Wild: Yummy Soy. While cost ratings for It’s Wild: Peanut Butter have improved significantly across both locations, quality ratings have improved only in Katete. There were no significant changes in the ratings of It’s Wild: Yummy Soy in either location. On nutritional value and taste, again, It’s Wild: Peanut Butter is preferred significantly to It’s Wild: Yummy Soy. However, It’s Wild: Yummy Soy significantly improved its taste ratings in both locations and in Katete also for nutritional value. Both brands have improved their acceptability ratings among both adults and children. By the same token, respondents are now more likely to recommend both brands or to buy them again. On overall satisfaction, It’s Wild: Peanut Butter remains steady since 2015; however, in Katete, it improved its satisfaction rating significantly. It’s Wild: Yummy Soy shows a significant improvement in overall satisfaction compared to 2015. Frequency of Brand Purchase Kenya: Not enough observations to report a finding. Zambia: Sixty percent of consumers of It’s Wild: Peanut Butter purchase the brand once a month or less. The preferred pack size is 1kg, which sells for ZMK 20.86. The main obstacles to frequent purchases are the perceived high cost of It’s Wild: Peanut Butter as well as inconsistent availability. Some also consider the brand “cheap” or of low quality. Fifty-five percent of consumers buy It’s Wild: Yummy Soy once a month or less often; however, a quarter (23%) buy once a week. The majority (73%) buy the 1kg pack, which sells at ZMK 13.52. The main obstacles for more frequent purchase are unavailability (54%), low quality (27%) and high cost (18%). Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 133 Findings from the Consumer FGDs The consumer FGDs examined respondents’ shopping habits, including where they purchased food items, what the purchased, how often they purchased it, and what factors determined their purchasing decisions. The consumer discussions centered around five different types of retail outlets that one is likely to find in the urban settlement areas or rural zones where the case study research took place and where respondents shop: supermarkets, kiosks, neighborhood shops, open air markets, and mini￾markets/convenience stores. Characteristics of Retail Outlets where FGD Respondents Shop Supermarkets Kenya: Respondents prefer to do their bulk shopping at supermarkets because of the wide variety of products, pack sizes and brands on display. Supermarkets also make it easy to compare prices and thus get the best value for money. Those in formal employment (with a monthly wage) generally do a monthly bulk shop for items such as maize meal, cooking oil, sugar, rice and tealeaves. For those who do not have a regular paycheck (self-employed or causal workers), the bulk shop is done when funds allow. Although respondents only use supermarkets once or twice a month, they spend about a third of their monthly budgets here. Zambia: Respondents are so cash strapped that they rarely venture into supermarkets. They believe these are for “wealthy” people who have regular monthly salaries. Only such people, in their opinion can afford to buy in the quantities offered at such stores. Also, these stores are not conveniently located for the FGD respondents.  “Things that are found there go for 25K, 30 something, 50 something, so if you have 10 Kwacha what will you buy?”  “They don’t sell in small quantities and there is no credit.” Kiosks and Neighborhood Shops Kenya: Most (over 50%) of the monthly food budget is spent at these retail outlets. Kiosks are common in the residential areas and are usually semi-permanent structures (informal trade). They are close to home and sell all the basic foodstuffs. The bulk of perishables (bread, milk, vegetables) are bought here and because the shoppers do not have fridges, they buy goods as they need them visiting such outlets once or twice a day. Respondents buy fresh food items (such as bread or milk) a day-to day-basis usually from a favored local kiosk. These outlets also sell vegetables and some will even chop these vegetables to order. Neighborhood shops are permanent structures but otherwise similar in function to kiosks. They sell a wide range of items in small quantities. Customers expect to find the same range of items as in kiosks but in addition these shops will also sell cleaning and personal hygiene products. Generally customers do not enter the store but are served through a window. Both kiosks and neighborhood shops will break down pack sizes to suit the needs of the customer, and both offer informal credit to regular customers. Zambia: Ntembas (kiosks) are the most widely used retailer by Zambian FGD respondents. Ntembas are small stalls (tables or benches) or makeshift stores. They are located in the neighborhood and sell a wide variety of household necessities. However, the Ntemba trader is willing to break down retail packs to Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 134 very small quantities specifically to meet the financial needs of the buyer. This means the shopper can afford the range of products required to make one meal (charcoal, cooking oil, a tomato, some greens and some maize meal) even with only a very small amount of cash. Otherwise, if she had to buy a standard retail pack she would get only the maize meal.  “We stay in a compound called Magazine, and it’s far from the market you need to get on a bus to get there, so if you have a K20 you will just decide to send kids to buy from the Kantemba like small packed sugars and so on.”  ”When you do some part time works and you are given a K15 you won’t be able to go to town with that because getting on a bus is K10 so you can’t go with K5 into town so you just buy from the Kantemba that are found within the area and buy the small fish for K2 or soap and salt.” Open Markets Kenya: Open markets are used when shopping for bulk vegetables and pulses and also non-perishables such as clothes or shoes. Here products can be obtained cheaper and are considered fresher but the shopping experience is not very pleasant and it is time consuming. Markets are often out-of-the-way and require transport costs. The haggling with stallholders for the best price can be harsh. The environment is crowded and hot and dusty (or wet and muddy). Security is an issue with pick-pocketing common. Nonetheless up to 15% of monthly budget is spent here. Zambia: For Zambian shoppers, open markets are the preferred retail outlet. Products are fresher and prices lower than any other outlet type. Open markets offer a wide range of products, and because of the concentration of vendors at the markets, consumers have bargaining power. However open markets may be some distance from home, so the cost of transportation has to be factored in, and for this reason they are not the outlet type used most often. Due to lack of funds, most Zambian respondents claimed to buy very small quantities of all items (even soap and charcoal). They buy what they can afford on a day-to-day basis as funds become available. Thus the trip to the market (which involves bus fares) is only undertaken when they have a windfall and cash is more available. Open markets are better for fresh products (vegetables and meat) and have a much wider range of products.  “So we go to the market where we can buy in small quantities even with chicken we buy from the market the chicken is cut in small pieces.”  ”At open markets you even find products that are packaged in smaller quantities. There is measured mealie meal (measured using cups) we are able to buy what we can afford with the monies that we find.” Mini-markets and Convenience Stores Kenya: These are not outlets that our consumers use regularly. They have some of the advantages of supermarkets (modern, self-service, goods priced for easy comparison) but prices are too high. Zambia: Respondents are so cash strapped that they almost never venture into mini-markets or convenience stores. They believe these are for “wealthy” people who have regular monthly salaries. Only such people, in their opinion can afford to buy in the quantities offered at such stores. Also, these stores are not conveniently located for the FGD respondents. The table below summarizes the key characteristics, shopper profile and shopping frequency of the four types of retail outlets discussed above. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 135 Profile of Retail Outlets Type of Retail Outlet Key Characteristics Shopper Profile Shopping Frequency Kiosks (Ntembas)  Convenience, close to home  Wide range of products  Affordable, will sell very small quantities so low output of funds  Flexible-can bargain and may allow informal credit to regulars  Expired goods-no way to check expiration dates as they do not sell goods in original packs  Unhygienic-no access to clean water and dusty/dirty environment  Neighbors  Anyone who needs small quantities  Poor people  In Kenya also popular with lower/middle income consumers  Kenya-2-3 times a day  Zambia-Day-to-day as funds permit Neighborhood Stores  Convenience-close to home and stay open late  Pocket friendly-small quantities  Credit-allow informal credit to regulars  Short-change customers￾tamper with scales/water down milk  Old stock-some products on the shelves too long (e.g., sour milk, stale bread)  Neighbors  Anyone who needs small quantities  Popular with lower/ middle income consumers  Casual workers who do not have monthly salaries  Daily Open Markets  Crowded, noisy and insecure (e.g., pickpockets)  Wider variety of products at cheaper prices￾bargaining allowed  Small quantities— repackaged into small amounts  Unhygienic-food is uncovered in dusty/dirty environment  Inconvenient locations￾high transport costs  Time consuming  Cheat on quantities  At times expired or spoiled products  All types of people  Kenya-Once a week but not during rainy season  Zambia-Irregular, only when funds allow, but ideally weekly Supermarkets  Prices clearly marked  Large choice of goods  Cheaper per unit prices  People with regular income (e.g., employed, government workers)  Kenya-Once or twice per month  Zambia-Not at all Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 136  Large pack sizes, so higher overall cost  Can walk around and window shop  Encourages impulse purchases  No credit  Inconvenient locations and opening hours  Clean and hygienic Mini-Markets and Convenience Stores  Not used by target market  Not used by target market  Not used by target market Price and Affordability Kenya: Fresh food is generally purchased on a day-to-day basis, while non-perishable items are bought either weekly or monthly. The monthly (bulk) shopping is done when funds become available and to provide a feeling of ‘security.’ Supermarkets offer the best prices for non-perishables. Open markets offer the best prices for fruits and vegetables. Kiosks and neighborhood shops are the most expensive, but because they will sell in small quantities (half a loaf or a cup of sugar), they are frequently the most affordable. They also offer short-term credit to regular customers. For consumers on irregular incomes kiosks and neighborhood shops are often the only option. Zambia: The respondents in Zambia seem to be in dire straits financially. Purchasing power among the FGD respondents is extremely low (e.g., less than 50 Kwacha a month) and the inability to afford even basic food is quite common. Food is thus generally purchased on a day-to-day basis, as and when funds become available, and the quantities purchased per shopping occasion are very small. They cannot take it for granted that they will eat every day and when they do eat it is usually quite basic greens and maize meal (nshima). The greens are generally grown at home (only 22% do not grow their own vegetables).  “For me when I have a kwacha I think to myself what will I buy? So I start off and go to a Ntemba (makeshift stall) at the market to buy Munkwani (pumpkin leaves) just one bundle.”  “I buy small packs according to the money that I have. Small packs of soap, sugar…”  “I only go shopping after I have done some part time work and am paid afterwards. I would be paid maybe a 5 Kwacha then I go to the market and buy what is needed at home like small packaged items say 1 Kwacha sugar, 1 Kwacha for salt then I get tomatoes then that is my shopping.”  “I will balance it, I will make sure I buy charcoal for 1 Kwacha, I buy tomato 50 Ngwee, cooking oil 50 Ngwee, and I buy pumpkin leaves 1 kwacha and I balance with a candle for 1 kwacha, for the children to eat. Mainly we eat once a day.” What else they eat is determined by how much money they have on hand and what other calls they have on these funds. They do not have cash reserves and thus buy just enough to make each meal. Hence, they are unable to buy packaged products from formal retail outlets and rely on retailers who are willing to sell them small quantities; a cup of maize meal, a half kwacha of salt, one kwacha of sugar. Not surprisingly, therefore, price is the key determinant in food purchasing decisions. What to buy, where to buy it and how much to buy are all heavily influenced by how much money is available.  “Mostly we look at the price of the commodity so if it’s cheaper somewhere no matter how far the place is we make sure we walk to get the cheaper goods.”  “The thing I consider is how much money I have and what type of foods will I be able to buy in the money I have got. I don’t consider the distance.” Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 137 When cash is not available to purchase food, Ntemba owners will provide credit for trusted shoppers, or in lieu of this, respondents also rely on the generosity of neighbors.  “If we have a shortage they usually give us goods on credit because they know that we buy there.”  “Those who know you can lend you especially those who buy from them daily they will lend you when things are hard.”  “For the food what happens is if you relate well with your neighbors they will give you something. They will notice that the children are hungry and they will give them. No one can eat knowing a friend’s child is hungry.” Nutrition Kenya: Awareness and knowledge of nutritious food has not changed significantly since the baseline evaluation. As per the baseline, respondents understood the term ‘nutritious food’ to mean a balanced diet. They believe that by eating a range of different foods the body will be able to access important vitamins, carbohydrates, proteins and minerals and that this will result in good health and high energy. As before there is little clarity on what foodstuffs are in themselves nutritious. When asked what constituted nutrition foods, respondents were as likely to say something like the following:  “Nutritious food is food that gives the body energy and keeps you healthy.”  “Good nutrition is achieved through a balanced diet.”  “Fresh, well cooked food is best for the body.”  “Food that looks and tastes good is nutritious.’ How the food was prepared was also considered important for nutrition, but there was no clarity on what was necessary in the preparation. Foods generally recognized by FGD respondents as nutritious include vegetables (including tubers), milk eggs and meat (including fish and chicken) and carbohydrates (maize meal, tubers and rice). The nutritious foods mentioned by respondents are all readily available at the local markets and shops. Notwithstanding, food purchasing decisions are typically made on the basis of cost and (small) quantity, not on nutritional content.  “Sometimes your economic situation will force you to turn a blind eye to nutritional value. If I have just 100/- in my pocket, I can’t have maize flour, vegetables or anything else … I can just buy two packets of chips for us to eat.”  “Sometimes you don’t have money; it becomes hard to choose what is nutritious since you may not afford it.” Zambia: There is little clarity on what constitutes “nutritious foods.” Respondents understood the term “nutritious food” to mean “food that gives you energy”. They also had the general idea that eating a variety of different foods was healthy. For the most part they felt that their own diets were already nutritious. Overall, however, knowledge as to what actually constitutes nutritious food is vague, and much more so than in Kenya.  “We can’t really tell which foods are responsible for us being healthy because we eat simple foods like vegetable, pumpkins, mangoes and nshima.”  “I don’t really know exactly which foods are nutritious. We just buy what I can manage to buy. If it is vegetables or beans, then we eat.” Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 138  “It (soya chunks) makes you feel like you have had beef, it is a body building food, I don’t know if it’s a protective food.” Most of the knowledge they do have comes from advice given for childcare by clinics and hospitals. Thus nutrition is firmly linked to children. Other (weaker) linkages are “health” and “energy.” In the economic environment in which the respondents live, nutrition is a moot point. They can barely afford to eat regularly and are more concerned with finding foods to fill their stomachs.  “Nutritious foods are there, but finding enough money for you to go and buy food is not easy because the food is expensive.”  “Children at home tell us what to buy and which is nutritious as they learn from school, but finances don’t allow us to buy those foods, the heart really wants to have that food, but the money won’t allow us.” Findings from the Market Actor KIIs Kenya Wholesalers have limited options who they can buy from; generally they are supplied either directly from the producer or through a manufacturer’s appointed distributor. Goods are delivered to the wholesaler for cash on delivery. While credit is not routinely offered to the wholesaler, the seller may offer credit for up to seven days for trusted customers. As for the retailers, their main suppliers are the wholesalers who pay to transport their goods to the retailer’s place of business. A few mentioned buying some milled products direct from the maize miller, while other sources include van deliveries and agents. For retailers, desired qualities of suppliers include (1) reliability and quick delivery of products, (2) competitive prices, (3) able to supply in small quantities (small packages), (4) accepts payment on delivery and (5) able to provide fresh produce. Credit is not usually extended to the retailers, although in certain situations, the wholesaler is willing to make alternative arrangements for trusted customers. Cash flow is very tight for the retailers, and their business model depends on rapid turnover of inventory They are not able to keep large stocks on hand (owing to lack of space and refrigeration), therefore close proximity to their source of supply, and responsiveness by their wholesaler, are important to ensure quick and easy delivery.  "Distance between you and the wholesaler. If I am alone at the shop, I can close for two minutes, rush to get an item and be back before there is any inconvenience.”  ”They are close by, which makes it cheaper for me instead of buying from the company and then incurring costs of transportation.”  ”You consider the customer and the fact you might lose them if you don’t have the goods they need. So if you are busy, you can just make a call and they deliver, but if you are not busy, you can even just go and get the goods yourself.” Retailers believe that their customers value the security of knowing that the product they want will be readily available and that they will be able to make a choice of brands.  “It is important to have different brands, and also different items in stock. That way a customer knows if they come to your shop they will be able to get the different items they need, say for example salt, flour, milk, etc.” Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 139 The same is true for wholesalers as well.  “We sell a wide range of goods. We are reliable. We deliver the goods to our customers and on time. For example if a customer requires the goods within one hour we ensure that within that one hour the goods are at his door.” Because the retailers’ customer base is extremely price sensitive, retailers need to keep their price low. Thus they are very keen to find suppliers who offer savings, not only in terms of cash prices but also in terms of delivery costs.  “I go to them because they are cheaper...let’s say maize flour in Kibera will be going for 1,050, but in Eastleigh it will be 1,020. The same goes for wheat flour, here it may be going for 1,290, but I will get it there for 1,240.” The willingness of wholesalers to break down packages into smaller quantities is an expectation, and these retailers would not do business with any wholesaler who did not offer this service. Wholesalers are also expected to offer a wide range of products and brands.  “It may be difficult for me to buy a whole sack of rice; however, if they can sell to me half or quarter, I will be able to sell and go for another one.”  “All wholesalers want to sell, they are also in competition with each other, so you find that they all are able to be flexible in their packaging”  “I want to buy from a supplier who has the different brands. It is important because different customers have different preferences. So it is important to have the different brands.” The retailers claim that their customers are mainly lower class with limited spending power. Given their limited income, the retailers’ customers are looking for value for money. This need is met primarily through low prices and small pack sizes.  “I would say they are in the lower bracket based on the amount of goods that they purchase. For instance, they will buy cooking oil for as little as ten shillings. It is very hard to get someone who wants to buy say two packets of flour, or generally do a big shopping.”  “I normally pack. It comes from a sack of flour that I have which I call ‘local.’ This is the one I normally measure from for customers who have as little as even ten shillings. So you have to consider all the customers.”  “There are products such as rice, sugar and maize flour that you can repackage into smaller quantities.” Not only do the retailers repackage their products to suit the purchasing needs and capacity of their customers, so do the wholesalers.  “We re-package commodities into smaller quantities that they can afford. For example, bar soaps some of the customers cannot afford to buy the entire box. So we re-package them into small batch of four pieces. Nescafe they cannot buy entire 7 dozens, so we sell to them one dozen, snacks such as bites sold in a bail, but we re-package to a pack of eight sold at Kes 80 each. Cigarettes sold in a pack of 10. i.e. for Sportsman the entire pack is sold at Kes 1,190; however, for our retailers we sell to them one packet at Kes 120.” Retailers offer limited credit to their regular customers. This is an informal arrangement and is based entirely on trust. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 140  “I give to specific customers who are regular, with limit and the duration is one day. For example, if I know you take bread every day and today you don’t have money, I can let the customer take just a loaf of bread, but nothing else,”  “It depends on the relationship with the customer. For example if they are regular, coming by every morning, lunch and evening, if they don’t have money say in the evening, you can offer them credit. But not non-regular customers, those ones might disappear and never pay back.” Likewise, wholesalers will sometimes work out more flexible payment terms for their retailer customers, but this is less than norm than it is for retailers to provide credit to their customers.  “Yes if we supply the goods today we can collect the cash after three days. Only to those whom we have developed a relationship for longer period.” According to both retailers and wholesalers, customers pay little heed to the nutritional value of products. Rather, other factors drive purchasing decisions and include such things as cost (primarily), packaging, reliability, convenience, speed, personal relationships, and product variety. Zambia As in Kenya, the wholesale and retail sector is a cash-only sector. Payments are made on delivery in cash. In contrast to Kenyan market actors interviewed for the endline operating in urban settlement areas around Nairobi, Zambia market actors are operating in more remote rural areas and thus their options for sourcing goods are more limited. Sourcing from Lusaka is generally not an option due to the transport costs involved. In this less competitive environment, the services offered to retail and wholesale buyers tend not to be as extensive as in Kenya. For example, in Zambia transport is the responsibility of the buyer, not the seller. Retailers claim that their customers are mainly lower class with limited spending power. Customers are mostly women and children who come in daily for some items and monthly for other items.  “It’s just the people we deal with, it’s low class.”  “They are people from poor background.” Retailers try to offer customers as wide a range of products as possible. Although most of the wholesalers do not describe themselves as retailers, they also sell to the public; however, the range of products they offer is not as wide. As in Kenya, shoppers are looking for value for money, which both retailers and wholesalers provide them through low prices and small pack sizes.  “They get attracted the reason why you find that they do not have money to buy the whole pack which is 11 Kwacha for 1 kg so that 11 kwacha they don’t have, only 1 kwacha. So we will make sure that we will budget according to their budget” (Retailer)  “I buy cases of sugar, and when I come here, I repack them so instead of buying the pack for K20, they buy for K10 or K5.” (Wholesaler) Both retailers and wholesalers strive to build a relationship with their customers and believe that their customers value the security of knowing that the product they want will be readily available when they want them. How they relate to their customers is also important.  “When they tell me to say tomorrow we will come and buy such items and if I don’t have them in stock I make sure I go for orders very early before they come.” (Retailer) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 141  Trust is there because of the way we relate and because they find the commodities that they need. The way we attend to them that is why most of them buy from us.” (Wholesaler) Retailers offer limited credit to regular customers. As in Kenya, this is an informal arrangement that is purely trust based. “We do give credit to customers who usually buy from us, and they say I have a shortfall of maybe 1 kwacha for cooking oil let me go into town I will come and pay back - so we will give to that person.” Wholesalers generally do not provide credit to retailers, nor do they, as a rule, break down packages into smaller sizes, although there are exceptions for trusted customers or, in the latter case, for small retailers. Wholesalers selling to the retail market will offer their retail customers credit to their regular customers—it is hard to do business in the retail market without offering credit.  “What makes us give them products on credit is that some always buy from us and they are loyal. You will find that someone always buys cash but this time around they have a shortage, and they explain their situation to you. With such cases we usually give on credit.” Finally, there appears to be very little awareness among retailers and wholesalers as to which products are nutritious or the need for nutritious products. Neither go out of their way to provide nutritious food to their customers. In their product selection, they are guided solely by what they believe will sell best.  “We buy according to what the customers want - you find that people would ask for something then we go and order that.” (Retailer)  “We sell especially what people want because if you consider the nutritional value and people don’t want them, no one will buy from you, and you end up packing them until they expire then you just throw them.” (Wholesaler) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 142 ANNEX 8: HOUSEHOLD SURVEY FINDINGS, KENYA Table X. Distribution of Household Survey Respondents in Kenya Achievement Start End March March 101 Juja 21 20 20 21 7 3 102 Kalimoni 21 21 21 21 4 2 201 Makongeni 21 19 20 21 12 2 202 Kianduli 14 19 20 14 6 2 203 Waitethia 15 19 20 15 4 2 301 Kwa Kairu 14 21 22 14 3 2 302 Kimbo 14 20 21 14 7 2 303 Toll 14 19 22 14 2 1 401 Katwekera 8 19 21 8 5 1 402 Lain Saba 9 19 23 9 3 1 403 No. 8 10 21 21 10 5 1 404 Oympic 10 20 20 10 7 1 405 Karanja 8 19 22 8 3 1 501 Kwa Chief 11 19 20 11 6 1 502 Matopeni, 10 21 21 10 2 2 503 Soweto, 10 20 20 10 9 1 504 Nyando 12 19 20 12 8 1 6 Kitengela 900 Kitengela 45 20 22 45 23 6 601 Pipeline 22 21 22 22 12 2 602 Juakali 23 22 23 23 12 3 701 Quarry Slums 14 19 20 14 6 2 702 Phase III 14 22 22 14 9 1 703 Gituamba 14 20 21 14 8 1 801 Jonsaga 14 19 20 14 7 2 802 Kwa Chief 14 19 22 14 9 2 803 Kiamaiko 14 20 22 14 4 2 10 Athi River 1000 Athi River 42 19 21 42 20 5 438 438 203 52 Telephonic Verification Region Sample Points Dates Covered Achievement Number Name Quality Control TOTAL Urban Accompanied 1 Juja 2 Thika 3 Ruiru 4 Kibera 9 Huruma 5 Kayole 7 Tassia 8 Mwiki Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 143 Household Survey Detailed Results Food Security Status Baseline Endline Total Male Female Total Male Female Base (n): 428 151 277 438 188 250 % % % % % % LACK OF RESOURCES TO GET FOOD 24 19 27 53 55 52 Base: 102 28 74 243 103 131 RARELY (1-2 TIMES) 46 57 42 42 45 40 SOMETIMES (3-10 TIMES) 52 43 55 44 46 42 OFTEN (MORE THAN 10 TIMES) 2 0 3 10 18 7 DID YOU GO TO SLEEP AT NIGHT HUNGRY BECAUSE OF LACK OF FOOD? 17 14 19 23 22 23 Base: 74 21 53 99 41 58 RARELY (1-2 TIMES) 57 57 57 61 63 59 SOMETIMES (3-10 TIMES) 42 38 43 35 32 38 OFTEN (MORE THAN 10 TIMES) 1 5 0 4 5 3 DID YOU GO A WHOLE DAY AND NIGHT WITHOUT FOOD? 9 7 10 14 14 13 Base: 40 11 29 60 27 33 RARELY (1-2 TIMES) 63 73 59 67 67 67 SOMETIMES (3-10 TIMES) 38 27 41 32 33 30 OFTEN (MORE THAN 10 TIMES) 0 0 0 2 0 3 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 144 Foods Consumed Yesterday Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 145 Foods Consumed Yesterday (by Gender and Food Security Status) Total Male Female Food Insecure Food Secure Total Male Female Food Insecure Food Secure Baseline Endline Base: 428 151 277 102 326 438 188 250 234 204 % % % % % % % % % % CEREALS 96 97 96 93 97 97 96 97 96 98 OIL AND FATS 88 89 87 82 89 94 93 94 93 95 MILK AND MILK PRODUCTS 75 74 77 82 73 88 87 89 85 92 DARK GREEN LEAFY VEGETABLES 82 84 81 80 83 84 84 84 83 84 OTHER VEGETABLES 85 83 86 75 88 82 81 83 84 81 SPICES, CONDIMENTS, BEVERAGES 72 77 69 52 78 74 66 79 78 74 SWEETS 78 80 77 69 81 68 64 72 68 68 VITAMIN A RICH FRUITS 42 47 39 28 46 53 53 52 39 53 VITAMIN A RICH VEGETABLES AND TUBERS 46 51 43 31 50 52 46 57 45 52 LEGUMES, NUTS AND SEEDS 34 37 32 18 39 42 43 42 42 42 WHITE TUBERS AND ROOTS 31 28 32 18 35 39 35 43 37 39 EGGS 29 27 29 24 30 38 37 38 32 38 FLESH MEATS 28 33 25 21 30 37 38 36 31 37 ORGAN MEAT (IRON RICH) 15 21 12 10 17 18 18 18 17 18 OTHER FRUITS 11 11 12 10 12 14 14 14 16 14 FISH 8 10 8 9 8 5 5 5 1 5 INSECTS 0 0 1 2 0 3 3 2 3 3 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 146 Familiarity with Nutritious Foods (Base: 2016: N=438 / 2015: N=428) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 147 Importance of Outlet Types (Base: N=438) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 148 Outlets by Frequency and Expenditures (Base: N=438) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 149 Outlets and Food Items Bought Kiosks Super Markets Open Markets Butchers Mini Markets Neighborhood Shops Convenience Stores Base 349 235 180 68 57 48 12 % % % % % % % Milk and milk products 72 77 15 1 63 63 67 Cereals 54 89 22 1 81 77 83 Oils and fats 50 81 12 1 75 29 58 Eggs 50 41 23 1 42 19 50 Spices, condiments, beverages 40 49 11 1 56 27 75 Dark green leafy vegetables 38 12 86 1 18 13 17 Vitamin A rich vegetables and tubers 24 13 48 1 9 2 25 Vitamin A rich fruits 17 8 51 1 12 8 58 Sweets 16 29 3 0 21 6 33 Other vegetables 16 4 49 0 19 6 17 Other fruits 10 3 28 0 12 2 8 Legumes, nuts and seeds 10 24 31 0 28 6 25 Flesh Meats 9 11 9 93 9 4 17 Fish 5 6 9 0 11 2 42 White tubers and roots 3 3 33 0 5 2 17 Organ meats 1 8 1 24 7 4 8 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 150 Outlet Association (Base: N=438) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 151 Supermarkets vs. Kiosks – NET Scores (N=438) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 152 Store-Bought vs. Home-Grown Food (By demographics) Total Male Female Food Insecure Food Secure Base 438 188 250 234 204 ACCESS TO AREA FOR GROWING FOOD Yes, close to where I live 3 3 4 4 3 Yes, up-country 12 12 12 14 10 No 84 85 84 82 87 PROPORTION STORE-BOUGHT VS. HOME-GROWN FOOD Proportion Bought In Shops 77 77 78 76 79 Proportion Home-Grown 23 23 22 24 21 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 153 Ratings of Purchase Decision Factors (By Gender, Food Security Status and Region) Total Male Female Food Insecure Food Secure Total Male Female Food Insecure Food Secure B B B B B E E E E E Base: 438 188 250 234 204 Cost 6.13 5.87 6.26 6.59 5.98 6.38 6.47 6.30 6.69 6.02 ** ** Quality 5.61 5.38 5.73 5.06 5.78 6.11 6.16 6.06 6.11 6.10 Availability 5.43 5.30 5.49 5.98 5.25 5.97 6.15 5.83 5.91 6.03 ** ** Taste 5.41 5.45 5.39 5.44 5.40 5.78 5.80 5.76 5.86 5.68 Ease of Preparation 5.37 5.36 5.37 5.81 5.23 5.75 5.82 5.69 5.67 5.83 Nutrition Value 5.40 5.36 5.43 5.73 5.30 5.58 5.59 5.58 5.32 5.88 ** ** Familiarity/Tradition 5.09 5.23 5.01 5.10 5.08 5.38 5.37 5.38 5.12 5.67 ** ** Packaging 4.70 4.66 4.73 3.75 5.00 5.13 5.15 5.12 4.85 5.46 ** ** Brand Name 4.67 4.59 4.71 3.99 4.88 4.94 5.00 4.89 4.63 5.28 ** ** Requested by Adult 3.98 4.03 3.96 3.29 4.20 4.76 4.58 4.89 4.39 5.18 ** ** Requested by Children 4.44 4.55 4.38 4.30 4.48 4.53 4.37 4.65 4.09 5.05 ** ** Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 154 Importance of Purchase Decision Factors - Trend (Mean Scores) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 155 Brand Selection Brand KPIs (by Gender and Food Security Status) Total Male Female Food Insecure Food Secure Total Male Female Food Insecure Food Secure B B B B B E E E E E Base: 428 151 277 102 326 438 188 250 234 204 % % % % % % % % % % Awareness DAIMA MILK 96 97 95 98 95 92 88 96 93 91 SOKO UJI 66 62 69 73 64 42 35 48 36 49 WIMBI 68 60 72 73 66 35 27 41 35 34 KCC MY CHOICE 44 47 42 24 50 28 28 29 27 29 CLASSIC FRESH MILK 13 14 13 12 13 16 9 20 12 19 CAPITAL FLOUR 2 2 2 1 2 4 1 6 5 2 CLASSIC PORRIDGE 2 3 2 3 2 3 2 3 3 2 Trial DAIMA MILK 84 83 84 91 82 77 70 82 74 81 SOKO UJI 40 34 43 35 41 20 16 22 14 26 WIMBI 46 32 54 44 47 16 13 19 13 20 KCC MY CHOICE 27 26 27 7 33 11 11 12 8 15 CLASSIC FRESH MILK 4 5 4 2 5 4 4 3 1 7 CLASSIC PORRIDGE 0 1 0 1 0 0 0 0 0 0 CAPITAL FLOUR 1 1 1 1 1 0 0 0 0 0 Usage DAIMA MILK 67 62 70 85 62 58 49 64 57 59 SOKO UJI 19 11 23 18 20 9 6 11 6 13 WIMBI 25 16 30 26 24 7 5 8 7 7 KCC MY CHOICE 13 15 12 4 16 6 6 6 4 8 CLASSIC FRESH MILK 2 2 2 0 3 1 1 1 0 1 CLASSIC PORRIDGE 0 0 0 0 0 0 0 0 0 0 CAPITAL FLOUR 0 1 0 0 1 0 0 0 0 0 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 156 Brand Strengths - Trend Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 157 Brand Ratings Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 158 Demographics 1/4 Demographics 2/4 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 159 Demographics 3/4 Demographics 4/4 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 160 Kenya Case Study Mapping-General View Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 161 ANNEX 9: HOUSEHOLD SURVEY FINDINGS, ZAMBIA Table X. Distribution of Household Survey Respondents in Zambia Achievements Start End March March 101 Magazine Market, Magazine Compound 20 21 21 2 2 102 Mayamiko Mill Magazine 20 22 22 2 3 103 Bethel Primary 22 21 24 3 3 104 Muchini Markets Muchini A 20 23 23 2 3 105 Muchini Market B 16 23 24 2 4 106 Mayamiko School Magazine 19 22 25 3 3 107 Mayaniko Shop Muchini 19 22 25 2 1 108 Edimberg School Muchini 18 24 24 2 3 109 Soweto 12 21 22 3 3 110 Magazine Market 12 21 21 3 0 111 Bethel Corner Shop in Magazine 12 24 24 3 1 112 Muslim Association School 11 23 23 3 2 113 Red Grocery n Salon Muchini 11 23 23 3 1 114 New Apostolic Faith Mission Church Magazine 16 21 22 4 2 115 United Apostolic Faith Church Magazine 14 21 24 4 1 201 Roman Catholic Chibolya 19 21 22 4 2 202 Green Shop Chibolya 18 21 24 4 1 203 Community Borehole 19 21 23 4 1 204 Green Bakery Chibolye 18 21 23 3 1 205 Mutunzi School Chibolya 19 22 23 4 7 206 PGS Shop Chibolya 18 22 25 4 5 207 Katete Community Borehole 18 23 23 4 3 208 Katete Market 18 23 25 4 1 209 Mutunzi School 19 22 23 3 0 210 Blue Roof Chinese Warehouse Chibolye 18 24 24 3 3 211 Bethel School Chibolya A 18 22 24 3 0 212 Bethel School Chibolya B 20 23 24 2 0 464 83 56 CHIPATA KATETE Urban Accompanied TOTAL Telephonic Verification Region Sample Points Dates Covered Quality Control Number Name Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 162 Household Survey Detailed Results Food Security Status Baseline Endline Total Male Female Total Male Female Base: 657 116 541 ?? ?? ?? % % % % % % LACK OF RESOURCES TO GET FOOD 30 27 31 46 44 46 Base: 200 31 169 212 45 167 RARELY (1-2 TIMES) 50 45 51 46 56 43 SOMETIMES (3-10 TIMES) 36 42 34 43 36 45 OFTEN (MORE THAN 10 TIMES) 15 13 15 11 9 12 DID YOU GO TO SLEEP AT NIGHT HUNGRY BECAUSE OF LACK OF FOOD? 19 18 20 40 30 42 Base: 127 21 106 184 31 153 RARELY (1-2 TIMES) 46 43 51 57 65 56 SOMETIMES (3-10 TIMES) 39 38 32 38 32 39 OFTEN (MORE THAN 10 TIMES) 15 19 17 5 3 5 DID YOU GO A WHOLE DAY AND NIGHT WITHOUT FOOD? 13 15 12 33 28 34 Base: 83 17 66 152 29 123 RARELY (1-2 TIMES) 45 29 48 57 66 55 SOMETIMES (3-10 TIMES) 40 53 36 35 28 37 OFTEN (MORE THAN 10 TIMES) 16 18 20 8 7 8 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 163 Food Security Status – Trend (Total by Sample & by Region) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 164 Foods Consumed Yesterday Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 165 Food Items Consumed Yesterday (Trend – by location) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 166 Food Items Consumed Yesterday (by demographics) Total Male Female Food Insecure Food Secure Total Male Female Food Insecure Food Secure B B B B B E E E E E Base: 46 4 10 3 36 1 21 2 25 2 657 116 541 200 457 % % % % % % % % % % CEREALS 96 95 96 95 97 98 100 98 98 99 OIL AND FATS 75 80 73 67 81 90 90 90 87 91 DARK GREEN LEAFY VEGETABLES 69 70 69 72 67 85 85 86 83 86 OTHER VEGETABLES 83 80 84 75 90 79 79 80 76 81 SWEETS 65 74 63 46 81 73 73 73 63 77 SPICES, CONDIMENTS, BEVERAGES 78 85 76 78 78 72 76 71 72 72 VITAMIN A RICH VEGETABLES AND TUBERS 50 59 48 50 50 43 46 43 38 46 LEGUMES, NUTS AND SEEDS 31 36 29 25 35 36 39 36 31 39 MILK AND MILK PRODUCTS 30 36 29 17 41 36 48 34 24 42 FLESH MEATS 37 45 34 26 46 36 41 35 32 38 EGGS 31 42 28 25 37 27 41 25 22 30 FISH 17 14 19 15 19 21 22 21 18 22 WHITE TUBERS AND ROOTS 24 30 23 19 29 19 22 19 15 21 OTHER FRUITS 14 16 13 12 15 12 16 11 12 12 VITAMIN A RICH FRUITS 18 16 19 12 23 12 18 10 13 11 ORGAN MEAT (IRON RICH) 14 13 15 14 14 11 21 9 9 12 INSECTS 13 11 14 12 14 8 10 8 5 10 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 167 Familiarity with Nutritious Foods (Base: 2016: N=464 / 2015: N=657) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 168 Importance of Outlet Types (Total Sample; N=464) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 169 Outlets by Frequency and Expenditures (Base: Total Sample; N=464) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 170 Outlets & Food Items Bought Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 171 Outlet Association (Base: Total Sample; N=464) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 172 Open Markets vs. Kiosks – NET Scores (Total Sample; N=464) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 173 Store-Bought vs. Home-Grown Food (by demographics) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 174 Ratings of Purchase Decision Factors (by Gender, Food Security Status and Region) Total Male Female Food Insecure Food Secure Total Male Female Food Insecure Food Secure B B B B B E E E E E Base: 637 116 541 200 457 464 103 361 212 252 Cost 6.01 6.13 5.98 5.98 6.02 6.52 6.52 6.52 6.69 6.37 ** ** Quality 5.86 5.72 5.89 5.46 6.03 5.50 5.43 5.52 5.41 5.58 Availability 5.00 4.97 5.01 4.54 5.20 5.55 5.69 5.51 5.70 5.42 * * Taste 5.59 5.62 5.59 5.22 5.76 5.90 6.07 5.85 5.92 5.89 Ease of Preparation 4.04 4.46 3.95 4.35 3.91 5.44 5.52 5.42 5.21 5.63 ** ** Nutrition Value 5.34 5.60 5.29 5.40 5.32 5.58 5.59 5.58 5.32 5.88 ** ** Familiarity/Tradition 4.84 4.68 4.87 4.66 4.92 4.54 4.51 4.55 4.66 4.45 Packaging 5.07 4.84 5.12 4.61 5.27 4.49 5.15 4.29 4.21 4.71 ** ** * * Brand Name 4.52 4.63 4.50 4.00 4.75 4.33 5.16 4.08 3.96 4.62 ** ** ** ** Requested by Adult 5.71 5.50 5.75 5.75 6.69 5.23 4.85 5.34 5.23 5.22 * * Requested by Children 5.37 4.72 5.51 5.43 5.34 4.35 3.97 4.46 4.27 4.40 * * Significant at * = 90% ** = 95% *** = 99% Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 175 Brand Selection Brand KPIs (by Gender and Food Security Status) Total Male Female Food Insecure Food Secure Total Male Female Food Insecure Food Secure B B B B B E E E E E Base: 657 116 541 200 457 464 103 361 212 252 % % % % % % % % % % Awareness Seba Foods: Corn-Soy Blend 79 78 79 80 79 93 94 93 95 92 It's Wild: Peanut Butter 46 45 47 49 45 69 62 71 66 72 It's Wild: Yummy Soy 18 22 17 20 17 39 40 39 38 40 Freshpikt Peanut Butter 9 15 8 11 9 36 30 37 36 35 Jungle Beats Peanut Butter 7 13 6 9 6 7 7 7 5 8 Chankwakwa Corn-Soy Blend 4 8 3 7 3 4 5 4 4 5 Trial Seba Foods: Corn-Soy Blend 74 71 75 75 74 88 91 87 90 87 It's Wild: Peanut Butter 35 34 35 31 37 48 40 50 40 55 It's Wild: Yummy Soy 11 10 11 8 12 18 21 17 17 20 Freshpikt Peanut Butter 5 5 5 4 6 8 8 8 7 10 Jungle Beats Peanut Butter 3 4 3 3 4 2 1 2 1 3 Chankwakwa Corn-Soy Blend 1 4 0 1 1 1 0 1 1 1 Usage Seba Foods: Corn-Soy Blend 69 65 70 66 70 76 80 75 81 72 It's Wild: Peanut Butter 23 16 24 13 27 27 18 30 17 35 It's Wild: Yummy Soy 6 4 6 3 7 9 13 7 8 9 Freshpikt Peanut Butter 2 3 2 2 2 3 4 2 2 4 Jungle Beats Peanut Butter 1 1 1 2 1 - - - - - Chankwakwa Corn-Soy Blend 0 2 0 0 0 - - - - - Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 176 Brand Strengths - Trend Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 177 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 178 Brand Ratings (Base: Those aware of the brand; by Gender, Food Security Status and Region) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 179 Brand Ratings – Trend (Base: Those aware of the brand) 2016 2015 2016 2015 Base: 153 89 167 108 Quality 3.18 3.23 3.55 3.28 Cost 3.30 2.88 3.08 2.59 2016 2015 2016 2015 Base: 65 32 115 46 Quality 2.66 2.69 3.21 2.95 Cost 2.32 2.23 2.11 2.24 CHIPATA KATETE IT'S WILD: PEANUT BUTTER IT'S WILD: YUMMY SOY CHIPATA KATETE Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 180 Brand Ratings (Base: Those having tried the brand; by Gender, Food Security Status and Region) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 181 Brand Ratings – Trend (Base: Those having tried the brand) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 182 Sources of Brand Awareness (Base: Users of the brand: by Gender, Food Security Status and Region) Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 183 Sources of Brand Awareness – Trend • Since the previous study, word of mouth has become an even more important source of awareness for both brands. • While in Chipata, radio increased its importance, it declined in Katete; however, TV and billboards made more of an impact in Katete. • Road shows made no impact in Chipata, they recorded modest gains in Katete. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 184 Frequency of Purchase (Base: Users of the brand; by Gender, Food Security Status and Region) • While 84% of users buy It’s Wild: Peanut Butter at least once a month, only 73% of food-insecure do so. • Users in Katete purchase the product more frequently than those in Chipata. • Males purchase It’s Wild: Yummy Soy is purchased more frequently than females. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 185 Frequency of Purchase - Trend • Compared against the previous wave, purchase frequency for It’s Wild: Peanut Butter has increased: – In 2015, 72% purchased at least once a month. – In 2016, 84% purchased at least once a month. • At a slightly lower level, the same is true for It’s Wild: Yummy Soy: – The proportion buying at least once a month rose from 70% to 78%. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 186 Reasons for not Buying Frequently (Base: Those buying once a month or less often) Total Male Female Food Insecure Food Secure CHIPATA KATETE Base: 76 12 64 29 47 26 50 Too expensive 78 75 78 76 79 85 74 It's cheap 12 17 11 14 11 0 18 Not sold in the market 11 17 9 3 15 8 12 Too hard to find 9 17 8 7 11 4 12 Do not like taste 3 8 2 3 2 4 2 not packaged appropriately 3 0 3 3 2 4 2 13 use of traditional one 3 0 3 0 4 0 4 other adults in the family do not like it or do not want me to buy it 1 0 2 0 2 4 0 It is not a food for people like me 1 0 2 3 0 0 2 Total Male Female Food Insecure Food Secure CHIPATA KATETE Base: 22 8 14 7 15 4 18 Not sold in the market 27 38 21 29 27 25 28 Too hard to find 27 38 21 29 27 50 22 It's cheap 27 25 29 14 33 0 33 Too expensive 18 13 21 29 13 0 22 Do not like taste 9 0 14 0 13 25 6 too hard to prepare 5 13 0 0 7 0 6 Children in the family do not like it or do not want me to buy it 5 0 7 0 7 0 6 IT'S WILD: PEANUT BUTTER IT'S WILD: YUMMY SOY Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 187 Pack Sizes Bought (Base: Users of the brand; by Gender, Food Security Status and Region) • For both brands, the 1kg pack is the most popular pack size. Compared against 2015, smaller pack sizes have diminished in importance. • The average price for a 1kg pack of It’s Wild: Peanut Butter is ZMK 20.86. For It’s Wild: Yummy Soy, it is ZMK 13.52 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 188 Satisfaction with Brands (Base: Users of the brand: by Gender, Food Security Status and Region) • It’s Wild: Peanut Butter: The difference between 2015 and 2016 totals is not significant. In Katete, however, the improvement is significant at 99%. • It’s Wild: Yummy Soy: The difference in totals is significant at 99%. By location, however, they are not statistically significant. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 189 Extent of Liking of the Brands (Base: Users of the brand: by Gender, Food Security Status and Region) • Both brands have improved their ratings since 2015, especially It’s Wild: Yummy Soy, which now reaches scores approaching those of It’s Wild: Peanut Butter. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 190 Recommendation & Future Purchase (Base: Users of the brand: by Gender, Food Security Status and Region) • Both brands have improved on willingness to recommend and future purchase intention. Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 191 Demographics 1/4 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 192 Demographics 2/4 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 193 Demographics 3/4 Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 194 Demographics 4/4 • In the current wave, slightly more male respondents (22% vs. 18%) completed the survey; however, the gender ratio for head of household remained unchanged with 77% males vs. 23% females. • Respondents’ mean age increased from 34.02 to 35.05 in the current study. • There was a marginal increase in the average number of household members: 5.2 in 2015 to 5.5 in 2016. Total Male Female Food Insecure Food Secure CHIPATA KATETE Base: 464 103 361 212 252 242 222 Yes 25 31 23 15 33 26 23 No 75 69 77 85 67 74 77 Charcoal 85 87 85 82 88 84 87 Firewood 12 5 14 17 8 14 9 Electricity 3 8 2 1 4 2 4 Cooking Fuel Household Connected to Electric Grid Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 195 Zambia Case Study Mapping – General View Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 196 ANNEX 10: PFS TRAINING CURRICULUM Volunteers SGB Leads Country Leads Field Partners Welcome to PFS The Client Lead Role The Country Lead Role Agenda Welcome Agenda Agenda PFS Vision and Mission Agenda Volunteer Roles Volunteer Roles PFS Values Introduction to PFS Project work process (process maps) Project work process (process maps) Corporate Partners PFS Vision and Mission Working with Field Partners Working with Field Partners Project work process (process maps) PFS Values Calculating volunteer hours Calculating volunteer hours Working with PFS Volunteers  Volunteer Roles  Volunteer Org Chart Corporate Partners What to expect as a volunteer What to expect as a volunteer -What does a project volunteer do? Key Field Partners How to engage your team How to engage with Sr. Consultants How to engage with Clients  Business Analysis  Charters The PFS Model Foundational Meeting How to engage your team -Business Analysis Volunteer Roles Alignment Meeting Foundational Meeting -Charters Volunteer Org Chart Project kick off Meeting Alignment Meeting How to engage with Sr. Consultants How we Engage with Clients Graduation Process Project kick off Meeting How to find volunteers for the client projects Client Engagement Process More on Global Teams Graduation Process How to engage your team Field Partner's Role Agenda Review Foundational Meeting How Project Engagement Works Cross-Cultural Understanding Alignment Meeting What does a project volunteer do?  Project Kick Off Call  Working with the Field Partners  Working with the Client Cross-Cultural Communications Project kick off Meeting PFS Impact - Success Stories Building the Multicultural Team Graduation Process Time Commitment GlobeSmart BlueJeans Corporate Partner Company Support Review Other Technology Next Steps Review BlueJeans Basics Agenda What is BlueJeans How to attend a meeting Where to find help Cultural Exposure Agenda Cultural Context What life is like in… Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 197 SGB's Examples Review Global Teams Agenda Cross-Cultural Understanding Cross-Cultural Communications Building the Multicultural Team GlobeSmart Review Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 198 ANNEX 11: TERMS OF REFERENCE (TOR) C.2 SCOPE OF WORK 1) Objectives The TAP BFS-PE TO will support four PEs of BFS centrally-funded/centrally-managed projects or activities. 2) Proposed Activities To fulfill Evaluation Policy and ADS 203 requirements, the TAP BFS-PE TO Contractor shall, in coordination with BFS, manage, design, and implement four PEs of BFS Feed the Future centrally funded and centrally-managed activities. The TAP BFS-PE TO Contractor shall utilize BFS guidance and templates for the management, design, and implementation of PEs. Given the range and diverse nature of the activities implemented by the targeted BFS mechanisms, expertise across a range of research, M&E and capacity-strengthening food security-related areas will be required, including organizational and staff/consultant expertise in the use of both quantitative and qualitative methodologies in conducting PEs. The BFS centrally-managed mechanisms to be evaluated are listed below. The parameters of each PE are contained in Attachment J.2. Activity Final Performance Evaluation African Cocoa Initiative: The project develops the cocoa sector by fostering public-private cooperative investments in cocoa, improving the genetic quality and productivity of the cocoa, expanding farmer education and training programs, and improving cocoa input supply chains. It works in Cameroon, Côte d’Ivoire, Ghana, Nigeria, and Togo.(http://worldcocoafoundation.org/wcf-african-cocoa-initiative/) RESAKSS Analysis and Capacity Building: This is a consortium of CGIAR and regional organizations being led by IFPRI to deepen and broaden available food policy information and analysis, as well as networks of food security and policy researchers and practitioners both in country and regionally. (http://www.resakss.org/) Solutions for African Food Enterprises (SAFE): SAFE provides customized technical assistance to select food processors with the aim of increasing the competitiveness of the African food processing sector and to expand availability of affordable and nutritious foods. SAFE is carried out through an alliance between Partners in Food Solutions (PFS) and TechnoServe. PFS is a non-profit organization formed by General Mills and includes technical expertise from General Mills, Cargill, Royal DMS, and most recently Buhler. PFS provides remote-based, demand driven customized technical assistance to selected food processors which is carried out by TechnoServe, as well as broader trainings on key industry knowledge gaps delivered to larger sector wide groups through seminars. TechnoServe provides the on-the-ground food technology specialists and business advisors to facilitate the on-site customized technical assistance recommended by PFS. SAFE builds on the structures put in place by a two-year predecessor program, the African Alliance for Improved Food Processing (AAIFP) which ended November 30, 2012. SAFE operates in Kenya, Malawi, and Zambia (http://www.partnersinfoodsolutions.com/) Activity Midterm Performance Evaluation Solutions for African Food Enterprises (SAFE) Final Evaluation Report | May–June 2016 199 FTF Partnering for Innovation: This cooperative agreement is focused on finding and commercializing agricultural technologies that have the potential to impact smallholders and supporting private sector engagement within Feed the Future. The program works to engage the private sector to commercialize technologies to the smallholder farmer market and builds and procures public-private partnerships that support Feed the Future efforts.(www.partneringforinnovation.org) C.3 TASKS For each Performance Evaluation, the Contractor shall: Design, plan, and implement independent rigorous PEs of BFS centrally managed activities to determine project or activity performance and/or impacts, focusing on progress, successes, obstacles, constraints, and areas of need and for improvement. Conduct data collection and analysis, and produce reports and briefings on PE findings that follow USAID guidance and format. Contribute to building the capacities of host country institutions (government, NGO, and private sector) to conduct PEs by including local institutions and individuals in all phases of evaluation design and implementation, to the maximum extent feasible. In addition to the requirements listed above, the Contractor shall: 1. Confer with USAID and with the Implementing Partner to create the statement of work for the PE, including identifying the PE questions and evaluation methods to be used (within two weeks of PE start date) 2. Identify the evaluation team members and obtain BFS approval (within three weeks of PE start date) 3. Develop the PE Plan and obtain BFS approval (within three weeks of PE team’s start date) 4. Conduct desk review of required documents (timing of remaining steps determined in PE Plan) 5. Compile performance reporting data 6. Conduct stakeholder, key informant and/or focus group interviews in person or via phone or email 7. Conduct country visits to identified country(ies), as applicable 8. Conduct field visits while in country, as applicable 9. Provide exit briefing(s) to USAID mission(s) 10. Produce up to two versions of draft PE report, incorporating USAID feedback 11. Provide in-person or telephone briefing on the draft report to BFS and to the implementing partner 12. Produce the final PE report 13. Present the final PE report to USAID and partners in Washington DC 14. Provide data required for Open Data posting.