FY 2014 Food for Progress Millet Business Services Project (MBSP) Mid-Term Evaluation August 2018 2 FY 2014 Food for Progress Millet Business Services Project (MBSP) in Senegal Mid-Term Evaluation Program: Food for Progress Agreement Number: FCC-685-2014/055-00 Funding Year: Fiscal Year 2014 Implemented by: NCBA CLUSA Evaluation Authored by: Jackie Yiptong Avila DISCLAIMER: This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Accessibility Note: An accessible version of this document can be made available by contacting fas.monitoring.evaluation@usda.gov Foreword It has been our pleasure to perform the mid-term evaluation of the USDA Millet Business Services Project (MBSP) that is being implemented by NCBA CLUSA. The project is important to rural farmers in Senegal since this remarkable crop is very drought resistant, a vital quality in subsistence rural communities where soil fertility is poor and rainfall unpredictable. Beyond its importance for agronomic reasons, millet is nutritious and has deep cultural roots in Senegal. Millet is consumed during major family and religious celebrations by all ethnic and religious groups: it is served when a bride visits her new husband’s parents for the first time, or during a breakfast for the naming ceremony of a newborn. We are pleased to have fielded Jackie Yiptong Avila as our team leader. Jackie is a veteran project evaluator, is familiar with West Africa, and speaks fluent French and English. We would also like to recognize our partnership with Le Centre de Recherche pour le Développement Humain (CRDH), a local firm specialized in conducting household surveys, as well as the talented professionals who assisted with this project, especially given the constraints of time, budget, logistics, and the extensive work carried out during Ramadan. The evaluation team deserves special commendation for frequently working extremely long hours and up to 7 days a week to ensure that a quality report could be delivered under very tight time constraints. We would like to thank our NCBA CLUSA colleagues for their cooperation both from their headquarters in Washington, DC as well as their country office in Kaolack. We hope that this mid-term evaluation will help the project management team to learn lessons and adjust program activities as needed to implement the second term of this project successfully. Best Regards, Kevin X. Murphy, President and CEO J.E. Austin Associates, Inc. 1621 North Kent Street Suite 601 Arlington, VA 22209 USA 2 Table of Contents Foreword 1 Acronyms 4 Executive Summary 6 1. Background 11 2. Evaluation Purpose 12 3. Evaluation Design and Methodology 12 3.1 Qualitative Research 13 3.2 Survey Methodology 14 3.3 Ethical Responsibilities - Risks and Benefits 17 3.4 Limitations of the Study 17 4. Program Activities and Partners 18 5. Data Analysis - Findings and Recommendations 25 6. Profile of the MBSP Beneficiaries 25 6.1 Millet Producers 25 6.2 Millet Producer Organizations 27 6.3 Processing Units 28 6.4 Other Beneficiaries 32 7. Activities of the Millet Producers 33 8. Millet Production 34 8.1 Input Providers - Seed 38 8.2 Input Providers - Equipment 40 8.3 Pre-Flowering Techniques 41 9. Effectiveness of the MBSP Training Program 43 9.1 Satisfaction of Participants with the Training Program 44 9.2 Soil Preparation 44 9.3 Seeding 46 9.4 Crop Monitoring and Assisted Natural Regeneration 46 9.5 Harvesting and Storing 47 9.6 Business and Marketing 48 9.7 Challenges 49 10. Importance of Membership in an Organization of Millet Producers 51 11. Quality of Processed Millet 53 11.1 Hygiene 53 11.2 Packaging 54 12. Linking Producers and Processors to the Millet Market 55 3 12.1 From Producers to Processing Units 55 12.2 Processing Units to National Market and Export 58 12.3 Referrals from NCBA CLUSA 58 13. Access to credit 59 14. Efficiency of the Project Management 63 15. Conclusions 66 Annexes 69 Annex A – Evaluation Design Matrix 70 Annex B - Analysis of the Performance Indicators 71 Annex C – MBSP Theory of Change 72 Annex D – Qualitative Interviewers Guides 73 Annex E – Survey Questionnaires 74 Annex F – List of Documents Reviewed 75 4 Acronyms AI Appreciative Inquiry ANCAR Agence Nationale de Conseil Agricole et Rural (National Agricultural and Rural Advisory Agency) APIX Agence Nationale pour la Promotion des Investissements et des Grands Travaux (National Agency for Investment Promotion) APS Agents Prestataire de Service APS-IF Agent Prestataire de Services Financier (Intermediate Financial Service Providers) CAPI Computer Assisted Personal Interviewing CCIAK Chambre de Commerce, d'Industrie et d'Agriculture de Kaolack (Chamber of Commerce, Industry and Agriculture of Kaolack) CF Conservation Farming/Agriculture de conservation CISP Credit Intermediary Services Providers CO Community Organization CRDH Centre de Recherche pour le Développement Humain DRDR Directions Régionales De Développement Rural (Directorate of Rural Development) FAS Foreign Agricultural Service FGD Focus Group Discussion FIARA Foire Internationale de l'Agriculture et des Ressources Animales (International Fair of Agriculture and Animal Ressources) FFPr Food for Progress FTF Feed the Future Initiative FTI Food Technology Institute GIE Groupement d’intérêt économique HACCP Food Safety and Hazard Analysis Critical Control Point ISRA Institut Sénégalais de Recherches Agricoles (Senegalese Agricultural Research Institute) ICS Industries Chimiques du Sénégal (Chemical Industries of Senegal) JAA J.E. Austin Associates, Inc. MECAT Mutuelle d’épargne et de crédit Aprofes Teranga Savings and Credit Mutual Association APROFES (Association for the Promotion of Senegalese Women) - TERANGA M&E Monitoring & Evaluation MBSP Millet Business Services Project MC Management Committee 5 MFI Micro-Finance Institution MIS Market Information System MTE Mid-Term Evaluation MVCP Millet Value Chain Project ANSD Agence Nationale de Statistique et de la Démographie (National Agency for Statistics and Demographics) NCBA CLUSA National Cooperative Business Association Cooperative League of the United States of America NGO Non-Governmental Organization PAMECAS UM-PAMECAS Union des Mutuelles du Partenariat pour la Mobilisation de l’Épargne et du Crédit Au Sénégal (Partnership for the Mobilization of Savings and Credit in Sénégal) PU Processing Unit P-HT Post-Harvest Technologies PMP Performance Monitoring Plan PO Producer Organization SEDAB Sahélienne d’Entreprise de Distribution en Agrobusiness (Association of Sahelian Agribusinesses and Distributors) SENCHIM Agro-Chemical Manufacturing Company of Senegal SILC Savings and Internal Lending Communities TOR Terms of Reference U-IMCEC Union des Institutions Mutualistes Communautaires d’Epargne et de Crédit (Union of Community Mutual Institutions for Savings and Credit) USDA United States Department of Agriculture 6 Executive Summary The Millet Business Services Project (MBSP) is a follow-on project to the U.S. Department of Agriculture’s (USDA) Millet Value Chain Project (MVCP) implemented by NCBA CLUSA from May 2009 to January 2014. According to the MVCP Final Evaluation Report1 , the project has successfully increased millet yields and sales in the regions of Fatick Kaolack and Kaffrine by providing technical assistance to increase production. The project facilitated access to financing and credit as well as to market information. It supported the organization and formation of millet producer groups and strengthened their business capabilities. The MBSP continues the work of the MVCP in addition to focusing on introducing new farming technology to increase productivity and mitigate the effects of climate change. The project is addressing quality in post-harvest handling and millet processing as well as building producer organizations’ ability to access markets. The project is working with all the actors along the Millet Value Chain from the input suppliers to the millet traders. The MBSP is building private sector extension services for agricultural training. During its project period, the MBSP is aiming to work with 22,150 producers, 16,630 trade beneficiaries, numerous government agencies, and the private sector in Fatick, Kaolack and Kaffrine, which is the Peanut Basin of Senegal, and in Dakar where most of the millet processing activities take place. The MBSP began in October 2014 and is scheduled to end in March 20192 . It is funded through a Food for Progress (FFPr) grant. The two objectives of the program are to: 1. Increase the agricultural productivity of the millet value chain by: • Developing processing systems; • Building producers’ capacity to improve production and quality; • Training producers and processors in improved techniques, post-harvest handling, marketing, and seed production; and • Providing grants and loans for equipment and inputs. 2. Expand trade of millet by: • Developing public-private relationships; • Facilitating trade relationships, researching export opportunities, and building agricultural extension capacity; and • Promoting food safety issues and requirements. In February 2018, J.E. Austin Associates, Inc. (JAA) was awarded the contract to conduct this independent third-party evaluation as per the requirements of USDA. This Mid-Term Evaluation (MTE) has assessed whether the MBSP is on track to achieve the expected results outlined in its results framework during the period starting October 2014 to April 1st, 2018. JAA used a mixed-approach methodology to conduct this mid-term evaluation, collecting data using both quantitative and qualitative research methods. This mid-term evaluation finds that the Millet Business Services Project was well thought out and its activities were designed to meet its two objectives listed above. The Evaluation applauds NBCA CLUSA for taking a calculated approach in the program’s design to prevent dependency on grants and donations on the part of the beneficiaries. This mid-term process evaluation finds that the program is in line with USDA 1 USDA Food for Progress Enhanced Food Security for Senegal via an Improved Millet Value Chain, Final Evaluation Report, April 2014. 2 This end date is based on the current contract with USDA. However, NCBA CLUSA has requested an extension of the project through December 2019 7 objectives for sustainable economic growth and development and confirms that the activities and initiatives that were agreed by USDA and NCBA CLUSA have been put in place. With regards to the first program objective, which is to increase agricultural productivity of the millet value chain, the Evaluation finds that there are positive trends in the adoption of new farming technology and signs that the MBSP beneficiaries are cultivating and producing better quality millet. The initiatives of the programs directed to this first objective include training programs in millet cultivation knowledge, pest and disease management, soil fertility management and mastering climate change adaptation concepts. Millet Producers and the Agents prestataire de services (APS) agree that the training is very useful, and they will recommend the training to others. There are several kinds of APS in this program: • APS Semenciers, trained to produce and sell better quality millet seeds to farmers. • APS Fertilisants, who will supply the farmers with approved fertilizers. The Program has not started working with these persons at the time of the evaluation and this initiative was not reviewed by the evaluation. • APS Artisans, who are building kiosks for the Cantine Fondé Initiative. Here NCBA CLUSA works with street vendors who sell a millet porridge called Fondé. Fondé is usually eaten for breakfast and dinner in rural areas and is also used in family ceremonies. MBSP believes that by introducing millet, especially Fondé, as a nutritious staple option to urban households, the program will support income generation in rural areas while increasing food security across the country. The program trains street vendors in hygienic millet processing and cooking to professionalize Fondé sales and brand a high-quality millet product. APS Artisans are also manufacturing rippers for soil preparation and seeding. The ripper was originally designed by the Zambia Conservation Farming Unit and patented by the African Intellectual Property Organization located in Cameroon. NCBA CLUSA has improved the model for easier use and efficiency. • APS Packaging are participating in the program to promote the use of Kraft and aluminum paper, which are environmentally friendly compared to plastic as well as labelled plastic bags. The APS’s are trained by NCBA CLUSA and its partners l'Agence national de conseil agricole et rural (ANCAR) and les Directions régionales de développement rural (DRDR). These two agencies have been contracted to deliver the training program to the Millet Producers. The Program is also training Extension Agents who support millet farmers in the villages. The training and trainers are rated very highly. However, all agree that training is challenging for illiterate farmers, especially for women. The Evaluation recommends that NCBA CLUSA supplement current teaching methods with additional resources for farmers who are facing difficulties during training. The Evaluation and ANCAR found that limited access to funds and credit is a major barrier to project success; however, NCBA CLUSA is addressing this by subcontracting Catholic Relief Services (CRS) to assist beneficiaries to raise funds and access credit. When Producers do not have access to funds, they do not always apply the new techniques they have learned, even if they believe in the new technology. Producers explain that they do not have the money to purchase inputs such as high-quality seeds and fertilizers or tools and equipment. In turn, the APS Artisans cannot sell rippers as easily and often as they had hoped, although the rippers have good reviews. Institut Sénégalais pour les recherches agricoles (ISRA) has purchased them. The APS Artisans explain that lack of money also prevents them to build and stock rippers. The Processing Units mention that they cannot invest in tools and equipment, buy raw millet in large quantities or buy Kraft paper for packaging. It should be noted that the MBSP has a grant and loan initiative for input acquisition, including equipment. This requires that the applicant funds part of the 8 costs. The evaluation recommends that the program beneficiaries are further encouraged to apply for the grants. CRS is implementing its flagship Pathway to Prosperity Model as an initiative of the MBSP. CRS has established a network of Intermediate Financial Service Providers called APS-Financier (APS-IF); they have received training in the CRS Savings and Internal Lending Communities (SILC) methodology, as well as on financial education, marketing basics, and working with financial institutions. SILC is a savings-led microfinance approach that allows poor households to save and borrow to increase their income. After passing an exam, the trained APS-IF become CRS certified APS-IF; their role is to promote the SILC methodology among farmers, support the creation of SILC groups and encourage the SILC members to carry out income-generating and savings activities within their communities. They also support the farmers in their loan applications with financial institutions. CRS has recruited 17 APS-IFs since the start of the program and created 416 SILC Groups among millet producer organizations with total savings of $203,000 USD (over 113 million CFA francs) to date. By May 2018, a total of $836,000 (over 466 million CFA francs) in loans were distributed to farmers by SILC groups and banks3 . The Evaluation finds that the producers and the Producer Organizations are very positive towards the SILC initiative. NCBA CLUSA noted that the savings generated by the SILC Groups are not always used in millet related agricultural activities, therefore there is still additional room for benefit for the MBSP from the SILC program. NCBA CLUSA cited a case where SILC monies were used to start a business in soap manufacturing. The Evaluation thinks that given the high rating of SILC, it is not surprising that the villagers are using this methodology to raise money for other purposes, especially when the farmers, who are generally poor, face other pressing needs for money. However, the Evaluation finds that it is fundamental that the persons who receive training from the MBSP are committed to directing their SILC Savings to millet-related activities, expenditures and investments,such as purchasing rippers. The Evaluation recommends that NCBA CLUSA reviews the objectives of the SILC Program and the other activities related to access to credit with CRS. The Evaluation finds it important that MBSP establish clear targets for the number of SILC Groups formed and funds raised for investments in the millet sector. The Evaluation also recommends that Catholic Relief Services keeps track of the disbursement of the SILC savings, categorizing them in millet and non-millet expenditure and investments for monitoring and research purposes. The Evaluation has learned that the certified APS-IFs are paid by the communities or SILC Groups for the service they provide. CRS explained that, as such, the program is creating work for the individuals who have been certified and hold the position of APS-IFs. The Evaluation learned that APS-IF, who are usually young producers and members of POs, enjoy a high social status in the communities they serve. Because of this, the Evaluation believes that APS-IF work should be closely monitored to ensure there is no misuse of power. The Evaluation recommends that NCBA CLUSA review the role and responsibilities of the CRS staff and APS-IFs on this project and ensure that there is no misrepresentation of the organization, MBSP or USDA. With regards to millet processing, the Evaluation finds that there is an increase in the volume of raw millet treated by the MBSP beneficiaries since they have entered the program, with additional growth possible when intermittency issues are resolved. For example, the Evaluation noted that in 2017, less than 50% of the MBSP Processing Units operated throughout the year. The Evaluation finds that 3 Sustainable Financial Services Tailored to the Millet Value Chain, Catholic Relief Services, June 2018. 9 disruptions in operation diminish the reliability of demand and supply for raw millet and finished millet product. Increasing Processing Units’ access to equipment and credit would likely improve operation uptime. The census of the processing units found that after training unit cleanliness, equipment maintenance, and respect for quality and hygiene norms and standards all improved. However, not all processing units are adopting hygiene and safety practices at the same rate following training. The evaluation recommends that NCBA CLUSA continues to emphasize hygienic standards and draw attention to deficiencies during site visits. To help its beneficiaries increase sales, the Program assisted and financed the presence of the Producer Organizations and Processing Units at the business forums and agricultural fairs for visibility with wholesalers, distributors and exporters. The participants spoke highly of their experiences and the sales made there. NCBA CLUSA has provided a copy of a contract for the purchase of raw millet from a GIE by an export company and the purchase order for processed millet. The Evaluation believes that the program has the opportunity to expand the trade of millet even further and suggests that it appraises its program strategy with processors, especially the small and medium size units, to improve hygiene and safety in their operations, support the acquisition of tools and equipment, and promote year-long operation and long-term contracts with the Hubs for the supply of raw millet to improve operational sustainability. The Evaluation also suggests that training programs in processing technology and business methods are customized to the production capacities of the processing units to supply millet products to local, national and export markets. To reach and support producers, the Program is using two methodologies: 1. Multi-Services Hubs, where several POs are consolidating their activities, and 2. Employing Agricultural Extension Agents who accompany the producers who have been trained in farming and business techniques. The Evaluation finds that the Hubs offer several advantages, among them is the increased negotiating powers of the POs with the actors in the Millet Value Chain, including the financial institutions and the ability to negotiate the sale of large volume of millet as producers combine their harvest. The Evaluation believes that setting SILC Groups within the Hubs can ensure that the monies are used in millet-related activities. The Evaluation thinks that the Hubs can be more effective in supporting illiterate farmers than the support of Agricultural Extension Agents, since the Hub environment provides constant assistance. Furthermore, safety and quality standards are more easily enforced at the Hubs than at the level of the individual millet producer. This evaluation recommends that the Multi-Service Hub approach be examined further for program cost￾efficiency and quality of outcomes and output. The evaluation recommends that future studies, including the end-line evaluation, compare the impact of the program on the Multi-Service Hubs Producers and on independent producers, i.e. those who are not members of a Hub. Such investigation will identify if the Hubs approach fosters more effective integration of input suppliers, millet producers and processors in the value chain while offering products of higher quality in large volumes that can sustain trade both nationally and abroad. The survey shows that the program is having the most positive results in Kaolack. Since there are no baseline data for the three regions, it is not known whether the millet sector was stronger in Kaolack than in Fatick and Kaffrine before the program started. It is interesting that the yield study performed by DRDR shows that the production of millet is superior per hectare in Kaolack compared to the two other regions both for the demonstration and control groups. NCBA CLUSA has explained that there was a greater presence of the Feed the Future (FTF) program4 in Kaolack than in Fatick and Kaffrine. This may have 4 Yaajeende is the FTF program in Senegal 10 contributed to a healthier and more developed agricultural sector in Kaolack. The Evaluation finds that the program results trail in Kaffrine, where there is a higher proportion of women among the MBSP beneficiaries. Although women are quite active in the agriculture sector, they are known to face more challenges than men. For example, women are also responsible for running their households in addition to their work in the field and they often have limited access to land ownership. The Evaluation finds that it would be reasonable to reinforce the program in Fatick and Kaffrine during the second period of this program. Farmers are for the most part cultivating land that has been passed down to them through inheritance. They are unable to lease more land to cultivate millet because of the lack of funds. The Evaluation finds that limited access to land can constrain the quantity of millet that farmers can produce, which can delay or diminish achieving the project’s second objective of expanding trade of millet. The Evaluation finds that to confront all the actors along the value chain in a single program is an ambitious and challenging undertaking. This mid-term evaluation concludes that MBSP program will achieve its goals and, with some minor adjustments, can improve its impact even further. The Evaluation Team believes that beyond reaching the project’s performance indicator targets, it is also essential that the gains made by the beneficiaries through MBSP be sustained in the long run. Hence, in the event of natural disasters that can be expected as the result of climate change, for example, excess rainfall that caused damage to millet production in 2016, the greatest accomplishment of this program will be its resilience and ability of its beneficiaries to continue to cultivate and process millet. 11 1. Background The Millet Business Services Project (MBSP) is a follow-on project to the U.S. Department of Agriculture’s (USDA) Millet Value Chain Project (MVCP) implemented by NCBA CLUSA from May 2009 to January 2014. The MVCP Millet Value Chain Project was a contribution to the Food for Progress Program, whose goals are to accelerate agriculture sector growth and to improve the nutritional status of people. According to the MVCP Final Evaluation Report5 , that project successfully increased millet yields and sales in the regions of Kaolack, Fatick, and Kaffrine by providing technical assistance to increase production and improve productivity. MVCP undertook a series of successful training, coaching and capacity building programs for entrepreneurship development and businesses, which, according to the MVCP Final Evaluation, led to the formalization of business relationships and cooperation between millet value chain actors and their partners. The evaluation found that improved credit management and quality input supplies through intergroup associations and unions contributed to Millet Value Chain development in Senegal’s Peanut Basin. Working with Producer Organizations (PO), the project furthered conservation and sustainable management of natural resources, allowing millet producers to have a better, deeper understanding of land degradation, poor farming practices and actions to restore and avoid adverse effects on the environment. MBSP continues to provide training to millet producers, focusing on environmentally friendly farming techniques to increase productivity and improve yield. In addition, the project is addressing quality in post-harvest handling and millet processing as well as building producer organizations’ ability to access markets. The project works with all the actors along the Millet Value Chain, from the input suppliers to the millet traders. The MBSP builds private sector extension services for agricultural training, which are in short supply in Senegal. During its project period, which runs from October 2014 to March 20196 , the MBSP aims to work with 22,150 producers, 16,630 trade beneficiaries, numerous government agencies, and the private sector in the regions of Senegal which initially benefited from MVCP, namely, Kaolack, Kaffrine, and Fatick, as well as Dakar and its surrounding areas. The two objectives of the program are to: 1. Increase the agricultural productivity of the millet value chain by: • Developing processing systems; • Building producers’ capacity to improve production and quality; • Training producers and processors in improved techniques, post-harvest handling, marketing, and seed production; and • Providing grants and loans for equipment and inputs. 2. Expand trade of millet by: • Developing public-private relationships; • Facilitating trade relationships, researching export opportunities, and building agricultural extension capacity; and • Promoting food safety issues and requirements. The MBSP is funded through a Food for Progress (FFPr) grant. The USDA requires that USDA-funded projects undergo interim evaluations to assess implementation progress, provide early signals of interventions’ effectiveness, document lessons learned to assess the sustainability of efforts to date, and 5 USDA Food for Progress Enhanced Food Security for Senegal via an Improved Millet Value Chain, Final Evaluation Report, April 2014. 6 This end date is based on the current contract with USDA. However, NCBA CLUSA has requested an extension of the project through December 2019 12 discuss and recommend mid-course corrections. This Mid-Term Evaluation (MTE) of the MBSP was conducted per USDA Monitoring and Evaluation (M&E) Policy requirements. In February 2018, J.E. Austin Associates, Inc. (JAA) was awarded the contract to conduct this independent third-party evaluation. The evaluation period extends from October 2014 to April 1st, 2018. 2. Evaluation Purpose This Mid-Term Evaluation (MTE) has assessed whether the MBSP is on track to achieve the expected results outlined in its results framework. JAA designed and conducted this study while consulting the USDA M&E Policy Manual7 . The findings of the MTE will allow the USDA, the Government of Senegal, and NCBA CLUSA and its partners to assess the following aspects of the MBSP: • Relevance and progress of project activities towards achieving project goals and objectives; • Soundness of project approach and efficiency of program systems and processes; • Early signs of project effectiveness and performance in relation to achieving targeted objectives, project ownership by beneficiaries and partners, and sustainability of results achieved; • Constraints, lessons learned, and evidence-based best practices; • Strategic opportunities to build upon project achievements thus far to scale activities in the second half of the project term and/or recommend actions for mid-course correction. This present report provides the results of this MTE. 3. Evaluation Design and Methodology JAA used a mixed-approach methodology when conducting this mid-term evaluation, collecting data using both quantitative and qualitative research methods. JAA has examined the RFP’s evaluation questions and formulated sub-questions to better respond to these initial questions. This process guided the development of the MTE’s data collection tools. JAA then prepared an Evaluation Design Matrix adapted from Rist and Morras’ model8 . This matrix, presented in Annex A, provides descriptions of the following items for each sub-question, ensuring that data is collected for each evaluation question: • Measures/Indicators • Data collection instruments • Data analysis methods The evaluation questions have been addressed using multiple lines of evidence. JAA combined primary quantitative and qualitative data with information gathered from document review of program reports, research on the millet sector in Senegal and information obtained from the project monitoring system. While performing the evaluation, the Evaluation Team adhered to ethical principles. The data collection protocol respected the rights of all interviewees; consent was obtained from survey respondents, participants in Focus Group discussions (FG), and from Key Informants (KI) before any interview started. The Evaluation Team reviewed the data collection methodology of the MBSP Baseline Study as well as documents provided by NCBA CLUSA following the initial kick-off meeting among JAA, the USDA, and NCBA CLUSA. After careful review of the methodology used for the Baseline Study, the program activities, and the performance indicators, and after discussions with the NCBA CLUSA Monitoring and Evaluation 7 Monitoring and Evaluation Policy, USDA Food Assistance Division, Office of Capacity Building and Development. 2013. 8 The Road to Results: Designing and Conducting Effective Development Evaluations, Ray C. Rist and Linda G. Morras Imas. 2009. Page 243. 13 Expert in Kaolack, the Evaluation Team has adopted a new strategy to conduct the quantitative and qualitative research components of the MBSP’s Mid-term Evaluation. This MTE did not follow the methodology used in the Baseline Study; it did not use or adapt the Baseline Study’s data collection tools. It has instead followed the methodology specified in the Term of Reference and has collected data to specifically answer the evaluation questions. The Evaluation Team examined the MBSP’s performance indicators, which have been summarized by the Evaluation Team Lead; the list of Performance Indicators for the project are listed in Annex B. Where feasible, the Evaluation Team collected data to provide estimates of the quantitative indicators. This evaluation relied on the following sources of information: 1. Document Review 2. Qualitative Research • Key Informant Interviews (KIIs) of NCBA CLUSA Staff and Partners • Focus Group Discussions (FGDs) of Millet Producers and Managers of Processing Units • Field observations 3. Quantitative Research – Four surveys were conducted: • Survey of Millet Producers • Survey of Producer Organizations • Survey of Processing Units • Survey of Trained Processing Unit Staff. 3.1 Qualitative Research Qualitative interviews were undertaken by an agronomist and sociologist over a nine-day period. Table 1 shows interview types and participants by region. Each FG lasted about 45 minutes and KIIs averaged 30 minutes. The Interview Guides can be found in Annex C. Table 1 - Qualitative Interviews by Type and Region Kaffrine Kaolack Fatick Diourbel Dakar TOTAL KII FG KII FG KII FG KII KII Producers Male 1 1 1 3 Producers Female 1 1 1 3 Processors 1 1 1 2 2 7 Trainers (Ancar and DRDR) 1 1 1 3 APS Seed Suppliers 1 1 1 3 APS Artisans 1 1 1 3 APS Packaging 1 1 NCBA CLUSA and CRS Project staff 7 1 8 Financial Institution 1 1 2 TOTAL 4 3 11 4 4 3 2 2 33 14 3.2 Survey Methodology Questionnaire Design The survey questionnaires were designed by the Evaluation Team and reviewed by the NCBA CLUSA Country Office’s M&E Officer. The survey questionnaires are summarized below; copies of each questionnaire are presented in Annex D. Survey of Millet Producers This survey aimed to collect data related to the several output and outcome indicators (see Annex B). It collected information that allowed the measurement of changes in production practices, producers’ millet sales and revenues of people who have received assistance from the MBSP. Producers were asked to provide data on production, revenue, and expenses for 2015, 2016, and 2017. Producers were also asked to report their satisfaction with the MBSP training program and their perception of the influence of the program on the community and non-beneficiaries. Other interventions explored in this survey included: • Producers’ relationships with the buyers of the millet they grow and harvest • Access to markets • Access to credit Survey of Producer Organizations In this survey, a senior officer of the organization was interviewed to provide information about the members of the organization and the type of support and assistance received from the MBSP, including access to credit, supply of material, and equipment and training. The survey also asked about the difficulties that organization’s members face in the Millet Value Chain, such as input supply, storage, markets, and pricing. Survey of Processing Units This survey targeted the owners or senior officers of Processing Units. The survey asked about processing methods and equipment, handling and storage of finished products, hygiene practices, and satisfaction with support received from the MBSP. Data were also collected regarding the origin and amount of millet processed as well as Processing Units’ revenues and expenses. Survey of Trained Processing Unit Staff This is a satisfaction survey of the Processing Unit staff regarding the training they received from the MBSP. The survey also investigated whether staff applied knowledge they obtained from the training. Sample Design JAA obtained the sample frame for each survey from NCBA CLUSA. This frame listed program beneficiaries and their contact information. The Evaluation Team selected survey respondents from these lists after verification and duplicate elimination. Table 2 provides a description of the sample design that was used for each survey. For each sampling protocol, JAA has weighed the statistical rigor of different options against the data collection costs and time to find the optimal balance. The margin of error is expected to be in the order of 5% to 7% at a 95% confidence level for aggregated country estimates. For regional estimates, the margin of error is expected to be no more than 5% at a 90% confidence interval. Table 2 - Summary of Surveys with Description of the Sample Design Fatick Kaolack Kaffrine Dakar Diourbel Total Method Targeted Sample Size Actual Sample Size Response Rate Sample Design Selection Method Producer organizations (POs) 319 239 177 0 0 735 Sample Survey 200 164 82% Stratified by Region Systematic PPS (probability proportional to size) Random selection Millet Producers 5,855 4,696 2,127 0 0 12,678 Sample Survey 868 802 92% Stratified by Region two Stage Cluster: 1st stage Village, 2nd stage Producer Random selection of villages and Producers. 22 villages in each region, 15 producers from each village Processing Units 19 31 34 89 2 175 Census 175 120 69% Census Select All CLUSA trained Employees 38 62 68 178 4 350 Sample Survey 266 194 73% Stratified by Region SRS (simple random sample) without replacement 2 CLUSA trained personnel in each of the Processing units selected Program Benficairies and Partners Population as of March 2018 Data Collection Methodology 3.3 Survey Data Collection Methodology Interviewers conducted the surveys using Computer Assisted Personal Interviewing (CAPI) techniques via tablets. The questionnaires were programmed using CSPro9 software, taking advantage of live data editing. Enumerators also had paper questionnaires in the event that an electronic device failed during field collection. Each interview location’s geospatial coordinates were collected. JAA has sub-contracted the Centre de Recherche pour le Développement Humain (CRDH) to administer the surveys. Five teams of interviewers, each team consisting of a supervisor and four interviewers, collected the data. The local consulting firm hired survey enumerators who had previous experience conducting agricultural surveys and who spoke local languages. Nevertheless, the interviewers were provided with detailed instructions on field procedures, such as introducing the survey to the respondents, handling tablets and safeguarding data. The teams received a three-day in-depth training on the content of the four survey questionnaires. Because of the survey interviewers’ experience conducting agricultural surveys, a survey pilot test was not deemed necessary. Additionally, there were no millet growers near Dakar and it would have been difficult to conduct pilot tests. Figure 1 shows the percentage of interview respondents by language for each survey. Over 90% of the surveys were conducted in Wolof; however, Figure 1 shows that surveyors were able to conduct surveys in the respondent’s language when necessary. Figure 1 - Language of Interview for each Survey: Percentage of Respondents The survey of Millet Producers and the survey of Producer Organizations were well received, with 92% and 82% response rates, respectively. The surveys conducted inside the Processing Units did not attain such high response rates: 69% for the units and 73% for the staff. Several units were not in operation 9Census and Survey Processing System (CSPro), U.S. Census Bureau. July 2, 2018. https://www.census.gov/data/software/cspro.html 89.8 90.3 4.8 3 1.8 95.1 0 0.8 4.1 95.1 1.5 0.5 3 0 10 20 30 40 50 60 70 80 90 100 Wolof Sérére Peulh Français/French Percentage of Respondents Language of Interview Producers Organisation of Producers Processing units Trained staff 17 during the data collection period or had reduced hours of operation due to Ramadan, which made it difficult to reach the selected respondents. 3.3 Ethical Responsibilities - Risks and Benefits This study was designed and conducted to eliminate any real or potential risks to the population. Measures undertaken to ensure protection and confidentiality during data collection, analysis, and report dissemination included, but were not limited to, receiving informed consent and adherence to national and international ethical responsibility standards; enumerator and interviewer training in confidentiality and respect; protection of all data and identifying information during collection, storage, and analysis; and exclusion of individual names in any and all qualitative research notes, field notes, and reports unless explicit informed consent was given. 3.4 Limitations of the Study This study took place under a tight schedule and limited budget, and, although the data collection exercise was completed on time, the team encountered challenges that could have affected the quality of the survey data. The following issues made data collection challenging, and JAA recommends that special attention be given to these issues in future studies. Documentation: The Evaluation Team found that documentation was not complete nor sufficiently detailed. As a result, survey tool development took longer than expected, requiring several discussions with the country M&E officer and thus shortening the time for tablet questionnaire programming and testing before field collection started. Consequently, the time initially allotted for tablet programming was shortened, resulting in reduced testing time and no pilot test. Several problems were found during data processing, requiring more data clean-up and coding. This caused delays in data analysis and subsequently the delivery of the evaluation report. Timing of the Evaluation: The evaluation was conducted during Ramadan. Most interviewers and respondents were fasting and were thus easily tired. This may have contributed to a lack of attention on the part of the enumerators; for instance, several questions needed recoding because interviewers often unnecessarily entered respondents’ comments or explanations when, in fact, the information provided by the respondent was among the response choices offered in the questions. NCBA CLUSA Database of Beneficiaries: During the survey, the team found that the village names recorded in the NCBA CLUSA database as places of residence of millet producers were in fact the villages where trainings took place. Thus, several producers could not be found in the sampled villages, reducing the expected number of respondents. There were two instances in which no one was found in the selected village because of this problem; thus, two additional villages were selected. During data collection, the team also discovered that several individuals were recorded in more than one database entry with different Numéro d’identification personnel (NIP)- the National Identification Number assigned by the Government of Senegal to its citizens. These duplicative entries most often occurred when the back and front numbers of the person’s ID card were recorded or when numbers were incorrectly entered. The survey team was unable to find several Processing Units and found several duplicates on the list. The Evaluation recommends that NCBA CLUSA reviews its monitoring system so that it can identify unique beneficiaries and adds quality assurance measures when updating files. 18 4. Program Activities and Partners The Millet Business Services Project is hard at work shaping and improving the millet value chain from input suppliersto farmersto millet processorsto consumersin the regions of intervention shown in Figure 2. The map also shows the intervention zones from the Phase 1 Project that were kept, those that were removed and intervention areas new to the millet project. Figure 2 - Zones of Intervention of the Millet Business Services Project (MBSP) The objectives of the program are to: • Increase the agricultural productivity of the millet value chain; and • Expand millet trade. To meet its objectives, the MBSP is implementing activities for actors along the millet value chain. The direct beneficiaries are: • Input providers: machinery, seed, and fertilizers • Millet Producer Organizations (PO) including the Groupements d’intérêt économique10 (GIE) • Millet Producers • Millet Processing Units and their Staff 10 Economic Interest Group (EIG). 19 The Evaluation Team has learned that NCBA CLUSA has emphasized sustainability of the program objectives in the design of this program by fostering public private partnerships and reducing dependence on giveaways. The MBSP is using the following principles to develop the activities: • The Organization of Millet Producers is the focal point. The beneficiaries are selected from the membership list, allowing for future follow-up, coaching, and mentoring. • The MBSP is building capacity within the PO, introducing new technology to improve productivity and millet quality while mitigating the hazards of climate change, developing transferable skills, and providing training in management and business operations while simultaneously introducing governance and democratic systems that can ensure the organization’s effective long-term operation. • To avoid dependency on the program and encourage local ownership, millet producers must pay a fee to the PO to receive MBSP training and benefit from other support. Individuals do not receive grants or donations. For example, no money or material is provided for free to the beneficiaries; all material program contributions are directed to the PO Processing Units and are subjected to the same rule: workers must be members of an association to receive training and support from the MBSP. • NCBA CLUSA links the various actors along the value chain and teaches sound business practices, but it does not interfere in transactions between actors. For example, NCBA CLUSA would not negotiate nor draw a contract for the millet producers. • The MBSP does not provide monetary credit; instead, Catholic Relief Services (CRS), sub￾contracted by NCBA CLUSA, is implementing its flagship Pathway to Prosperity Model. In the MBSP, CRS has established a network of Intermediate Financial Service Providers, or APS￾Financier (APS-IF), who have received training in the CRS Savings and Internal Lending Communities (SILC) methodology, as well as on financial education, marketing basics, and working with financial institutions. The APS-IF then train beneficiaries on group savings mobilization, credit, and governance. Given the project’s focus on capacity building, the MBSP has many training components, which include millet cultivation demonstration plots and collective farms. The MBSP has trained seed producers known as Agents prestataires de services (APS) Semenciers to produce better quality seeds for millet farmers. The project will soon enroll APS Fertilisants, who will supply the farmers with fertilizers. The program actively promotes composting and the use of organic fertilizers in millet cultivation. The MBSP is also training agricultural Extension Agents in the villages. Their role is to coach and support the MBSP beneficiaries while the project is being implemented and after the end of the project. The MBSP has worked with the APS artisans to produce an improved version of a ripper, which was originally designed by the Zambia Conservation Farming Unit and patented by the African Intellectual Property Organization, located in Cameroon. The ripper is an animal-drawn tool that makes furrows that are 15 cm wide and 15 to 30 cm deep, depending on the type of soil. It is a versatile tool that allows producers to rip into the soil; hoe or weed; plough and make ridges. NCBA CLUSA has worked with the APS Artisans to improve the original design and adapt the tool to soil in the project target zones. They went through several trials and produced the “ripper G4”, or the fourth prototype that has been adopted by the producers. MBSP has purchased several rippers that were donated to the POs and villages. The APS artisans are allowed to manufacture, market and sell this tool to the public at large. NCBA CLUSA is also working with the APS Artisans to build kiosks for the Cantine Fondé (Cantine) Initiative. The goal of the initiative is to make millet more visible on the local markets while enhancing the 20 respectability of millet producers and vendors. Millet is a staple food in Senegal, but value-added products like enriched flour have not been popular on the market. Inadequate processing and sales conditions have contributed to low-quality products. MBSP is aiming at increasing the productivity of growing millet as well as increasing consumption to support a sustainable and solvent millet market. Millet porridge, called Fondé, is usually eaten for breakfast and dinner in rural areas and is also used in family ceremonies. This MBSP Cantine Fondé initiative works with porridge street vendors, who the program trains in hygienic millet processing and cooking. The program hopes that professionalization of the Fondé sellers leads to product branding and a reputation for high-quality product. MBSP believes that by introducing millet, especially Fondé, as a nutritious staple option to urban households, the program will support income generation in rural areas while increasing food security across the country. This will subsequently support a sustainable and solvent millet market. APS Artisans are building the kiosks according to the design specifications of NCBA CLUSA and the kiosks carry similar store signs. Staff wear the same outfits and the same cooking utensils are used. Only approved millet products can be sold from these kiosks. The products are sold in hygienic packaging and standardized quantities. The millet kiosks are only available to members of millet processing cooperatives whom the program has trained in business practices and hygienic processing. The Cantine owners are encouraged to brand their product and build a reputation for quality. To promote and enhance the quality of processed millet, the MBSP is working towards the production of quality products in hygienic conditions, sold in clean and attractive packaging. The MBSP emphasizes the importance of hygiene in the post-harvest storage, handling, and processing steps of the millet value chain. 21 Millet has long been cultivated as a subsistence crop in Senegal, and farmers are still using traditional cultivation methods. To enable millet’s transition from a subsistence crop to a local and export commercial crop, NCBA CLUSA has renovated millet storage warehouses and converted storage areas into Multi-Service Hubs which are efficient warehouse organizations that are run by POs. The services that the hubs provide include: • Storage • Threshing • Ploughing • Support and advice (agricultural extension workers) • Pre-processing of millet • Sale of raw millet • Equipment rental • Sale of inputs (compost, chemical fertilizer, and seeds) • Access to loans for agricultural production to be repaid in kind. This last service, i.e. reimbursement of loans in kind, was initiated by the POs to reduce unpaid debts and increase financial institutions’ trust towards millet producers. NCBA CLUSA has explained that its involvement in this case has been limited to assisting producer organizations in making connections with bank and financial institutions. The organization is not involved in the negotiation process between the banks and the millet suppliers. This is done by the POs, which take the responsibility to stock clean millet that is sold to repay the loans contracted by the farmers with the financial institutions. At the pre-flowering stage, the hubs negotiate with input providers to obtain quality products at competitive prices for farmers. Farmers bring their harvest to the hubs, where the millet undergoes post￾harvest processing to produce high-quality raw millet, which is then sold to Processing Units. The multi￾service hub thus functions as a millet cooperative in rural Senegal. Furthermore, the construction of the Multi-Service Hubs has generated positive externalities by creating jobs for brick-layers trained by NCBA CLUSA. After working on the Hub project, the trained brick-layers are able to find jobs in their community. The project is also working with APS Emballage towards the establishment of local plants to provide packaging material. MBSP trained the Processing Units on different packaging methods, especially Kraft paper, which is used to make paper bags that preserve product freshness, avoiding contamination and reducing environmental damage. To facilitate Buyer-Seller Relationships, MBSP is organizing business forums and supporting the presence of its beneficiaries at the annual International Fair of Agriculture and Animal Resources exhibition, la Foire Internationale de l'Agriculture et des Ressources Animales (FIARA). Participation in the FIARA is an opportunity to highlight the partnerships between USDA, NCBA CLUSA and various communities, as well as to improve the chances that project beneficiaries acquire new partners. MBSP has also trained individuals to become APS en commercialisation. These service providers support the various actors in their commercial negotiations. NCBA CLUSA partners with several stakeholders, including CRS, to enable beneficiaries to access credit and funds. NCBA CLUSA has partnered with l'Agence national de conseil agricole et rural (ANCAR) and les Directions régionales de développement rural (DRDR) to train farmers. In Phase 1, the millet program provided training in millet production and marketing services while promoting and providing support to establish the PO. In Phase 2, MBSP is focused on developing processing systems, improving production quality, post-harvest handling, and expanding millet trade through public-private partnerships. NCBA 22 CLUSA is reinforcing the PO’s capacity across all cultivation phases and millet sector marketing along the value chain as well as teaching them good governance practices. The MBSP’s partners are shown in Figure 3 and the Program activities in Figure 4 on the next pages. The Evaluation finds that The Millet Business Services Project is a large and ambitious undertaking which involves many players and collaborators over a wide geographical area. This broad footprint has presented numerous challenges, some of which have been identified by this evaluation and described in the following section of this report; for example, training illiterate beneficiaries, and financial limitations of beneficiaries hindering acquisition of inputs, such as quality seeds and tools designed by the program. 23 Figure 3 - Partners in the Millet Business Services Program Partners of NCBA CLUSA Beneficiaries of MBSP (CA) Facilitate Buyer-Seller Relations Facilitate Buyer-Seller Relations Facilitate Buyer-Seller Relations Facilitate Buyer-Seller Relations Input Production Post Harvesting Processing Market Retail Wholesalers Export Financial Services: Facilitate Agricultural Lending Micro Finance Institutions (MFIs) Village Based Credit intermediary Service Providers (CISPs) Association of savings and Credit Unions Teranga Credit and Savings Union Independent Cooperative for the Reinforcement of Economic Initiatives and Microfinance Mutual Credit of Senegal National Agricultural Credit Fund of Senegal Agrodealers and Input Suppliers Seed Providers Fertilizer providers Conservation Agriculture (CA) Plowing Services Machinery providers Agro-chemical manufacturing company of Senegal Capacity Building: Agricultural Extension Agent Services, and the Govefnment Sector Senegal Institute for Agriculture Research (ISRA, in French) Direction Régionale de Développement Rurale (DRDR) Agence Nationale de Conseil Agricole et Rurale (ANCAR, National Agricultural And Rural Advisory Agency) Unions Inspection Régionale des Eaux et Forets (IREF) West Africa Agricultural Productivity Program Regional Seed Producer Union Local Development Support Group Food Technology Institute Producer Organization (OP) Groupement d’intérêt économique (GIE) Millet Producers Millet Processors Catholic Relief Services 24 Figure 4 - Activities and Initiatives in the Millet Business Services Program along the Value Chain Facilitate Buyer-Seller Relations Facilitate Buyer-Seller Relations Facilitate Buyer-Seller Relations Facilitate Buyer-Seller Relations Millet Business Services Project Initiatives Input Production Post Harvesting Processing Market Retail Wholesalers Export Training: Improved Agricultutal Production Techniques Training: Improved Post Handling and Processing Infrastructure: - Training bricklayers - Training of bricklayers - Renovation of Storage Units - Construction of Hubsstruction of Cantine Fondé Trade Shows Facilitate Access ro credit and implementation of the Savings and Internal Lending (SILC) methodology of the Catholic Relief Services Training: Business and Management Training: Input Providers - Machinery, Seeds Develop Demonstration Plots and Collective Farms 25 5. Data Analysis - Findings and Recommendations The Evaluation Team found that all informants were very willing to participate in discussions. The Evaluation Team performed in-depth analysis of the information collected during the interviews and focus group. The survey data was first edited for consistency and then weighted by region for the producers, the PO, and trained staff of the Processing Units. Although a census was conducted for the Processing Units, weighting for non-response was required. The Processing Units were operating fewer hours during Ramadan, making it difficult to reach the survey respondents, as explained above. The Evaluation Team prepared data tables and performed a descriptive analysis, paying attention to sampling errors and the number of missing values. The data collected was then analyzed and findings from each data source were triangulated before inferences were made. Wherever possible, the evaluation has analyzed the data by sex to identify gender differences, if any. The Evaluation team was provided with the list of performance indicators for MBSP according to the USDA guidelines for developing Food for Progress indicators11. The Evaluation has reviewed and updated this list, referring to the Food for Progress Identification as well as identification of data sources. Annex B contains the Performance Indicators along with the estimates calculated using data from the surveys conducted by this evaluation, where possible. This evaluation has not used the data from the MBSP Baseline Surveys because of the limitation of the methodology and data collection tools of the study. Instead, where possible and relevant, the evaluation has collected information on millet production and processing for 2015, 2016, and 2017. The Evaluation is very much aware of the limitations and issues that result from recall and proxy responses. The Evaluation has hence focused on trends rather than quantitative survey estimates in its analysis. This Mid-term Evaluation has made recommendations for enhancement in the program implementation based on its findings. It has taken into consideration the program context and the feasibility of the adjustments suggested when making these recommendations. 6. Profile of the MBSP Beneficiaries The Evaluation has collected information on each of the direct beneficiaries of the Program and presents its findings on the demographic variables of individuals, size, and legal status of the companies. 6.1 Millet Producers According to the data received from NCBA CLUSA, as of March 2018, there were 12,678 registered millet producers participating in the program. The ratio of men to women is roughly 3 to 1, as shown in Table 3. However, in Kaffrine, this ratio is reversed, with more women in the program than men. 11 Food for Progress and McGovern-Dole Indicators and Definitions, USDA Food Assistance Division, Office of Capacity Building and Development, August 16th, 2016. 26 Table 3 - Number of MBSP Millet Producer Beneficiaries, by Sex12 Region Male / Homme Female / Femme Total Fatick 3811 2044 65% 35% 5855 Kaolack 3056 1640 65% 35% 4696 Kaffrine 806 1321 38% 62% 2127 Total 7673 5005 61% 39% 12678 The Survey of Producers shows that over 80% of the Millet Producers are cultivating land that has been passed to them by inheritance, 24% are renting land, and 13% are working on land loaned to them temporarily. The results of the survey regarding land access is shown in Figure 5. These numbers do not add up to 100% since the Millet Producers can have more than one mode of access to land. Figure 5 - Types of Access to Land The Producers say they are limited in the amount of land that they can cultivate; they cannot lease more land due to lack of funds. Hence, there are limitations for the farmers to increase the land surface that they wish to cultivate, which subsequently prevents MBSP from achieving its goals with regards to increased areas under millet cultivation and the amount of millet produced by the program beneficiaries. 12 Source: NBCA CLUSA. 85% 4% 5% 24% 13% 2% 0% 7% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Property through inheritance / Propriété par héritage Property purchased / Propriété par achat Property donated / Propriété par don Leasing / Location Loan (temporary transfer to counterpart) / Prêt (transfert temporaire à un homologue) Affected national domain / Domaine national affecté Non-affected national domain / Domaine national non affecté Co-ownership / Copropriété Percentage of Producers 27 6.2 Millet Producer Organizations According to the NCBA CLUSA M&E system, MBSP has partnered with 533 Producer Organizations; 222 (40%) are in Fatick, 206 (37%) in Kaolack and 129 (23%) in Kaffrine. The Survey of Producer Organizations has found that 72% of the Producer Organizations (POs) accept both male and female farmers, as can be seen in Figure 6. Kaffrine has the highest percentage of mixed POs(78%) as well as women only POs(20%), while close to 25% of the POs in Fatick accept men only. Figure 6 - Producer Organization Staff disaggregated by Gender The Survey of the POsshows that MBSP is working mostly with Economic Interest Groups or Groupements d’intérêt économique (GIE). In Senegal, GIE are groups made up of at least two individuals, are registered by the Chamber of Commerce of Senegal and the Agence nationale pour la promotion des investissements et des grands travaux (APIX) and run non-profit economic activities. In Fatick, the survey found that 7.9% of the POs benefiting from MBSP have yet to be formalized. Table 4 - Producer Organizations by Legal Type Region Association GIE Cooperative/ Coopérative Non-Registered Organization / Organization non￾formalisée Fatick 18% 71% 3% 8% Kaolack13 17% 82% 2% 0% Kaffrine 5% 95% 0% 0% Total 13% 82% 2% 3% 13 The total for Kaolack adds up to 101% due to rounding. 24% 15% 3% 5% 18% 20% 71% 67% 78% Fatick Kaolack Kaffrine Percentage of Organisations of Producers OP men only / homme seulement OP women only / femme seulement OP mixed / mixte 28 NCBA CLUSA has explained that several POs currently beneficiaries of MBSP are in the process of formalizing their organization. Recommendation #1: That MBSP continues to support the Producer Organizations in Fatick in acquiring formal and legal status. 6.3 Processing Units As explained by NCBA CLUSA and confirmed by the Survey of Processors, millet processing is labor intensive and is predominantly performed by women. Processors in Dakar have the highest percentage of male staff who, as can be expected, are needed to lift heavy equipment or millet loads. MBSP is working with the Processing Units to promote efficiency and productivity by teaching improved processing methods, good management and marketing practices promoting, as well as the adoption of safety and hygiene standards. Figure 7 shows the distribution of the processing methods used based on a self-assessment of survey respondents who are the manager or owner of the Processing Units. When asked to describe their processing methods, 55% of Processing Units describe their operation as artisanal. At 70%, Kaffrine had the highest number of Processing Units that describe their methods as artisanal while Kaolack had the lowest, at 24%. Figure 7 - Processing Methods Used by Producers (Self-Assessment) In Kaolack, over 50% of units reported that they are semi-industrial processors using more effective machines that can process millet faster. Fatick had the next-highest level of units reporting semi-industrial processing methods (31%). Modern processing involves some machinery, but still requires manual handling of millet to produce consumption-ready finished products, such as Couscous, Thiaky, Araw, and Semolina. Only 16% of Processing Units reported modern processing methods. The survey data was analyzed to find the differences between each of the three categories. The survey 63% 24% 70% 61% 55% 6% 24% 20% 14% 16% 31% 52% 10% 19% 26% Fatick Kaolack Kaffrine Dakar Total Percentage of Millet Producers Artisanal/ Artisanal Modern/ Moderne Semi-Industrial/ Semi industriel 29 asked the processors to identify the tools and equipment that they own and use for millet processing14 . Although artisanal, modern and semi-industrial processors all reported owning the tools listed by the survey, it was noted that firms who identify themselves as modern or semi-industrial are more likely to own sizers, de-stoning machines, electronic scales and huskers than processors who are artisanal. The average number of persons working at the modern units is 19 compared to 36 in artisanal and semi￾industrial processors, suggesting that modern Processing Units are using less labor-intensive techniques to process millets. Artisanal processing relies on traditional methods and basic hand tools to process the crop into edible flour, making it a laborious and time-consuming task. It should be noted that equipment is obtained by donations from NGOs, hence even when a Processing Unit describes its method as artisanal, it may own tools that are more up to date. It is also worth noting that two-thirds of the Processing Units in Dakar reported that they are using artisanal methods. All Processing Units report facing similar challenges and limitations, including: • Lack of modern equipment, such as threshers and vibro-sieves • Shortage of funds and storage areas to facilitate buying raw millet in bulk and at lower prices • Inability to contract millet purchases because of a lack of funding guarantees • High transportation costs, especially for the units in Dakar, when the raw millet is obtained from producers from rural areas. The program has established a training program specifically for Processing Unit personnel. Table 5 shows the number of persons working in the MBSP Processing Units as estimated by the survey. This Table also includes the number of persons trained, which was obtained from the NCBA CLUSA monitoring system. According to the data gathered and shown in Table 5, 86% of the people working in the beneficiary Processing Units in Kaffrine were trained. This is the highest percentage of trained staff among the regions. Half of the staff in Fatick have been trained whereas 1 out of 4 was trained in Dakar. Kaolack saw the smallest percentage of Processing staff being trained (12.7%). Table 5 also indicates that MBSP is concentrating its efforts on improving millet processing in Dakar, where there are 72 units, 45% of all Processing Units participating in the program. It is not known how many Processing Units existin each region nor the percentage that are being supported by MBSP. NCBA CLUSA has explained that it selected the Processing Units for MBSP after scanning units already operating in each region. To be eligible for the program, the unit had to be a member of an association of millet processors. Non-members were compelled to join an association in their area if they wanted to take part in the program. 14 Question 3.6, Survey of Processing Units. 30 Table 5 - Number and Percentage of Persons Trained in the Processing Units Region MBSP Monitoring System Survey Estimates - Staff of Processing Units Number of Processing Units Number of Men Trained/ Nombre d'hommes formés Number of Women Trained/ Nombre de femmes formées Total Persons Trained / Nombre total de personnes formés Number of Men working in the Processing Units / Nombre d'hommes travaillant dans les unités de transformation Number of Women working in the Processing Units / Nombre de femmes 'hommes travaillant dans les unités de transformation Total Number of persons working in the Processing Units / Nombre total de personnes travaillant dans les unités de transformation Percentage of Persons Trained by MBSP in the Processing Units / Pourcentage de personnes formées par PSME dans les unités de transformation Fatick 19 11 269 280 8 539 547 51.2% 3.93% 96.07% 1.5% 98.5% Kaolack 31 9 89 98 21 752 773 12.7% 9.18% 90.82% 2.7% 97.3% Kaffrine 34 47 383 430 18 479 497 86.5% 10.93% 89.07% 3.6% 96.4% Dakar 72 35 394 429 125 1654 1779 24.1% 8.16% 91.84% 7.0% 93.0% Diourbel 2 5 80 85 N/A N/A N/A N/A Total 158 107 1215 1322 172 3424 3596 N/A 31 The survey found that over 80% of the Processing Units are registered companies. In Senegal, registration in the Registre du commerce et du crédit mobilier (RCCM) or the Trade and Personal Property Credit Register, is a mandatory procedure prior to the exercise of any commercial activity. Kaolack has the highest rate of registration at 96%, as can be seen in Figure 9, which also shows the percentage of firms with a Product Certification, FRA number, and a label. Figure 9 - Legal Status of Processing Units In Senegal, companies can request a certification from l’Association sénégalaise de normalisation, the Senegalese Standards Association, which will certify the firm if its product(s) conforms to standards or technical specifications and specific qualities set up by the Association. According to the survey, 76% of the Processing Units in Kaolack have received such certificate. Figure 7 shows that less than a third of the Processing Units in Fatick, Kaffrine and Dakar have this product certification. The production and marketing authorization, commonly known as the FRA number, authorizes the manufacture, processing and packaging, and sale of all products intended for human or animal consumption in Senegal. This authorization is the responsibility of the Ministry of Commerce and more specifically of the Directorate of Internal Trade through the Division of Consumption and Consumer Safety. Figure 7 above shows that close to half of the Processing Units in Kaolack have an FRA, compared to 37% of Processing Units in Dakar, 19% in Fatick, and only 5% in Kaffrine. Kaolack and Fatick show the highest percentage of Processing Units with labels for their products, 72% and 63% respectively. In Kaffrine, only a quarter of companies have product labels. NCBA CLUSA has explained that since the beginning of the project, it has been supporting businesses in raising the quality of their products to ensure that they are safe for consumption in order to qualify for 88% 31% 19% 63% 96% 76% 48% 72% 85% 25% 5% 25% 81% 26% 37% 26% 0% 20% 40% 60% 80% 100% 120% Registered Business / Commerce enregistré Production Certification / certification des produit FRA Authorisation / Existence d'une autorisation FRA Label / Existence d'un label Percentage of Processing Units Fatick Kaolack Kaffrine Dakar 32 the FRA status. NCBA CLUSA did not provide the number of companies that have received an FRA number due to the support of MBSP. Recommendation #2: That NCBA CLUSA continues to promote company registration, product certification, and labelling as well as supporting Processing Units to meet the quality standards necessary for obtaining the FRA number, especially in Fatick, Kaffrine, and Dakar. The survey data shows that MBSP is mostly supporting Processing Units that have been in operation for six years or more: 44% of the MBSP Processing Units have been in operation for more than ten years and 23% for 6 to 10 years. The Program also supports newer Processing Units; 15% of the beneficiaries have been operating for less than 3 years, as shown in Figure 10. The program is supporting both mature and newer Processing Units. Figure 10 - Number of Years in Operation 6.4 Other Beneficiaries In addition to the Producers, Processing Units, and POs, MBSP also trains and supports APS and Extension Workers. The data shown in Table 6 was obtained from the NCBA CLUSA M&E System. The project had not started work with APS fertilizers by the start of this evaluation. They are thus not included in this study. 19% 8% 10% 19% 15% 19% 12% 30% 16% 18% 38% 24% 15% 21% 23% 25% 56% 45% 44% 44% 0% 10% 20% 30% 40% 50% 60% Fatick Kaolack Kaffrine Dakar Total Percentage of Processing Units Less than 3 years / moins de 3 ans 3 to 5 years / 3 à 5 ans 6 to 10 years / 6 à 10 ans More than 10 years / plus de 10 ans 33 Table 6 - Other MBPS Beneficiaries Region APS Seed Providers / Semenciers APS Artisans APS Packers / Emballage Agricultural Extension Workers / Vulgarisateurs Fatick 10 10 2 31 Kaolack 13 6 19 Kaffrine 7 1 11 Dakar 1 Total 30 17 3 61 7. Activities of the Millet Producers The Evaluation investigated the agricultural activities of the MBSP Millet Producers. In 2017, according to the Survey of Producers, cash crops and subsistence farming were the Millet Producers’ two main activities, as shown in Figure 11. Given the low rainfall in the intervention areas, Millet Producers barely practiced irrigation and flood cultivation methods. Survey respondents reported doing some gardening and fruit growing. Figure 11 - Type of Agricultural Activities in 2017 When questioned about the crops that they grow, 97% of the MBSP Millet Producers responded that they also produced peanuts in 2017, almost equal to those cultivating millet (98%), as shown in Figure 12. This is not surprising, given that the intervention areas are in Senegal’s Central Peanut Basin. However, this data indicates that millet and peanut are competing crops in the agricultural activities of the MBSP beneficiaries. Corn is the third main crop, grown by 59% of the Millet Producers. 1 out of 5 millet producers grow watermelon, a very popular fruit in the country. Rice and sorghum are grown by roughly 10% of the producers. The Survey of Producers found that 2% of the MBSP Beneficiaries did not grow millet last year. 36% 35% 8% 8% 10% 2% 1% 41% 39% 6% 6% 5% 2% 1% 38% 37% 7% 13% 4% 1% 0% Subsistence Crop / Culture vivrière Cash crop / Culture de rente Gardening (cold season) / Maraîchage de contre saison (en saison froide) Gardening (rainy season) / Maraîchage d’hivernage Fruit growing / Arboriculture Irrigated crop / Culture irriguée Flood crop / Culture de décrue Type of Cultivation Fatick Kaolack Kaffrine 34 Figure 12 - Crops Cultivated by Millet Producers in 2017 8. Millet Production The Evaluation found that the MBSP Millet Producers have increased the area that they farm by 8% from 2015 to 2018. The area under millet cultivation has increased by 5% during this time. Figure 13 shows that land farmed exclusively by women experienced similar trends, with a 12% increase in the number of hectares farmed and an 18% increase in millet cultivation. This increase did not change the percentage of farmland under millet cultivation, which remained steady at approximately 41% of total farmland. Figure 13 - Trend in the Cultivation of Millet by the MBSP Beneficiaries 98% 59% 11% 8% 97% 20% 32% 14% 18% Millet / Mil Corn / Mais Sorghom / Sorgho Rice / Riz Peanuts / Arachide Watermelon / Pasteque Vegetables / Légume Arboriculture Other / Autre culture Percentage of Millet Producers Crops Cultivated 0 20000 40000 60000 80000 100000 120000 Total Area Farmed Total Land under Millet Cultivation Area farmed by exclusively by women Area cultivated exclusively by women with Millet Hectares 2015 2016 2017 35 The Evaluation also studied the participation of the producers in millet trading. The survey shows that since 2015 there are more producers selling their millet harvest. As can be seen in Figure 14, a total of 33.5% of farmers were selling their harvest in 2017, compared to 26.7% in 2015. Kaolack and Kaffrine showed the largest increases. Figure 14 – Percentage of Millet Producers who sold all of their millet harvest It is worth noting that when asked why they did not sell their entire stock of millet, 90% of the producers indicated that they needed the millet for their own consumption, as shown in Figure 15. The survey data suggests that the farmers are not producing enough millet to be able to sell. Finding buyers and low prices are not issues faced by the producers, as shown in the graph below. Figure 15 - Reasons for Not Selling Millet Harvest Furthermore, the producers report that they have more business in 2017 compared to 2015, and 75% of producers say their millet sales revenue has increased. In Kaolack, the results are better, with 89% of 15.3% 15.9% 18.8% 35.4% 33.1% 46.5% 38.0% 33.9% 44.4% 26.9% 25.5% 33.5% 0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% 45.0% 50.0% 2015 2016 2017 Fatick Kaolack Kaffrine Total 88.9% 0.2% 1.6% 3.2% 0.9% 1.2% 87.6% 0.8% 1.0% 4.3% 3.3% 0.4% 89.1% 0.3% 1.9% 1.4% 0.2% 0.2% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Production for own consumption / La production pour autoconsommation No buyer/ Pas d'acheteur Bad Quality Millet/Du mil de mauvaise qualité Price too low/Le prix d'achat est faible Market Overflow/ Marché Inondé Points of sale too far /Lieux de vente très éloignés Reasons for no millet sale in 2015 / Raisons de mévente en 2015 Reasons for no millet sale in 2016 /Raisons de mévente en 2016 Reasons for no millet sale in 2017 /Raisons de mévente en 2017 36 producers reporting higher millet revenue, as shown in Figure 16. Over half of the producers are finding it easier to find clients and sell their millet. Producers in Kaolack appear to be doing better than those in Fatick and Kaffrine. 73% of producers in Kaolack reported that more processors and brokers are interested in buying from them. Figure 16 - Changes in Millet Marketing If the farmers are not cultivating millet on more land, the amount of millet produced can still increase if the farming methods are improving yield per hectare. The Evaluation has not used the data the survey collected, given that 2016 was a bad year for millet due to excess rain and the data for 2017 was found to be insufficiently reliable. Instead, the Evaluation has reviewed secondary data which compare the yield per hectares of the NCBA CLUSA demonstration plots with traditional millet production15. This information was found in reports produced by DRDR, which was contracted by NCBA CLUSA to evaluate the yield of 89 demonstration plots in Fatick, 75 in Kaolack and 62 in Kaffrine. The study also studied the yield of traditional cultivation using 89 control plots in Fatick, 47 in Kaolack and 32 in Kaffrine. The Evaluation has calculated the average yield of millet in kilograms per hectare for each region and compared the results. It has assumed that the selected Demonstration and Control Plots are representative samples of the two different millet cultivation methods in each region. The results are shown in Table 7. Table 7 - Comparison Millet Yield between Demonstration Plots and Traditional Cultivation Kilos/Hectare Demonstration Plot Traditional Cultivation Difference in Yield in kilo per hectare Fatick 964 724 240 Kaolack 1619 1431 188 Kaffrine 902 759 143 In Kaolack, the yield from traditional cultivation is much higher than in Fatick and Kaffrine. The Demonstration Plots appear to be more successful at increasing the yield per hectare in Fatick, providing an increase of 240 kilos per hectare compared to 188 kilos in Kaolack and 143 kilos in Kaffrine. 15 Rapport sur la pesée des carrés de rendement région de Fatick, Direction Régionale du Développement Rural (DRDR) Rapport sur la pesée des carrés de rendement région de Fatick, Livrable 1 : Rapport à mi-parcours, Direction Régionale du Développement Rural (DRDR/KAOLACK), Bon de Commande Numéro. 11/09/17 DRDR-KL-PSEM-4 Date d’effet : 11/09/2017 Rapport de carrés de rendement DRDR Kaffrine VF 13012018, Rapport de carrés de rendement DRDR Kaffrine VF 13012018, EXCEL FILE 46% 48% 38% 73% 72% 73% 75% 89% 48% 52% 39% 54% 56% 61% 48% 75% Easier to find buyers / Plus facile de trouver des acheteurs More clients / Plus de clients More Processors and brokers more interested in buying your millet / Plus de transformateurs et courtiers interresés à acheter votre mil Increased Revenue from sale of Millet / Hausse de Revenu provenant du mil Fatick Kaolack Kaffrine Total 37 The findings of the DRDR’s study confirms that the techniques taught by MBSP are superior to the traditional methods. Although there is no evidence that the farmers are obtaining the same results as the demonstration plots, the DRDR study supports the belief of producers that the millet quality and yield have improved since 2015. Figure 17 shows that producers in all three regions think that more farmers are growing millet and their crop is of better quality than in the past. A higher percentage of respondents in Kaolack are of this opinion. Figure 27 - Improvement in Millet Production Recommendation #3: That NCBA CLUSA investigates why the yield per hectare varies between regions. The Evaluation finds that the MBSP Beneficiaries are more positive about their future and they also feel more confident about their farming abilities. Over 95% of MBSP Producers were more confident about their agricultural competencies and the future of their farms compared to three years ago, as shown in Figure 18. When asked about their preparedness for drought, a common threat in this part of Senegal, 82% of producers in Kaolack reported that they are more prepared than in the past, a higher percentage than in Fatick or Kaffrine (74% and 71% respectively). Figure 18 – Millet Farmers’ perception about future of their businesses 81.0% 82.7% 87.4% 87.4% 80.1% 81.9% More Growers Better Quality More Growers Better Quality More Growers Better Quality Fatick Kaolack Kaffrine Percentage of Millet Producers 97% 97% 97% 98% 94% 95% 96% 97% 71% 82% 74% 75% Fatick Kaolack Kaffrine Total Percentage of Millet Producers More confident in agricultural competency / Plus confiant dans vos compétences en agriculture More confident in the future of the farm / Plus confiance dans l'avenir de votre ferme Better prepared in case of drought / Mieux préparé dans le cas d'une secheresse 38 8.1 Input Providers - Seed When asked which kind of millet varieties they were cultivating, survey respondents from all three regions indicated Souna 3 as their favorite. Souna 3 is the variety of millet that the project is promoting. Figure 19 shows that in Kaffrine only 79% of the MBSP Producers grow Souna 3, compared to a much higher level in Fatick and Kaolack, 95% and 96% of farmers respectively. 53% of farmers in Kaffrine also grow Thialack 2, which is only grown by 16% of the farmers in Fatick and 30% in Kaolack. Millet Producers agree that Souna 3 is a high-yield variety with a short cultivation cycle. Both the APS Semenciers and the Millet Producers agree that the pre-flowering techniques taught by NCBA CLUSA bring higher yield and better millet quality. The Millet Producers indicated that Institut Sénégalais pour les recherches agricoles (ISRA) also produces high quality seeds. Figure 19 - Variety of Millet Grown As previously explained, MBSP has trained APS Semenciers in millet seed production. The Evaluation investigated the popularity of the APS as input supplier among the producers. Figure 20 presents the findings of the survey on seed supply to the producers in 2017. Over 70% of producers save the millet that they produce for seeding; in fact, this was one of the reasons given when producers were asked why they had not sold their harvest. Although they believe that the APS Semenciers sell good seeds, Millet Producers report that they are not buying from them; this was also the finding of ANCAR16. The producers have explained during the qualitative interviews that they cannot afford to buy the seeds. In turn, the seed producers complained that they cannot find clients. The survey found that in Kaolack Millet Producers are buying seeds in higher numbers from authorized seed resellers (23%); they also buy at local markets (11%). In Fatick, 25% of producers report buying from a contractor and 6% from an 16 Accompagnement technique des producteurs dans la confirmation de la mise en œuvre des technologies agricoles de production du mil à Fatick, Livrable III : FATICK, ANCAR 2017. Accompagnement technique des producteurs dans la confirmation de la mise en œuvre des technologies agricoles de production du mil à Kaolack, Livrable III : Kaolack, ANCAR 2017. Accompagnement technique des producteurs dans la confirmation de la mise en œuvre des technologies agricoles de production du mil à Kaffrine, Livrable III : Kaffrine, ANCAR 2017. 96% 16% 0% 6% 6% 4% 95% 30% 2% 0% 2% 2% 79% 53% 2% 2% 2% 7% Souna 3 Thialack 2 ICMV IS 89305 / ICMV IS 89305 GAWANE / GAWANE Unknown varieties / Variétés inconnues Other varieties / Autres variétés Percentage of Millet Producers Fatick Kaolack Kaffrine 39 authorized seller. It is important to note that in Fatick and Kaffrine 25% of the Millet Producers report receiving seeds from NGOs. These donations are in direct competition to the APS Semencier and reduce producers’ needs to buy seeds from a supplier. Figure 20 – Millet Seeds Supply in 2017 Given that the producers cannot afford to buy seeds, the Evaluation suggests that producers be given further training in the production and selection of high-quality seeds for seeding in the following year. When examining the data regarding seed production training, the Evaluation found that only 88% of the beneficiaries have been trained in the identification of good quality seed. An almost equal percentage claim that they can now identify good seeds. In Kaolack 93.5% of the farmers reported being able to identify good millet, compared to 79.7% in Fatick and 90.75% in Kaffrine, as shown in Figure 21. Figure 21 - Identification of Quality Millet Seeds 78% 26% 4% 6% 4% 22% 0% 2% 70% 9% 11% 23% 7% 7% 5% 0% 71% 2% 2% 12% 5% 24% 0% 5% Own seeds / Propre semences Contractor or broker / Contracteur ou courtier Local market / Marché local Louma Purchased from authorised seller / Acheté auprès d'un vendeur agréé Donation from government or Research Centre / Don du gouvernement ou Centre de Recherche Donation from NGO / Don d'un ONG Donation from friends or family / Don des amis ou famille Other / Autres Percentage of Millet Producers Kaffrine Kaolack Fatick 85.0% 92.7% 87.9% 87.9% 79.7% 93.5% 90.7% 86.3% Fatick Kaolack Kaffrine Total Percentage of Millet Producers Training - How to identify good millet seeds / Formation -Comment distinguer de bonnes semences de mil Can identify good seeds / Peut identifier des bonnes semences 40 Recommendation #4: That NCBA CLUSA reinforce its training for Millet Producers on producing and identifying high quality seeds given that farmers are using the seeds that they produce for seeding the following year. Recommendation #5: That NCBA CLUSA and CRS support the Millet Producers to find credit and/or to save to buy seeds from the APS Semenciers. Recommendation #6: That NCBA CLUSA assist the APS Semenciers in negotiating contracts with the Producer Organizations and further build their skills in marketing their products. The APS Semenciers indicated that the training received from NCBA CLUSA has been very beneficial; they are now only producing good quality seeds for sale and for their own consumption. They are also using the improved storing methods taught by NCBA CLUSA that keep their products clean and away from rodents. 8.2 Input Providers - Equipment As noted above, NCBA CLUSA has designed a ripper that the APS Artisans are now making (see Exhibit 1). The Evaluation found that the millet producers are quite impressed by the ripper. It has learned that ISRA has also found the ripper a good tool and has purchased several from the MBSP APS Artisans. However, Millet Producers are not purchasing this tool in large quantities, explaining that they do not have the money. In fact, the APS artisans complained that they have not sold as many as they were expecting. They further explained that they are only making rippers when they receive an order, since they also do not have the money to buy material to build the rippers and keep them in stock. During the qualitative interviews, the Evaluation found that there is an expectation on the part of the farmers that the rippers should be given to them for free by the program. Meantime, the MBSP has a grant and loan initiative for input acquisition including equipment which requires that the applicant funds part of the cost. Recommendation #7: That NCBA CLUSA and CRS investigate how Millet Producers, the PO, and the Village Council can access funds through financial institutions and savings to purchase rippers, making this tool a priority piece of equipment in millet cultivation. The beneficiaries should be further encouraged to apply for the MBSP grants for the acquisition of equipment. 41 Exhibit 1. Ripper Designed by MBSP 8.3 Pre-Flowering Techniques The Evaluation has investigated the percentage of beneficiaries trained and now putting in practice the various Pre-Flowering techniques taught in the program. The Survey of Producers shows that over 85% of the beneficiaries have been trained in choosing plots of land that are adequate for millet cultivation. Over 80% of the trained beneficiaries report that they now know how to choose a plot; this percentage is lowest in Fatick, where only 71% say that they are comfortable making this choice, as shown in Figure 22. Figure 22 - Selecting a Plot for Millet Cultivation The survey found that over 90% of the Millet Producers have been trained in soil preparation. Figure 23 shows the percentage of Millet Producers practicing Hoe Sine crossover at least 6-7cm deep, spreading organic manure,raking, grubbing, and clearing as well as hoeing. These are recommended soil preparation techniques taught by the program. The data shows that in Fatick and Kaffrine, 99% and 96% of the farmers are grubbing and clearing land, compared to only 76% in Kaolack. This may well indicate that farmers are preparing new plots of land for farming in Fatick and Kaffrine. Raking is practiced by almost everyone, but spreading organic manure is done by more farmers in Kaolack (84%) than in Fatick (79%) or Kaffrine (63%). 85% 92% 87% 87% 71% 89% 88% 80% 0% 20% 40% 60% 80% 100% Fatick Kaolack Kaffrine Total Percentage of Millet Producers Choosing the plot - Training Received / Choisir la parcelle - Formation reçue Knows how to choose a plot / Sait comment choisir la Parcelle 42 Figure 23 - Soil Preparation Techniques used by MBSP Millet Producers Although most producers practice first and second hoeing, fewer practice third hoeing. This may indicate that the farmers consider that their millet plants are doing well enough and that they feel a third hoeing is unnecessary. The percentage of farmers practicing a third hoeing is lower in Kaolack compared to Fatick and Kaffrine, as shown in Figure 24. Since third hoeing is required in the Conservation Agriculture that MBSP is promoting, it is therefore necessary that the Program insists on the practice of third hoeing. Figure 24 - Practice of hoeing The survey indicates that the practice of seeding methods is not uniform across the three regions, as shown in Figure 25. In Kaffrine, more are doing wet sowing (96%) compared to dry sowing (17%) whereas in Fatick the reverse is done. In Kaolack, farmers are using the techniques equally. In Kaolack, spacing the lines and pockets at .9 meters is practiced by only 58%, a practice more common in Fatick (85%) and Kaffrine (74%). On the other hand, the use of super eco drills with 4 holes for burying is more common in Kaolack. 99% 97% 79% 74% 79% 98% 84% 71% 96% 98% 63% 70% Grubbing and clearing / Dessouchage et défrichage Raking / Ratissage Spreading of organic manure / Epandage de 2T/ha de fumure organique (Compost, FPR) Hoe Sine crossover at least 6-7cm deep / Passage croisé à la houe sine sur au moins 6 à 7 cm de profondeur ou avec une charrue à soc réversible Kaffrine Kaolack Fatick 99% 99% 79% 100% 98% 67% 99% 100% 78% First hoeing / Premier sarclobinage (bakhaaw) Second hoeing / Deuxième sarclobinage (beyatt) Third hoeing / Troisième sarclobinage (balargny) Kaffrine Kaolack Fatick 43 Figure 25 - Practices for Sowing Recommendation #8: That NCBA CLUSA and the Agricultural Extension Agents investigate why certain practices are not being followed; that retraining and/or support is provided to farmers where necessary. The Millet Producers and the POs assert that the quality of millet produced has improved since 2015. During the qualitative interviews, they attribute the improved quality to the application of the techniques learnt from MBSP. In addition, the Producers reported that they are now producing enough millet to meet the needs of their families. Their next goal is to produce greater quantities to sell for revenues. 9. Effectiveness of the MBSP Training Program The program has an important training component and, as mentioned before, NCBA CLUSA has partnered with ANCAR and DRDR to deliver the training. NCBA CLUSA Coaches are responsible for organizing and monitoring the training sessions, which are given through the POs or at the village. This evaluation has investigated the effectiveness of the training program by first identifying the percentage beneficiaries who have been trained and, of those trained, how many are applying the methods that they have been taught. The Survey found that the overall training participation rate is 62.5%, as shown in Table 8, which also shows the rate by region. Table 8 - Participation Rate in Training by Region Region Participated in Training / Participé à une formation du projet Yes / Oui No / Non Total Fatick 60.8% 39.2% 100.0% Kaolack 61.0% 39.0% 100.0% Kaffrine 69.1% 30.9% 100.0% Total 62.5% 37.5% 100.0% 86% 39% 85% 88% 58% 17% 96% 74% 71% 69.2% 64.9% 73.5% 82.2% Dry sowing / Semer à sec Wet sowing / Semer en humide Spacing of .9 m between lines and pockets / Ecartement de 0,90 m entre les lignes et les poquets Super eco-drill with 4 holes burying / Semoir super-éco muni de disque de 4 trous Total Kaffrine Kaolack Fatick 44 9.1 Satisfaction of Participants with the Training Program During the Focus Group discussions, Producers and APS agree that the training was very useful and that they will recommend it to others. In fact, the Survey of Producers finds that all producers rate the training and the trainers quite highly. Figure 26 shows how the Millet producers rated the Extension Agents, ANCAR and DRDR agents as well as the NCBA CLUSA staff. They found that the trainers were pleasant and encouraged questions from the participants. They also found that the trainers knew the material very well, but they were not always on time for class. Figure 26 - Rating of Trainers by Producers The Evaluation Team was informed that the Extension Agents are Millet Producers and PO members. According to the Extension Agents, they were selected to train and coach producers because of the enthusiasm they showed during MBSP training. They noted that they are well regarded and respected among villagers and in the community. They receive a daily fee of 10,000 CFA from the program for days when they teach. However, they wish that their position would become a paid job within the PO, since it is not easy to make a living while providing training on an ad hoc basis. They further explained that transportation is their biggest challenge, as it is not easy to travel around and reach the villages and producers. They suggested that the Program provide them a motorcycle. Recommendation #9: That NCBA CLUSA reviews the role of the Extension Agents and investigate how to sustain their position within the PO to ensure that they can continue to support and coach the Millet Producers when the Program ends. 9.2 Soil Preparation Figure 27 shows the percentage of Millet Producers trained in soil preparation and fertilization techniques. The survey findings are positive with close to 90% of farmers receiving training and over 80% of those trained are using methods learnt. The exception is composting which is the least practiced methods. During the qualitative interviews, producers explained that they have difficulties in finding primary material for their compost bins. The Millet Producers also complained of being unable to purchase fertilizers because of the high cost. 100% 99% 99% 91% 100% 100% 99% 83% 99% 100% 99% 85% Trainer knew the material very well / Formateur connaisait très bien la matière Trainer was pleasant / Formateur était aimable Trainer encouraged questions / Formateur encourageait les questions Trainer on time for classes / Formateur l'heure pour les classes Extension Agents / Vulgarisateurs ANCAR or/ou DRDR CLUSA staff / Personnel CLUSA 45 Figure 27 - Percentage of Millet Producers Trained in Fertilization Techniques 85% 71% 93% 87% 95% 68% 93% 86% 93% 86% 92% 89% 95% 91% 91% 78% 96% 90% 96% 90% 87% 88% 92% 92% 89% 65% 92% 80% 92% 80% 87% 80% 93% 89% 92% 70% 93% 86% 93% 86% Training / Formation Practice / Pratique Training / Formation Practice / Pratique Training / Formation Practice / Pratique Training / Formation Practice / Pratique Training / Formation Practice / Pratique Choosing the plot / Choisir la parcelle Preparing the soil / Préparation du sol Composting techniques / Techniques de compostage Use of fertiliser / L'utilisation des engrais Use of fertiliser and organic manure / Utiliser les engrais et fumure organique Fatick Kaolack Kaffrine Total 46 Exhibit 2. Composting in Médina Sabakh, Kaolack Recommendation #10: That NCBA CLUSA reinforces its training in composting, encouraging and supporting the use of compost to fertilize the soil. 9.3 Seeding Figure 28 shows the percentage of producers trained and practicing seeding techniques. It is interesting to note that Kaolack has the highest training and practice rates in each category. Figure 28 - Percentage of Millet Producers Trained in and Practicing Seeding Techniques 9.4 Crop Monitoring and Assisted Natural Regeneration The survey data shows that the percentage of producers who have received training in Crop Monitoring, shown in Figure 29, is lower than the percentage of those who were trained in soil preparation, fertilization and seeding techniques: 86% compared to 96%. With regards to performing Assisted Natural Regeneration (ANR), roughly 70% of the producers have been given training and only 60% of those trained are performing ANR. 85% 80% 93% 88% 90% 86% 93% 93% 98% 91% 92% 87% 88% 91% 93% 84% 90% 83% 88% 86% 94% 88% 91% 85% Training / Formation Practice / Pratique Training / Formation Practice / Pratique Training / Formation Practice / Pratique How to identify good millet seeds / Comment distinguer de bonnes semences de mil Seeding techniques (sowing period, spacing, tools and materials used) / Les techniques des semis (période de semis, écartement, outils et matériels à utilisés) Crop maintenanc e techniques / Les techniques d'entretien des cultures Fatick Kaolack Kaffrine Total 47 Figure 29 - Percentage of Millet Producers Trained in and Practicing Crop Monitoring and Assisted Natural Regeneration (ANR) The Evaluation was provided with the reports of ANCAR on the training that it was contracted to provide to millet producers by NCBA CLUSA in Fatick, Kaolack and Kaffrine17 . Producers were trained on millet cultivation, pest and disease management, soil fertility management and mastering climate change adaptation concepts. ANCAR also found that the training sessions were well attended, but that in all regions the technology adoption rates are somewhat low. Among the many topics proposed by the agricultural extension, only a small number are adopted. ANCAR found that Producers are more reluctant to move away from traditional practices, even if they are convinced of the positive contribution of the recommended methods. ANCAR has recommended that the training sessions take place before wintering and to involve the PO in the choice of training sites. Recommendation #11: That NCBA CLUSA works with its partners and the POs to reinforce knowledge and promote adoption of methods taught with reminders and refreshersessions prior to the wintering season. The Evaluation also found that the Producers are not always applying new techniques even if they believe in the new technology. The reason given by the farmers for not practicing the new techniques was lack of funds. 9.5 Harvesting and Storing 17 Accompagnement technique des producteurs dans la confirmation de la mise en œuvre des technologies agricoles de production du mil à Fatick, Livrable III : FATICK, ANCAR 2017. Accompagnement technique des producteurs dans la confirmation de la mise en œuvre des technologies agricoles de production du mil à Kaolack, Livrable III : Kaolack, ANCAR 2017. Accompagnement technique des producteurs dans la confirmation de la mise en œuvre des technologies agricoles de production du mil à Kaffrine, Livrable III : Kaffrine, ANCAR 2017. 86% 79% 84% 74% 72% 58% 88% 86% 88% 82% 68% 57% 83% 84% 86% 79% 70% 66% 86% 82% 86% 78% 71% 60% Training / Formation Practice / Pratique Training / Formation Practice / Pratique Training / Formation Practice / Pratique How to recognise millet diseases / Comment reconnaitre les maladies du mil How to fight millet pests / Comment lutter contre les ravageurs du mil How to perform Assisted Natural Regeneration (ANR) / Comment réaliser une régénération naturelle assistée (RNA) Fatick Kaolack Kaffrine Total 48 With regards to harvesting and storing methods, the evaluation also finds that the rate of persons who say that they receive training is lower than for soil preparation. The survey results are shown in Figure 30. Over 80% of producers were trained to identify the maturity of the crop and how to harvest, dry, and store millet, while only 75% were trained in threshing and calculating their plot’s yield. Recommendation #12: That NCBA CLUSA verifies and ensures that the different components of training are being provided uniformly to the beneficiaries. Figure 303 - Percentage of Millet Producers Trained In and Practicing Harvesting Techniques Since millet farmers are often illiterate and have low levels of education, calculating plot yield is likely to be challenging for the beneficiaries; this is reflected in the survey data. 9.6 Business and Marketing The MBSP focuses on training its beneficiaries in business and marketing techniques to support the trade of millet. The Program also stresses the importance of the POs in the millet value chain. The survey data shows that roughly 70% of beneficiaries have received training in these areas (Figure 31). According to the survey data, only 40% of famers are practicing the sales and marketing techniques they learned. In Kaolack, this number is highest, with 60% of the producers practicing the techniques. 84% 87% 84% 84% 80% 78% 76% 73% 84% 83% 73% 50% 94% 89% 92% 91% 83% 79% 78% 76% 86% 85% 81% 76% 88% 90% 86% 85% 80% 82% 79% 76% 92% 91% 68% 53% 88% 88% 86% 86% 81% 79% 77% 74% 86% 85% 74% 58% Training / Formation Practice / Pratique Training / Formation Practice / Pratique Training / Formation Practice / Pratique Training / Formation Practice / Pratique Training / Formation Practice / Pratique Training / Formation Practice / Pratique How to recognise millet for harvest / Comment reconnaitre un mil mature pour la récolte How to harvest millet / Comment procéder pour faire la récolte du mil How to dry millet / Comment procéder pour le séchage du mil How to thresh millet / Comment procéder pour le battage du mil How to store millet harvest / Comment stocker les recoltes de mil How to calculate plot yields / Comment calculer les carrés de rendement Fatick Kaolack Kaffrine Total 49 Recommendation #13: That NCBA CLUSA reinforcesits training in sales and marketing, especially in Fatick and Kaffrine. As in the case of calculating plot yield, only 60% of the surveyed farmers are calculating production costs, a process which requires numeracy. The survey shows that 76% of farmers in Kaolack calculate production costs, compared to 52% and 56% in Fatick and Kaffrine, respectively (see Figure 31). Trainers and the producers noted that the beneficiaries do not have the materials necessary to perform basic book￾keeping. Figure 31 - Percentage of Millet Producers Trained In and Practicing Marketing Techniques 9.7 Challenges During focus group discussions regarding the challenges they faced when undergoing training, Millet Producers and trainers identified literacy and numeracy as major barriers to understanding the training material. The survey data shows that 22% of the MBSP beneficiaries are illiterate. Kaffrine has the highest level of illiteracy, at 29% of beneficiaries, as shown in Figure 32. In Kaffrine, 22% of the producers also found the training material too hard, compared to 13% in the two other regions. The trainers noted that women rarely take notes. Trainerstell us that farmer field schools and learning from peers (e.g. visitsfrom an experienced producer) will help to support the learning process of the illiterate millet producers. 66% 35% 69% 52% 74% 66% 70% 60% 80% 76% 64% 57% 78% 53% 73% 56% 62% 45% 70% 46% 73% 60% 68% 58% Training / Formation Practice / Pratique Training / Formation Practice / Pratique Training / Formation Practice / Pratique Sales and marketing techniques / Techniques de vente et de marketing Calculating production costs / Calcul des couts de production Importance of Producer Organisations in Value Chain / Importance des Organisations de Producteur dans la Chaine de Valeur Fatick Kaolack Kaffrine Total 50 Figure 32 - Challenges and Difficulties with Training Recommendation #14: That NCBA CLUSA finds alternative methods to teach illiterate farmers and facilitate their learning. It must be noted that 42% of the producers found that the material was too easy. Kaolack had the highest level of producers reporting material was too easy, at 48%. Recommendation #15: That NCBA CLUSA customizes its training methods according to the beneficiaries’ literacy level, agricultural experience and knowledge of millet cultivation. The survey found that producers do not understand the value chain well. As seen in Figure 32 above, 37% of producers mentioned that they do not understand the value chain. Recommendation #16: That NCBA CLUSA emphasizes the value chain in its training program and use case studies and hands-on methods to teach business concepts and marketing and to explain better the role of each actor along the value chain. Training session timing was raised and discussed as another challenge faced. The survey found that 14% of all producers experienced delays in their work due to attending training. In fact, 37% of producers said the training sessions were too long, while only 16% thought that they were too short (Figure 33). The ANCAR trainers found that NCBA CLUSA staff were often too busy to accommodate their time and the beneficiaries’ availability. They recommend avoiding scheduling any training around important religious celebrations. 22% 13% 39% 41% 4.3% 8.2% 15% 13% 48% 32% 9.8% 17.1% 29% 22% 40% 35% 7.5% 8.4% 22% 15% 42% 37% 6.6% 10.8% Illiterate / Analphabète Training material too hard / Materiel de formation trop Difficile Training material too easy / Materiel de formation trop facile Does not understand the Value Chain / Ne pas comprend pas la chaine de valeur Classes were boring / Classes ennuyantes Did not learn anything new / Rien n'appris de nouveau Total Kaffrine Kaolack Fatick 51 Nearly 20% of farmers found that trainings were held too far away from their home. Producers in Kaolack had the highest level of complaints regarding the distance required to get to class. Figure 334 - Other Challenges with Training Recommendation #17: That NCBA CLUSA reviews the training material to make sessions more concise and practical; that training is scheduled to be least disruptive to trainers and beneficiaries. 10. Importance of Membership in an Organization of Millet Producers The Survey of Producers shows that 68% of producers have received training on the POs’ importance in the millet sector. However, the survey finds that only 41% of the producers are members of a PO. This contradicts the beneficiary selection criteria of NCBA CLUSA, which is to select Millet Producers who are registered members of an PO. Figure 34 shows the percentage of Millet Producers who are registered PO members as well as the length of time that they have been members. Fatick shows the highest percentage of membership, at 46%, and Kaffrine the lowest with 19%. Fatick also shows the highest rate of farmers who have been members for over five years. In Kaffrine, 58% of the producers have joined an PO association in the last 3 years or since they have joined MBSP. Close to a third of the farmers in Fatick and Kaolack have registered in the last 3 years. The Evaluation finds that the program is encouraging the farmers to become members. 10% 20% 16% 14% 14% 28% 21% 19% 36% 34% 40% 37% 14% 20% 16% 16% Fatick Kaolack Kaffrine Total Percentage of Millet Producers Training delayed my work / Formation retardait mes travaux Training was given far away / Grande distance à faire pour aller à cette formation Training sessions too long / Durée de la formation trop courte Training sessions too short / Durée de la formation trop longue 52 Figure 34 - Membership Rate and Period in a Producer Organization Millet Producers agree that the PO can provide mutual support during planting and harvesting and find better prices for both inputs and their millet output. As can be seen in Figure 35, a high number of producers think favorably of the PO leaders and believe that they can lobby the government, brokers, and NGOs. Figure 35 - Percentage of Millet Producers Who Find that the PO Can Bring Them Benefits Recommendation #18: That NCBA CLUSA verifies the membership status of its beneficiaries in the PO that it is supporting. 46% 37% 37% 41% 12% 5% 11% 10% 30% 33% 58% 37% 15% 34% 13% 20% 43% 28% 19% 34% Fatick Kaolack Kaffrine Total Percentage of Millet Producers Member of an OP Less than a year / Moins d'un an 1 -3 years / ans 3 - 5 years / ans > 5 years / mois 89% 89% 90% 89% 87% 84% 83% 93% 96% 93% 87% 89% 84% 77% 87% 81% 88% 86% 89% 89% 89% 88% 84% 94% 91% 90% 89% 89% 87% 83% 85% 89% Percentage of Millet Producers Fatick Kaolack Kaffrine Total 53 11. Quality of Processed Millet One of the MBSP’s objectives is to expand millet trade by promoting high-quality millet and introducing food safety measures in millet processing. The Evaluation examined the training program set in place for Processing Unit staff focused on hygienic practices and proper handling and packaging of raw and processed millet. 11.1 Hygiene Respondents to the Survey of the Processing Units found that after training, unit cleanliness, equipment maintenance, and respect of quality, hygiene norms and standards all improved. However, this census found that the percentage of units that have adopted the measures that are being promoted by the MBSP lies at 45%, a number that is not as high as could have been expected, as can be seen in Figure 36. It is possible that the importance of hygiene is not well understood among Processing Unit staff. During visits at the Processing Units, the Evaluation Team noted that sacks of millet were left open after-hours. At one site, there were rodents roaming the premises. Nets were not properly secured over the drying tray; it was also observed that weighing and packaging of finished millet products were being done close to the floor rather than on a table. The Evaluation Team was also told by the processors that wearing gloves is uncomfortable and slows their work. Figure 36 - Change in Hygiene Practices 15% 14% 15% 14% 14% 13% 24% 24% 25% 24% 31% 27% 18% 18% 20% 19% 23% 20% 43% 43% 40% 44% 31% 40% Improved cleanliness in processing unit / Plus de Propreté dans l'unité Higher respect for hygiene standards / normes d'hygiène mieux repectés (port de masque, gants etc) Improvement in maintenance of equipment and material / Equipement et du matériel mieux entretenus Higher respect for quality standards / Normes de qualité mieux respectés Improvement in maintenance of drying equipment / Equipements de séchage mieux entretenus Improved Personal hygiene / Amelioration dans l'Hygiène Personnel Percentage of Processing Units Dakar Kaffrine Kaolack Fatick 54 Recommendation #19: That NCBA CLUSA continues to emphasize hygienic standards and draw attention to gaps and deficiencies during site visits. 11.2 Packaging The Program is encouraging processors to have a company label and is promoting bag fabrication using Kraft and aluminum paper, which are environmentally friendly compared to plastic. Processors find designing their label challenging and the cost of printed plastic bags high. Informants noted that paper bags are popular with customers who are looking to export or send the product abroad. However, Processing Units report that Kraft paper is expensive and supply is limited. Furthermore, making the bags is time consuming. The survey found that staff have been trained to use Kraft paper for packaging, as shown in Figure 37; however, the Processing Units are not necessarily using it. Figure 37 - Training in Packaging of Finished Millet Products One Processing Unit showed the team plastic bags obtained from NCBA CLUSA but bearing another company’s label. The processor mentioned that this was a waste because they did not intend to package their products in these bags. Recommendation #20: That NCBA CLUSA reviews the relevance of its packaging initiative and its partnership with the APS Packaging. 59% 73% 79% 72% 72% 10% 38% 11% 11% 16% 76% 76% 61% 28% 50% 86% 86% 89% 55% 72% Fatick Kaolack Kaffrine Dakar Total Percentage of Trained Staff Sealed Bags/ Sacs soudés Kraft Paper Packaging / Emballage en papier Kraft Lined Plastic Bags / Sacs en plastique doublé d'un film PE Plastic Bags / Sachets plastiques 55 12. Linking Producers and Processors to the Millet Market MBSP has put in place several initiatives to increase demand for millet, including Cantine Fondé and providing support for the Millet Processors to attend agricultural trade shows. The Program has developed new millet recipes and worked with producers and processors. The Evaluation has studied the trade of millet along the value chain. 12.1 From Producers to Processing Units The survey finds that 59% of producers are growing millet for auto-consumption, the main reason cited for not selling the millet they grow. The qualitative interviews show that farmers complain of low prices; they often hold on to their stock hoping that prices will increase. It is possible that unless higher demand leads to higher millet prices, it is unlikely that they will be motivated to increase their production of millet. They complain that it is difficult to find buyers, although 54% of surveyed farmers reported that it is now easier than before to find buyers, processors, and brokers who are willing to buy their crop. The survey shows that only 29% of Processing Units have a supply contract for raw millet from a MBSP Producer, as shown in Figure 38. Kaolack has the highest rate at 55%, while in Fatick and Kaffrine the percentages are 38% and 35%,respectively. In Dakar, the Processing Units reported that the MBSP Multi￾Services Hubs and Producers sell clean and high-quality raw millet. However, the survey finds that only 12% of Processing Units in Dakar have signed contracts with MBSP producers, preferring to buy raw millet from the local market. Figure 38 - Sources of Raw Millet for the Processing Units 19% 28% 50% 4% 19% 38% 44% 55% 70% 58% 38% 56% 35% 12% 29% 31% 24% 30% 16% 22% Fatick Kaolack Kaffrine Dakar Total Percentage of Millet Processors Contract with an independent / Contrat avec un fournisseur particulier Contract with MBSP producers / Contrat avec Producteurs PSEM Local market / Marché local Own production/Autoproduction 56 The Processing Units complain that Millet Producers set their prices too high. On the other hand, Millet Producers have indicated that if they think the market price is too low, they will hold on to their stock in the hope that the price will rise later, as mentioned previously. The Evaluation finds that there may be some lack of understanding on product pricing on the part of each party. Recommendation #21: That NCBA CLUSA and Extension Agents reinforce the business and marketing training of the Millet Producers and emphasize price setting based on supply and demand. Recommendation #22: That NCBA CLUSA reviews the marketing strategy of the Multi-Service Hub; that they encourage negotiation with Processing Units, especially the beneficiaries of the MBSP secure long￾term agreements for program sustainability. NCBA CLUSA has explained that it is providing grants to the Multi-Service Hubs for the acquisition of equipment to process raw millet into flour. This will facilitate making several millet products, for example couscous, since the processors will no longer need to hull the kernels and grind the cereal to make flour themselves. The Evaluation agrees with NCBA CLUSA that this initiative will likely increase productivity for finished products; however, given that Processing Units have limited funds and storage capacity, it remains to be seen whether there will be a demand for millet flour. The Evaluation finds that the Processing Units may not be holding constant operation. The survey shows that in 2017, only 48% of MBSP Processing Units operated throughout the year, an increase from 42% in 2015. In Kaffrine, only 30% of units are in operation year-round, whereas in Kaolack 68% are in operation year-round; this is the highest percentage reported and is an increase of 8% from 2015, as shown in Figure 39. The survey data also shows that 5% of MBPS Processing Units in Kaffrine and 13% in Dakar did not operate in 2017. Figure 39 - Percentage of Processing Units in Operation Year-Round 50% 60% 25% 37% 42% 50% 64% 30% 39% 44% 56% 68% 30% 44% 48% Fatick Kaolack Kaffrine Dakar Total of all Regions Percentage of Processing Units 2017 2016 2015 57 Recommendation #23: That NCBA CLUSA verifies whether all Processing Units are currently in operation. If not, it is recommended that NCBA CLUSA documents closure reasons to help identify more effective beneficiary selection criteria. The partial operation prevents regular and constant demand for raw millet and supply of finished millet products during the year. Recommendation #24: That NCBA CLUSA investigates why the MBSP Processing Units are not in operation throughout the year. When asked about difficulties and challenges that they face, Processing Units reported absence of equipment as the most common problem, as shown in Figure 40. In Fatick and Kaolack, this issue was reported by half of the Processing Units; in Kaffrine, 75% of units reported this issue; in Dakar, 61% complained about lack of equipment. Access to credit was the second most common issue. One-third of all processors reported having problems accessing credit, with processors in Fatick and Kaffrine reporting the highest levels of issues (50% and 45% respectively). Figure 40 - Challenges faced by Processing Units The Evaluation finds that trading between Millet Producers and Processing Units for regular supply of raw millet especially in large quantities cannot be guaranteed by mere signing long-term contracts for quantities unless the Processing Units run effective and efficient operations; and unless they can 50% 25% 19% 13% 50% 52% 28% 28% 28% 24% 75% 35% 30% 30% 45% 61% 30% 30% 28% 28% 60% 30% 28% 26% 33% Absence of equipment / L'absence d'équipements Lack of availability and consistency of raw material / Manque de disponibilité et de régularité de la matière première Low demand and local consumption / Faiblesse de la demande et de la consommation locale High production costs / Couts de production élevés Difficulty accessing credit / Difficultés pour accéder au crédit Percenatge of Processing Units Total Dakar Kaffrine Kaolack Fatick 58 identify and increase their processing capacity in the long run. Otherwise, their demand for raw millet is not predictable, which will be challenging to meet the agreement terms with producers and providers of raw millet. 12.2 Processing Units to National Market and Export The Evaluation found that Artisanal Processing Units retail their products to small resellers on the local market. The semi-industrial firms sell labelled products to supermarkets, but there is no evidence that processors have signed contracts to sell large volumes on a long-term basis on the national market or to export. While NCBA CLUSA provided a copy of a sale of 250 tons of raw millet made by a GIE and a copy of a sale of finished millet products which included Araw, Thiaki and Thiere for an amount of approximately 14 million CFA there was no enough evidence to conclude if these two orders would result in a long-term relationship or whether these were onetime orders. As millet production rises, the Evaluation questions whether the Processing Units targeted by the program will be able to purchase and process large volumes of raw millet. This is concerning given that most beneficiaries still use artisanal processes and lack modern equipment. These artisanal Processing Units have limited inventory space and have limited cash flows and capital. The semi-industrial units will be able to absorb larger volumes of raw millet with more efficient technology, and the Evaluation believes that the program should support the Processing Units according to their current capacity and processing methods rather than through a “one size fits all” initiative. Modernizing equipment and acquiring new technology will require access to credit. NCBA CLUSA and CRS should support all Processing Units to find financial support as needed. Recommendation #25: That NCBA CLUSA reviews its strategy for supporting the Processing Units, focusing on equipment that can increase efficiency and productivity. Since the Multi-Service Hubs also include millet processing operations, the Program should prepare the POs to start looking for domestic and export markets for their processed millet. Recommendation #26: That NCBA CLUSA reinforces the marketing strategy of Multi-Service Hubs for processed millet. 12.3 Referrals from NCBA CLUSA The project provides training and marketing tools to the beneficiaries. The project has linked the Millet Processors and help them reach potential clients through business fairs and business forums; however, NCBA CLUSA does not sign contracts with clients on behalf of its beneficiaries. The processors were asked whether the Program gave them referrals or put them in contact with potential buyers of their products. Figure 41 shows the findings of survey of Millet Processors; 62% of respondents report that NCBA CLUSA has not linked them to the market. This number is lowest in Kaolack, which may indicate that it is in this region that the program is supporting the processors more often. Figure 41 also shows that some of the Processors report that the Program has linked them with villagers, resellers and restaurants. In total, 5% of the Processors reported that the Program has placed them in contact with wholesalers and distributors. In the case of Kaolack, 12% reported that the program has helped them in this manner. In Fatick there was no report of NCBA CLUSA linking them to these traders. In fact, the Program has 59 supported and financed the presence of the Processing Units at the business forum and agricultural fairs where they gain visibility with wholesalers, distributors and exporters. The participants spoke highly of their experiences and the sales made there. NCBA CLUSA has provided a copy of a contract for the purchase of raw millet from a GIE by an export company. Figure 41 - Processing Units Referral by NCBA CLUSA to Potential Customers Recommendation #27: That NCBA CLUSA continues to promote synergy between processors, consumers, wholesalers, retailers, and the export market and encourage the establishment of long-term partnerships among the trading actors in the Millet Value Chain. The Millet Processors reported an increase in the amount of millet that they have processed since 2015. According to the survey data, there was an overall increase of 80% in the amount of millet processed from 2015 to 2018, suggesting that, just as described in Figure 42, there is more trading of both raw and processed millet by the MBSP Beneficiaries since they have taken part in the program. About 75% of the total millet is processed in Dakar. The Evaluation finds that MBSP is having an overall positive impact on the processing of millet in the targeted areas despite the limitations and challenges identified by this study. The Evaluation recommends that NBCA CLUSA monitors the progress of the Processing Units more closely. 13. Access to credit Catholic Relief Servicesis NCBA CLUSA’s partner and sub-contractor for supporting the MBSP beneficiaries in saving money and accessing credit for investment in millet-related activities. CRS has established a network of Intermediate Financial Service Providers (APS-IF) trained in the CRS Flagship Savings and Internal Lending Communities (SILC) methodology. APS-IF are community agents, trained by CRS in 2% 9% 2% 4% 4% 2% 61% 6% 13% 6% 0% 0% 0% 75% 12% 8% 8% 12% 4% 4% 56% 5% 0% 0% 5% 10% 5% 60% 5% 8% 3% 5% 4% 3% 62% Villagers / Villageois Resellers / Revendeurs (Bana-bana, Vendeurs de rue) Restaurants Wholesaler and Distributors / Grossistes et Distributeurs Exporters / Exportateurs Othr processing units / Autres unités de transformation No Referrals / Aucune mise en relation Percentage of Processing Units Dakar Fatick Kaolack Kaffrine Total -All Regions 60 financial education, marketing and how to work with financial institutions. The role of APS-IF is to promote the SILC methodology among farmers, support the creation of SILC groups and encourage the SILC members to carry out income generating and savings activities within their communities. Figure 42 - Trend in Millet Processing by Region from 2015 to 2017 The APS-IFs must pass an exam before receiving full certification, during which they receive an allowance of $45 per month for a maximum of 9 months. They are technical advisers supporting producers and linking them to financial institutions. The certified APS-IFs are paid by the communities they train and support. The APS-IF can offer additional training in financial education, entrepreneurship and basic marketing which can then be promoted to SILC group members for a fee. According to CRS, the project has recruited 17 APS-IF since the start of the program and created 416 SILC groups among millet producer organizations with savings of $203,000 USD (over 113 million CFA francs) to date. By May 2018, a total of $836,000 (over 466 million CFA francs) in loans were distributed to farmers by SILC groups and banks18 . According to CRS, this demonstrates the impending possibility of sustainable relationships between these financial services and millet value chain actors without financial guarantee. The Evaluation Team was further informed that SILC groups of no more than 30 producers are formed in villages by an APS-IF. When this number is exceeded, a new group is created. Each village sets up meetings through its PO; these are often weekly meetings where the members of the SILC group bring their contribution to the SILC savings funds. Contributions to the SILC account are not set amount but vary depending on the group that decides where and how the funds should be disbursed. The APS-IF can deal and negotiate with Input Suppliers, overseeing the transactions for purchase. During the qualitative interviews one APS-IF noted that he is paid 300 CFA per producer per month. Hence, the more SILC Groups formed and supervised by him, the more he earns. As an APS-IF, his role included visiting the fertilizer supplier to verify product quality before a purchase contract is made, as with the APS 18 Sustainable Financial Services Tailored to the Millet Value Chain, Catholic Relied Services, June 2018. 0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 Fatick Kaolack Kaffrine Dakar Total Amount of Millet Processed in tonnes 2015 2016 2017 61 Semencier ensuring that the seeds were of good quality. He explained that seeds are bought in a large amount and packaged in 4kg bags, which were sold to the MBSP producers for 2400 CFA each. This APS￾FI thinks that SILC is a good methodology; it allows farmers to save to buy inputs and equipment and the opportunity for the SILC Group to make further business, as in the case of seeds reselling described here. CRS explained that the program is creating new jobs for the individuals who were trained as APS-IFs. The position of APS-IF gives the person a new socio-professional profile at the community level and raises his/her social status. This was confirmed during the qualitive interviews. APS-IFs are producers and members of POs; they are usually young producers and are paid by the SILC Groups or the community. The Qualitative Interviews revealed that the producers and the POs also feel very positive towards the SILC initiative. The Evaluation Team heard of two cases where producers raised 15 million CFA and 10 million CFA for capital investments, including equipment. The producers benefit immensely from the fact that there are no interest nor fee payments when financing through the SILC initiative, unlike financing through traditional loans from financial institutions. The informants find that preparing documents for loan applications takes great effort and that management fees are high. All individuals involved praised the support CRS provided. Moreover, the financial institutions indicated that since CRS became involved, debt holders have a much better understanding of the responsibilities and commitments of debt repayments and/or loan renegotiations. 2016 was not a good harvest year for millet due to unusually high rainfall. Nevertheless, the financial institutions feel confident that the renegotiated terms of loans not reimbursed on time will be honored. However, the financial institutions lament that farmers are no longer depositing their savings in the bank because of SILC. The Evaluation did not question where SILC funds are kept. The SILC Group has an opportunity to partner with financial institutions so that SILC funds can earn interest deposited at the bank. Recommendation #28: That CRS investigates the possibility of depositing SILC funds at banks to earn interest when large sums of money are waiting to be spent on the next wintering season. When questioned whether they would like to have a loan, 84% of the millet producers answered affirmatively. Producersin Kaolack are more willing to obtain financing compared to producers in the two other regions, as can be seen in Figure 43. Figure 53 - Percentage of Millet Producers Who Would Like to obtain a Loan 80% 91% 81% 84% 16% 3.1% 7% 2.0% 15% 2.9% 13% 2.7% Fatick Kaolack Kaffrine Total Percentage of Millet Producers Yes / Oui No / Non Don't know / Ne sait pas 62 In addition to millet producers, CRS is facilitating access to credit for processors to buy machinery and equipment from local artisans or purchase raw millet from producers. CRS explains that it is currently working to support development of business plans for processing enterprises, threshing suppliers, and storage services in line with the expected results of USDA MBSP. While CRS, APS-IFs, the producers and the financial institutions feel very positive about the SILC Initiative, NCBA CLUSA on the other hand, does not feel that MBSP is fully profiting from this initiative. NCBA CLUSA has indicated their concerns that the savings generated by the SILC Groups are not being used or invested in millet related agricultural activities. NCBA CLUSA cited a case where the savings were used to start a business in soap manufacturing rather than for the millet production. Since the SILC Methodology has been found to be successful, the Evaluation thinks that it is to be expected that the villagers are also using this methodology to raise money for other purposes. However, the Evaluation finds that it is essential that the individuals who receive training from the MBSP through CRS and the APS-IF, be committed to using SILC Savings for Millet-related activities, expenditures and investment; for example, commitment to purchase a ripper using the SILC Savings. The Evaluation recommends that NCBA CLUSA insists that SILC Groups be created on this basis i.e. that the SILC savings that have been raised under the auspices of MBSP shall be directed to the millet sector. The Evaluation recommends that NCBA CLUSA reviews with CRS the objectives of the SILC Program and any other activities that CRS is doing within MBSP, establishing clear targetsfor the number of SILC Groups formed and funds raised for investments in the millet sector. The Evaluation also recommends that CRS keeps track of the disbursement of the SILC savings, categorizing them in millet and non-millet expenditure and investments for monitoring and research purposes. It will be interesting to find out if the investment of the SILC outside the millet sector have a spillover effect or indirect unforeseen results on the sector in the long-run. Recommendation #29: That CRS and the APS-IF keep track of the disbursement of the SILC savings, categorizing them in millet and non-millet expenditure and investments for monitoring purposes and research. Recommendation #30: That NCBA CLUSA reviews the objectives of the SILC Program and any other financial activities that is being promoted within MBSP Catholic Relief Services. That targets are set for the number of SILC Groups formed and funds raised for investments in the millet sector. That CRS and the APS-IF form and train SILC Groups only with producers who formally agree to direct the SILC Savings to Millet related activities and investments. The Evaluation has concerns about the role and independence that the APS-IF are allowed to have in the community. It is not clear by whom and how the work of the certified APS-IFs is being monitored and if they are marketing their services under the banner of CRS, NCBA CLUSA, MBSP or USDA. The fact that they are viewed prominently in the community provides a certain level of authority and makes them trustworthy. The Evaluation finds that this raises the risk of abuse, misuse and mismanagement of the Program. NBCA CLUSA should ensure that there is no misrepresentation of the organization, MBSP and USDA in the SILC initiative. The Evaluation recommends a review of the objectives of the SILC Initiative 63 and the role of CRS staff and APS-IFs on this project. Furthermore, it will be important to set a target for the number of SILC Groups formed and funds raised for investments in the millet sector. Recommendation #31: That initiatives such as SILC training and SILC Groups initiated within MBSP by the APS-IF be directly related to the Millet sector and formally tied to a specified investment in millet production or trade before the initiative can start with an PO or in a village community. Recommendation #32: That the role and responsibilities of the APS-IF be reviewed; that the supervision and monitoring of the APS-IF be formally assigned to one of the stakeholders; that NCBA CLUSA ensures that there is no misuse of the CRS Certification and MBSP by the APS-IF. The evaluation suggests that the needs of the smallholder millet farmers be further investigated for a better understanding of why the farmers are so willing to obtain a loan. As shown in Figure 43, over 83% of the Millet Producers indicate that they would like to have access to credit. It is also suggested that the aspiration and expectation of the beneficiaries towards the program and their commitment to the millet production be investigated. The findings of such a study will allow NCBA CLUSA to identify and invest effectively in the capacities of the millet producers as well as conduct proper selection of the beneficiaries. Consequently, CRS and the APS-IF will be able to provide effective guidance to meet the aspirations of the Millet Producers and achieve the objectives of the program with investments of the SILC savings that will have an impact in the millet sector. Recommendation #33: That the needs of the smallholder millet farmers be further investigated for a better understanding of how the SILC savings are being spent and what are the aspirations and expectations of the beneficiaries when it comes to their commitment to the production of millet. 14. Efficiency of the Project Management The Evaluation has reviewed the Performance Indicators and several Activity and Results indicators have been calculated. The survey data shows that the number of hectares of land under one or more improved techniques or technologies as a result of the program have surpassed the set targets. However, it appears that the target for the number of individuals who are applying new techniques or technologies as a result of USDA assistance may have been set too low. Annex B contains the results of this analysis and provides some recommendations for the refinement of the Performance indicators. Overall, the Evaluation finds that the various activities of MBSP have been implemented as intended and that the program is on track. The Evaluation has examined the management structure of the program. As noted previously in this report, the delivery of the training and the implementation of the MBSP initiatives involves many players and stakeholders. The Organization Chart of the Program is shown in Figure 44, which shows that there is a Director of Finance and Administration as well as the M&E Specialist reporting to the Country Director, who also supervised five Program Specialists managing the initiatives for: 1. Financial Services; 2. Extension Services and Agricultural Services; 3. Processing and Post-Harvest Services; 4. Business Development Services/Marketing; and 5. Producer Organizations. 64 There are 13 coaches who are implementing the activities in the field under the supervision of Program Specialists. The Specialist and three of the coaches responsible for Financial Services are CRS personnel who are executing the access to credit activities and establishing the SILC program. Ten other coaches are working in the field, supervising the trainers from ANCAR and DRDR for the training program for the Producers; the Program activities with Input Suppliers; the Cantine Fondé initiative and the program with the Processing Units. In addition, they supervise the Extension Agents and APS. The Evaluation finds that targeting 3 rural regions and two urban areasthat span over a large geographical space to reach all actors along the Millet Value Chain is a very ambitious undertaking. By the end of the project, the program’s goal is to have 22,150 producers, 100 agro-dealers and 200 processors trained, accompanied, shadowed and/or mentored. This evaluation has not reviewed the statement of work for each staff member, but it seems that the coaches may have a heavy workload that does not allow time for proper monitoring and coordination with collaborating partners. The ANCAR trainers interviewed find that organizing and scheduling trainings at times that are convenient to all is an issue, since NCBA CLUSA Coaches are often unavailable. They mentioned that NCBA CLUSA has scheduled trainings during important holidays when producers are too busy to attend courses. They also mentioned incidents when NCBA CLUSA Coaches scheduled meetings with producers at times that interfere with the attendance of the person at the training session. The informants feel that there is a lack of coordination and task prioritization on the part of NCBA CLUSA staff. Since the training is assigned to consultants, ANCAR, DRDR and the Agricultural Extension Workers, it is very important that NCBA CLUSA monitors the quality of the training. Furthermore, the Evaluation finds it important that the advice and services provided by the Extension Agents to the beneficiaries are relevant, accurate and of high standards. As with the SILC APS-IF, the evaluation recommends that NCBA CLUSA ensures that there are no misuses or misrepresentations of the Project and USDA on the part of the field staff, partners and Extension Workers. Recommendation #34: That NCBA CLUSA reviews its staff’s workload and introduces quality assurance measures in its activities and program deliverables. 65 Figure 44 - Management Structure of MBSP19 19 Provided by NBCA CLUSA. 66 The Evaluation is aware of issues regarding staffing the Monitoring and Evaluation (M&E) Officer position. The current officer has been in post for less than a year and the implementation of the M&E system has been challenging. According to the survey data, not all producers have received training. The evaluation recommends that NCBA CLUSA investigate why all the beneficiaries have not been trained on their database. It is quite possible that the database that was provided to the Evaluation Team contained newly registered persons; nevertheless, the registration process of beneficiaries must be clear. In the case of the producers, it was not clear to the Evaluation Team who was selecting the beneficiary; what was defined as the start date in the program; who was responsible to collect the data on the persons for the M&E system and which training the beneficiary attended and when. A unique identifier should be assigned to each beneficiary. The Evaluation recommends that data is collected on electronic devices, such as laptops or tablets, for standardization of variables to reduce errors associated with using paper forms, such as unreadable entries and lost forms. The Evaluation recommends that data is also collected on beneficiaries who drop out of the program. It is important that information is accurately recorded since it is used to produce the Program Performance Indicators. Recommendation #35: That NCBA CLUSA reviews its monitoring system and sets clear procedures for the registration of beneficiaries; identifies and collects accurate information that is needed for performance monitoring. It has been difficult for the Evaluation to obtain complete and detailed documentation on the program in a timely manner for its document review. Information was gathered piecemeal from internal documents listed in Annex E. The Evaluation thinks that there are communication problems among staff who are not proficient in English; for example, staff who could not read English, did not seem to be fully aware of the USDA Agreement with NCBA CLUSA and the Term of Reference for this evaluation. Recommendation #36: That NCBA CLUSA makes important project documents available in English and French. The Evaluation has found that the staff at the NBCA CLUSA Country Office are very enthusiastic about project. They appear to be very dedicated to their work and to the advancement of the Millet Sector in their country. 15. Conclusions The evaluation has found the Millet Business Services Project was well thought out and its activities were designed to meet its two objectives, which are to increase the agricultural productivity of the millet value chain and expand trade of millet. The Evaluation applauds NBCA CLUSA for taking a calculated approach in the program design to prevent beneficiaries’ dependency on grants and donations. This mid-term process evaluation finds that the program is in line with USDA objectives for sustainable economic growth and development. The Evaluation confirms that the activities and initiatives that were agreed by USDA and NCBA CLUSA have been put in place. With regards to the first program objective, which is to increase agricultural productivity of the millet value chain, the Evaluation finds that there are positive trends in the adoption of new farming technology and signs that the MBSP beneficiaries are cultivating and producing better quality millet. Nevertheless, the evaluation has unveiled several challenges faced by the program. These could very well affect the quality of the expected output and results of the program. The Evaluation has found that farmers are 67 constrained in the quantity of millet they can produce, and this can delay or perturb the achievement of the second objective, which is to expand trade of millet. Due to lack of money, the Evaluation finds that farmers on one hand are prevented to access more land to cultivate millet, and on the other hand they are constrained in adopting new agricultural methods that require the purchase of inputs, such as high￾quality seeds and fertilizers or tools and equipment. In fact, lack of money is the major challenge faced by the MBSP beneficiaries. The Evaluation finds that the SILC methodology is able to help the beneficiaries to save money and raise funds. However, it appears that the Catholic Relief Services and NCBA CLUSA are not sufficiently insisting that the SILC Groups trained and mentored within MBSP allocate their savings to millet-oriented expenditures and investments. In fact, all farmers selected to participate in the program should in the first place be committed to millet production. It is hence important that the beneficiary selection criteria explicitly lead to the attainment of the project’s goals. Before selection, the potential beneficiaries should be explained that this project will not hand out monies but is instead building their capacity in millet cultivation and requires their active participation in the millet value chain. To reach and support producers, the Program is using two methodologies: 1. Multi-Services Hubs where several POs are consolidating their activities and 2. Extension Agents who accompany the producers who have been trained in farming and business techniques. The Evaluation finds that the Hubs strengthen the negotiating powers of the POs with the actors in the Millet Value Chain, including the financial institutions. Setting SILC Groups within the Hubs can ensure that the monies are used for millet related activities; the money can also subsidize the maintenance and growth of the Hubs as well as the operations of the PO. The Evaluation thinks that the Hubs may be more effective in supporting illiterate farmers than the Agricultural Extension Agents since the environment bring constant assistance. Furthermore, safety and quality standards are more easily enforced at the Hubs than with individual millet producers. This evaluation recommends that the Multi-Services Hub approach be examined further for program cost￾efficiency and quality of outcomes and output. The evaluation recommends that future studies, including the end-line evaluation, compare the impact of the program on the Multi-Services Hubs’ Producers and on independent producers who were not members of a Hub. Such investigation will identify whether the Hubs approach fosters more effective integration of input suppliers, millet producers and processors in the value chain while offering products of higher quality in large volumes that can sustain trade nationally and abroad. The Evaluation recognizes that the construction of Multi-Services Hubs requires large amounts of funds and limits the number of beneficiaries that can be reached within the large geographical areas that MBSP is targeting. The survey shows that the program is having the most positive results in Kaolack. Since there are no baseline data for the three regions, it is not known whether the millet sector was stronger in Kaolack than in Fatick and Kaffrine before the program started. It is interesting that the yield study performed by DRDR shows that the production of millet is superior per hectare in Kaolack compared to the two other regions both for the demonstration and control groups. NCBA CLUSA has explained that there was a greater presence of the Food for Progress program in Kaolack than in Fatick and Kaffrine. This may have contributed to a healthier and more developed agricultural sector in Kaolack. The Evaluation finds that the program results trail in Kaffrine, where there is a higher proportion of women among the MBSP beneficiaries. Women, although quite active in the agriculture sector, face more challenges than men. The 68 Evaluation has suggested that the needs of the beneficiaries are studied further. It finds that it is reasonable to reinforce the program in Fatick and Kaffrine during the second period of this program. Regarding the second objective of the program, which is to expand the trade of millet, the Evaluation believes that the program can do more and suggests that NCBA CLUSA appraises its program strategy with processors, especially the small and medium size units, to improve hygiene and safety in their operations, support the acquisition of tools and equipment, and promote year-long operation and long-term contracts with the Hubs for the supply of raw millet to support the sustainability of the program. The Evaluation also suggests customized training programs in processing technology and business methods, depending on the production capacity of the processing units to supply millet products to local, national and export markets. The evaluation finds that confronting all the actors along the value chain in a single program is an ambitious and challenging undertaking. This mid-term evaluation concludes that with some adjustments, this program will be able to achieve its goals. JAA, however, believes that although it is important to reach the targets set up for the performance indicators, it is also essential that the gains made by the beneficiaries through MBSP are sustainable in the long run. It finds that using the POs as the point of entry for beneficiaries is a sound approach; the POs should be able to continue the work of the program, especially if solid governance systems exist. In the event of natural disasters that can be expected as the result of climate change, for example excess rainfall causing damage to the millet production in 2016, the greatest accomplishment of this program will be resilience and the ability of its beneficiaries to progress in the cultivation and processing of millet. 69 Annexes 70 Annex A – Evaluation Design Matrix 71 Annex B - Analysis of the Performance Indicators 72 Annex C – MBSP Theory of Change 73 Annex D – Qualitative Interviewers Guides 74 Annex E – Survey Questionnaires 75 Annex F – List of Documents Reviewed