An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 1 Final Report – for the general public EVALUATION OF BASELINE ESTIMATES OF THE FY 17 FOOD FOR PROGRESS MARKET-ORIENTED DAIRY (MOD) PROJECT prepared for United States Department of Agriculture Washington, D.C. and IESC Washington, D.C. Prepared by Sam Daniels, BVSc, MSc, MA, PG Dip Econ and Gregory Sullivan, PhD. May 15, 2018 [ EVALUATION OF BASELINE ESTIMATES OF THE FY 17 FOOD FOR PROGRESS MARKET-ORIENTED DAIRY (MOD) PROJECT] Program: Food for Progress Agreement Number: [FCC-383-2017/031-00] Funding Year: Fiscal Year 2017 Project Duration: [FY 17-FY 22] Implemented by: [IESC] Evaluation Authored by: [Daniels, Sam Sullivan, Gregory] DISCLAIMER: This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Accessibility Note: An accessible version of this document can be made available by contacting fas.monitoring.evaluation@usda.gov An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 2 TABLE OF CONTENTS ACRONYMNS....................................................................................................................................... 4 GLOSSARY........................................................................................................................................... 5 EXECUTIVE SUMMARY................................................................................................................... 6 1. INTRODUCTION TO BASELINE EVALUATION..................................................................... 8 1.1. Summary of Baseline Survey 1.2. Objectives of the Baseline Review 1.3. Target Areas of the MOD Activities 1.4. Profile of Dairy Farmers 1.5. Dairy Imports 2. EVALUATION PLAN AND METHODOLOGY FOR BASELINE STUDY............................12 2.1. Evaluation Plan and Activities 2.2. Methodology and Tools 2.2.1. Mixed methods 2.2.1.1. Quantitative Data 2.2.1.2. Qualitative Data 2.2.2. Data Collection Tools 2.3. Lessons Learned/Key Things to Monitor Going Forward 3. INDICATOR #1. VOLUME OF COMMODITIES SOLD BY PRODUCERS…………..….16 3.1. Indicator and Major Assumptions 3.2. Target Milk Production Areas 3.3. Production Coefficients to Estimate the Baseline Volume of Milk 3.4. Three Dairy Production Models 3.4.1. Milk production (Model 1) 3.4.2. Intensive dairy production (Model 2) 3.4.3. Farms with improved European and Cross-Bred Dairy Cattle (Model 3) 3.4.4. Estimate of the baseline milk production 4. INDICATOR #2. VALUE OF SALES BY PROJECT BENEFICIARIES............................. 20 4.1. Indicator and Major Assumptions 4.2. Findings 5. INDICATOR #3. REDUCTION IN REJECTIONS FOR QUALITY AT COLLECTION POINTS .............................................................................................................................................22 5.1. Indicator and Major Assumptions 5.2. Current Situation 5.3. Possible measure of improvement in milk quality 5.3.1. Test for quality 5.3.2. Resazurin Test 5.3.3. Farm level 5.4. Conclusion 5.5. Recommendations An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 3 6. CONCLUSIONS, LESSONS LEARNED AND RECOMMENDATIONS FOR M&E FOR THREE BASELINE INDICATORS............................................................................................. 29 6.1. Conclusions 6.2. Lessons Learned 6.3. Recommendations 7. ANNEX......................................................................................................................................... 32 7.1. Data sources 7.2. Scope of Work for Baseline Consultant 7.3. Interview guides 7.3.1. Artificial Inseminator (AI) 7.3.2. Producer (individual and group at a chilling center) 7.3.3. Bulk Collector 7.3.4. Milk Collection Points 7.3.5. Processors 7.4. Milksheds for the major processing companies 7.5. Review of proposed methodology for collecting, compiling and reporting data 7.5.1. Overview 7.5.2. Collecting and Compiling 7.5.2.1. Profiling of target beneficiaries (farmers) 7.5.2.2. Collecting data from target beneficiaries (farmers) after interventions 7.5.3. Reporting 7.6. PowerPoint presentation describing approach, methodology and findings to MOD staff Slide 7.6.1. Debrief on Baseline Estimates – Market Oriented Dairy Slide 7.6.2. Approach to the Evaluation Slide 7.6.3. Target farmers for MOD Slide 7.6.4. Methodology Slide 7.6.5. Findings: Profile of Dairy Farmers Slide 7.6.6. Findings: Profile of Processors Slide 7.6.7. Findings: Profile of Input Suppliers Slide 7.6.8. Findings: Retail Market for Dairy Products Slide 7.6.9. Finding: Quality of Milk Slide 7.6.10. Preliminary Recommendations on Baseline Indicators LIST OF TABLES Table 1.1. Quantity of milk production and milk imports (liquid milk equivalent) in 2015 and 2016. Table 1.2. Number of dairy cows and cow milk production by provinces, 2016 Table 1.3. Dairy systems in each key dairy province Table 1.4. Quantity and value of imports of dairy products-January to December 2015, 2016, and 2017 Table 2.1. Numbers of people interviewed by type and geographic locations Table 2.2. Cows in milk for cattle and buffaloes and collection centers by top six dairy districts, 2016 Table 3.1. Provinces, districts and percent of area coverage for commercial production Table 3.2. Technical coefficients for milk production by province and production system Table 3.3. Model 1. Estimate milk production based on technical coefficients and number of dairy cows in province Table 3.4. Model 2. Milk production estimated based on the number of farms that are intensively managed. Table 3.5. Model 3. Milk production based on farms with European and Cross-breed cows Table 3.6. Weighted calculations of milk volume in the baseline An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 4 Table 5.1. Quality tests performed by processors at different stages in the formal milk supply chain Table 5.2. Steps taken to administer the Resazurin test Table 5.3. Reading the results of Resazurin test (10-minute Resazurin test) Annex Figure 7.4. Milkshed for the major processing companies An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 5 ACRONYMS AI Artificial Insemination AIT Artificial Insemination Technician AT Alcohol Test CO3 A type of hybrid grass (Napier) CP Collection Point DDO District Development Officer GMP Good Milking Practices GOA Government of Australia GOSL Government of Sri Lanka GT Gerber Test l liters l/d liters per day l/hd liters per head LKR Sri Lanka Rupee LM Lactometer LOP Life of the Project MCC Milk Chilling Center KII Key Informant Interview KPI Key Performance Indicator KQ Keeping Quality MOD Market-Oriented Dairy project PMP Performance Management Plan RT Resazurin Test SNF Solids-Not-Fat TS Total Solids UHT Ultra-High Temperature USDA US Department of Agriculture An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 6 GLOSSARY Alcohol test (AT) – is used to measure the bacteria in the milk and is done at the chilling center. Processors set their acceptance on milk based on analysis of different percent levels of sodium hydrate in the test. The alcohol test by processors can range from 65% to 85%. The higher the percentage and the milk tests negative, then the higher the quality of the milk. This test is especially important for the manufacture of UHT milk, evaporated milk, and powder milk. Adulteration – is a measure of additives added to the milk to improve the SNF. Adulterants can consist of sugar, salt, starch, and urea. Fat – is the most important constituent of milk as it is used as a basis for fixing the purchase and sale price of milk. It helps to detect adulteration like watering and skimming of milk. Gerber Test (GT) – used to determine level of fat in the milk using sulfuric acid (sp.gr.1.82), and amyl alcohol (sp.gr.0.82-0.83). Keeping Quality (KQ) – or keeping power is an expression used to indicate the length of time milk remains sweet and otherwise palatable and suitable for direct consumption, and as such an estimation of the commercial value of milk because the milk that is sour or unpalatable is useless however much it is rich in butter fat and solids not fat. All milk processors use the Resazurin dye reduction test for measuring the keeping quality of milk. Based on the degree of reduction of the color of the dye, a range of values from 0 – 6 is given, with value 6 being given with no reduction of color. Any milk with a value of 4 (dye changed to pink) or below is rejected. 1 ml of Resazurin solution is added to 10 ml of milk and heated to 37c degrees for 10 minutes and color changes noted. Color indicates the quality of milk (white color – reject, purple or pink suspicious, accept if light blue and blue). Lactometer (LM) – measures the specific gravity of the milk and is part of the calculation of Total Solids which affects the price paid for milk. Mastitis – is the inflammation of the cow which may be clinical (visible to the naked eye) or that is sub￾clinical (difficult to observe without testing). Mastitis can result in an imbalance in the salts in the milk making milk being positive for the alcohol test. The testing of milk for sub-clinical mastitis can be performed at the farm level using the California Mastitis Test. Milk tester equipment – an electronic unit which can measure several factors in milk quality from one small sample of milk. YARLCO dairy cooperative in Jaffna has bought nine milk testers to be based at collection points to test milk quality. Resazurin test – when bacteria grow in milk, they utilize oxygen dissolved in the milk. The removal of the oxygen from milk and the formation of reducing substances during bacterial metabolism causes the color of the Resazurin dye to disappear. As such this test is to determine the extent of bacteriological quantity of the milk SNF - Solids Not Fat – is a measure using the following equation based on the measurement of lactometer and the Gerber fat test. Equation to estimate SNF = .25*LM + .22*FAT +.72. If the SNF value is less than 8%, processors will reject the milk. Total Solids (TS) – is the SNF percentage + the Fat percentage and averages around 12.5 percent in the cow milk. An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 7 EXECUTIVE SUMMARY The evaluation team examined three main baseline indicators for the Market-Oriented Dairy (MOD) project: ➢ Volume of commodities (liters) sold by project beneficiaries; ➢ Value of sales by project beneficiaries (USD); and ➢ Percent reduction in rejections for quality at collection points and at milk chilling centers from beneficiary farmers. To validate the initial estimates made in the proposal phase, the team used a mixed-method approach to gather both quantitative and qualitative data from public and private stakeholders. Quantitative data were from both desk research and interviews with public and private sector stakeholders. Qualitative data were from interviews with key informants in the dairy value chain. The team conducted focus groups with producers at milk collection points and milk chilling centers. Some meetings were spontaneous during a visit (such as to a collection point); and in others, dairy development officers employed by processors arranged meetings at a specific time. Indicator #1. Volumes of milk (liters) sold by project beneficiaries The MOD intervention area will cover six provinces. In the team’s opinion areas of some districts in a province or the entire area of certain districts would be suitable for rapid commercialization of dairy operations. For this reason, the team chose to visit certain areas because of time available and because of the likelihood of producers in those areas becoming commercial dairy farmers during the life of the project (LOP). However, MOD may choose to work in some of these districts the team did not visit. The selection criteria were based on farming characteristics, agro-ecological conditions, species (buffalo) and breeds (indigenous). We found that the average milk yield per cow, number of milking cows, and lactation period varied across types of cows and the farming systems in the provinces. The Central province, and Badulla and Jaffna districts had more European crosses, and dairy cattle had longer lactation periods than the rest of the areas. In the proposal, the baseline projection in the proposal was 36,674,000 liters, and the evaluation team’s estimate is 37,338,000 liters, ranging from 35,452,000 to 40,762,000. The baseline quantity is within the range and is reasonable as the baseline figure. Indicator #2. Value of sales by project beneficiaries (USD) The team interviewed many producers with varying sizes of dairy herds. A few producers received prices in the range of LKR 70 to LKR 75 per liter. The project’s baseline estimate was at the low range of LKR 70 per liter of this range. However, on average, the team’s observation is a lower average farm gate price which results in a much lower estimate of the value reported in the performance management plan (PMP). The value of sales at the farm level is an estimated US$ 15.01 million versus the project’s estimate of US$ 21.9 million. The lower price per liter also reflects the lower average level of Total Solids (Fat plus SNF =TS) in the milk. The calculation for SNF is the results of the Gerber test for fat and the lactometer reading. The quality of the milk measured in bacteria, somatic cells and stability of milk (keeping quality) is not a factor in the current calculation of the milk price to producers. The MOD project team can work with the industry to improve milk quality and its composition. Over the LOP the producer can realize higher value for the milk produced with the coupling of TS and quality, and there is room in the price range to do this. The total value of sales by project beneficiaries over the LOP will improve because of improvement in quantity and quality of milk. The evaluation team recommends adjusting the baseline value of milk to US$15.01 million from the current estimate of US$21.9 million. An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 8 Indicator #3. Percent reduction in rejections for quality at collection points from beneficiary farmers The team found that the indicator of 80% reduction in the rejection of milk is not an effective measure for improving the quality of milk for the MOD project. There are two reasons: (1 st) the rejection rate is already low for milk received through the farm management societies (FMS) (preferred formal channel) because processors will take most milk supplied which either goes to ultra-high temperature (UHT) milk (passes the alcohol test (AT) at the 75% level or higher); or if milk passes at 68% alcohol, milk will go into processing and cultured products; and (2nd ) the industry is mainly focused on testing fat and SNF rather than other quality measures (bacteria or somatic cells). The industry does not pay a premium currently for milk with low levels of bacteria and somatic cell counts. As an example of an industry practice, Milco purchases milk for UHT from selected milk chilling centers close to the two factories which produce UHT milk, e.g., Polonnaruwa and Digana factories. At these centers the extension officers are more vigorously supervising the milk production at farm level and the milk transport time to the factory to maintain a higher quality standard for milk. Also, Milco uses 75% alcohol for testing the milk at these centers. At Milco’s milk chilling centers where milk is used for the manufacturing of other products, the company uses 68% alcohol for testing milk. Similarly, Richlife, which produces UHT milk, uses 75% alcohol for milk testing. Cargills uses 80% alcohol for testing milk at the factory. The other processors which use milk to manufacture milk powder or cultured products use 68% alcohol for milk testing. The industry needs to focus on geographic areas where the level of alcohol is set lower, e.g., at a level of 68% alcohol as milk quality can be improved with farmer training in good milking practices (udder cleaning, hand washing, and use of stainless-steel containers) and reducing the milk transport time from the farm to the chilling center. The MOD project team can collaborate with the processors on considering what interventions they would be willing to support to improve milk quality. Ultimately, processors would need to consider a price incentive plan for quality as well as Total Solids (fat plus SNF). Some suggestions of activities for the MOD project staff to consider to improve milk quality would include: • Implement an on-farm training program on good milk practices (animal care, site improvement, equipment, personal hygiene) and farmers use California Mastitis Test, and keep herd records; • Introduce nutritious feeding practices by producers throughout the herd life of the dairy cow; • Introduce innovative improvements in the on-farm storage and transport of milk to the collection point; and • Establish set milk times for delivery and pick-up so that milk arrives at the collection point at prescribed time for pick-up to minimize time from farm gate to chilling center. For measurement of this indicator for improved quality, the evaluation team concurs with the MOD staff’s recommendation that they train selected beneficiaries on hygienic milking including improved nutritious feeding and transport practices, and MOD will evaluate improvements based on premium prices paid for quality improvements. For the baseline period, the price premium is zero because the interventions and activities have not begun. By the end of the project, 80% of beneficiary farmers will be getting a quality￾based price premium from their baseline values. Data will be obtained through processors’ payment information/semi-annual survey and compared against the baseline value and if the value is higher the farmer would be counted as having obtained a price premium. As premium pricing is a composite of many quality attributes, it will serve as a practical quality indicator across the industry. An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 9 1. INTRODUCTION TO BASELINE EVALUATION 1.1. Summary of the Baseline Survey The MOD project (FY2017 Food for Progress Proposal No. 2017-0018: Sri Lanka Market-Oriented Dairy MOD) Project is a five-year USDA funded project implemented under the USDA’s Food for Progress Cooperative Agreement No. FCC-383-2017/031-00. MOD seeks private enterprise solutions to poverty reduction in Sri Lanka. The project employs a mix of technical assistance and volunteer expert support to assist stakeholder groups in the dairy value chain including producers, cooperatives, processors, financial institutions and government departments to promote growth in the dairy sub-sector. The project will target 5,4000 dairy producers during the life of the project (LOP). Milk production is important in the economy of Sri Lanka. The estimate of domestic production was approximately 454.6 million liters (l).The domestic milk supply in Sri Lanka increased from 2015 to 2016 (12.7%) which helped to hold the imports of milk and milk products to around 62% of milk consumed in Sri Lanka. For 2015, domestic production was 39% of total milk disappearance; and in 2016 the percentage fell slightly to 38.5% (See Table 1.1.) In 2016, the total availability of milk in Sri Lanka increased by 14.3% to 1,182.3 million Liters, and consumption per person was 55.5 liters per person (l/p) (DAPH, 2017). Separate industry estimates place domestic production at 380 mil. liters, and imports at 600 mil. liters on a milk equivalent basis (Pathumsha). The total supply is less than the amount reported by the DAPH. The annual average increase in domestic milk production has been around 5 percent per year based on herd growth and yield improvements from better livestock genetics. Table 1.1. Quantity of milk production and milk imports (liquid milk equivalent) in 2015 and 2016. 2015 2016 Percentage Change Cow Milk (liters) 331,197,597 377,972,997 14.1% Buffalo Milk (liters) 72,032,763 76,655,144 6.4% Total Domestic Production (liters) 403,230,360 454,628,141 12.7% Total Dairy Imports (LME) (liters) (1) 631,602,120 727,745,700 15.2% Total Milk Consumed in SLK (liters) 1,034,832,480 1,182,373,841 14.3% Domestic Production as Percent of Milk Consumption (%) 39% 38.5% Note 1. LME is liquid milk equivalent (number of liters of milk to produce one kg of powder) Source: Department of Animal Production and Health, 2015 and 2016 The dairy industry is comprised of small-scale producers, though there has been recently increases in the numbers of medium and large-scale dairies supported through GOSL and the Government of Australia (GOA). The dairy cattle population was 1,366,195 head in 2016 and the number of registered dairy farms was 314,725 having an average of 4.3 heads per household. For buffalo production, the numbers of buffaloes were 426,257 heads and the number of registered farms was 26,868 for an average of 15.86 heads per household. Of the 454.6 million liters of milk produced in 2016, the majority of milk was from dairy cattle (83 percent) and the remainder (17%) from buffaloes (DAPH). The volume of milk entering the formal milk market in 2016 was approximately 230.7 million liters; and the remaining milk was channeled through informal routes and consumed by households. The formal supply chain is described as milk that passes through Farmer Managed Societies (FMS), cooperatives, milk collectors, collection points and chilling centers and delivered to milk dairy companies for processing into An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 10 an array of products for the local market: fluid, powder and dairy products (e.g., yoghurt, cheese, butter, etc.). Milk will be tested as it moves through the supply chain. Finished dairy products are distributed to wholesalers and retailers for sale to consumers. The informal milk supply channel is milk that does not flow to commercial processors and can be unpasteurized milk that is sold to final consumers. The informal supply channel supports about 20 percent of the milk sold. The GOSL set standards for milk products in the Food Act. The Government of Sri Lanka (GOSL) does not strictly enforce its product standards. However, the GOSL does place a price ceiling on the retail price of milk powder to protect consumers against rising international prices.1 GOSL will arbitrarily adjust the price ceiling without clear guidelines. Industry representatives said that some exporters will sell powdered milk below the world price for a variety of reason, e.g., milk is past sales’ date or inventories of milk have to be reduced. A trade specialist in Colombo discovered powdered milk in 25 kg bags in warehouses with a price of LKR500 which is below the world price. This milk is probably for the food manufacturing industry because of its low quality. The unregulated supplies of imported powdered milk can dampen the price paid by processor for fresh milk at the farm gate. 1.2. Objectives of the Baseline Review The evaluation team addressed three key questions as part of the baseline evaluation. ➢ What is the current volume of milk (in liters) sold by project beneficiaries? ➢ What is value of sales by project beneficiaries (in USD)? ➢ What is the percent reduction in rejections of milk for improved quality at collection points from beneficiary farmers? The evaluation team examined the performance indicators estimated prior to the start of the MOD program. Both quantitative and qualitative data were collected from the following groups of stakeholders: dairy producers (whose herd sizes ranged from three to over 20 milking cows per household), collection center managers and employees, farmer cooperative managers, artificial insemination technicians, and veterinary staff. The team targeted milk producers with the potential to become medium-scale fodder producers – seen as entrepreneurs willing to invest in the dairy value chain, as well as banks, MFIs, and other financial institutions. The team identified only few commercial fodder producers, but the demand for fodder is high especially among producers participating in the GOSL-GOA program. The evaluation team realized the importance to address methodology issues in monitoring the project’s outputs, outcomes and impacts. 1.3. Target Areas of the MOD activities The evaluation team identified six provinces for dairy improvement: North Western, Northern, North Central, Central, Eastern, and Uva (See Table 1.2.) These provinces have the greatest potential for increased commercial production of milk. Some of the provinces have both large numbers of dairy farms and a dairy processing unit. The six provinces differ in agro-ecological conditions because of altitude, climate, and forage systems. The highest concentration of milk production and collection is in the Central, North Western, and North Central Provinces. The implications for the project is that those provinces and districts within provinces with favorable agro-ecological conditions hold the greatest potential for transitioning producers to commercial, modern dairy businesses. 1 Mendes, S.S. et al. (2014). “Milk Powder Imports and government policies: the case of Sri Lanka.” Journal of Agricultural Economics and Rural Development. 2 (3), p. 86-91. An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 11 Table 1.2. Number of dairy cows and cow milk production by provinces, 2016 Province Milking Cows (no. of heads) (1) Not Milking Cows (no. of heads) (2) Other Cows (no. of head) (3) Total Dairy Cows (no. of heads) Milk Production (liters) Milk Collection (liters) Central 34,523 19,699 7,261 61,483 121,557,887 74,675,407 Eastern 91,534 51,459 44,818 187,811 39,716,144 20,133,225 North Central 44,509 36,412 12,026 92,947 26,708,736 28,876,360 North Western 50,647 47,723 33,776 132,146 61,224,104 43,206,014 Northern 73,583 51,025 12,897 125,362 28,293,027 17,831,200 Uva 22,833 15,592 16,269 54,694 43,600,029 19,163,507 Sri Lanka 367,124 251,802 138,948 757,874 377,972,997 230,744,363 Note 1. Milking at present Note 2. Not milking (dry) at present Note 3. Other (infertile/aged) cows Source: Livestock Statistical Bulletin – 2016, Department of Animal Production and Health. In the target provinces, the Central Province has the greatest percentage of producers operating intensive production systems (See Table 1.3.) An intensive dairy system is where the cow is restricted to a barn or yard and feed is brought daily to the animal. The North Western Province has the highest concentration of semi-intensive production. Semi-intensive is when the dairy cow spends some time grazing during the day away from the milking barn and housed in the barn at night. The period of time on pasture can vary by available grazing land. The North Central, Northern, and Eastern Province have the largest percentage of farms in extensive production (see Table 1.3.) The cows in an extensive system will receive 100% of feed from grazing and animals may remain on pasture overnight or may come into a secure yard for milking before returning to the grazing area. Cows may be milked in the grazing area. This system is low cost and low milk productivity. Table 1.3. Dairy systems in each key dairy province Province Production Systems Percentage of Farms in the Management System Description Central Intensive 41% North Central Semi-intensive 58% North Western Semi-intensive 62% Northern Extensive 46% Eastern Extensive 59% Uva Intensive 19% Source: Production systems are from the Livestock Statistical Bulletin – 2016. 1.4. Profile of Dairy Farmers MOD’s target population is for dairy herd sizes of 10 to 15 dairy cattle. The current conditions are that producers’ cows are yielding around 6-7 liters per day (l/d) with cross-bred animals with a lactation An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 12 period ranging from 180 to 260 days. Milk production per lactation would range from 1,080 liters in the drier zones to 1,820 liters per lactation in the wetter hill areas. Forage production is from the producer’s average landholding of .5 to 1.5 acres. Producers will feed concentrates supplied by the milk processors or an agribusiness feed company. Milk processors will supply animal feed to producers and then deduct the costs from their milk check. Most producers only give concentrate to cows that are in milk production. 1.5. Dairy Imports Over 60% of dairy consumption is from imports. From 2015 to 2017, the value of milk and milk product imports have increased 41.2% to LKR 48,145 million (US$316,746,598). The largest single item in volume and value of milk products has been powdered milk (See Table 1.4.) Milk powder was 93%, 91% and 93% of the value of dairy imports in 2015,2016, and 2017. respectively. The opportunity for substituting dairy imports with domestic milk is very evident. Table 1.4. Quantity and value of imports of dairy products, January to December 2015, 2016, and 2017 Product Category Quantity (MT) - 2015 Value (LKR.Mn.) 2015 Quantity (MT)_ 2016 Value (LKR.Mn) 2016 Quantity (MT)_ 2017 Value (LKR.Mn) 2017 Value Change 2015 - 2017 Total Milk and Milk Products 86,327.48 34,087.85 99,593.43 36,338 98,863.91 48,145.48 3 41.2% Milk and Milk Cream 292.93 116.15 431.26 169.18 490.40 239.053 106% Milk cream fat <1.5% 9,271.19 3,362.77 9,436.67 2,790 8,578.10 3,031.80 -9.8% Milk cream fat >1.5% 72,487.82 28,479.04 84,578.45 30,846 84,548.90 41,849.60 46.9% Condensed Milk 49.71 8.57 76.47 10.42 125.56 12.78 49.1% Cheese and Curd 2,361.30 1,626.19 2,521.83 1,755.2 2,866.04 2,168.59 33.3% Butter and other fats 544.97 287.84 996.19 561.3 683.39 572.39 203.8% Butter milk and curdled milk 25.78 9.87 28.09 7.5 66.96 24.07 143.9% Whey and whey powder 1,293.78 197.42 1,524.00 197.00 1,504.56 247.2 25.2% Source: Sri Lanka Customs, Government of Sri Lanka An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 13 2. EVALUATION PLAN AND METHODOLOGY FOR BASELINE STUDY 2.1. Evaluation Plan and Activities The design of the baseline evaluation was to conduct interviews with many stakeholders, starting with the senior management of dairy processors. Processors provided information on their current and some future investments in collecting and manufacturing dairy products. After completing the interviews of senior managers in Colombo, the evaluation team proceeded to the field to conduct interviews with producers, operators of collection points and collection centers, and other stakeholders in the dairy value chain. Step 1. The team collected information from secondary sources on the Sri Lanka dairy sector (see the reference materials listed in the annex 7.1.). The team gathered information on past dairy projects funded by USAID and other donor projects. Step 2. Before the arrival of the international consultant, a working session on skype was held with IESC project staff to discuss the nature of the baseline information for collection during the upcoming field trip. Step 3. The evaluation team held an introductory meeting upon the arrival of the international consultant in Sri Lanka with the MOD team. The meeting focused on the key baseline indicators. The team discussed and agreed upon the time period for the field work: Week 1. Conducted meetings in Colombo with the key dairy processors and other stakeholders in the dairy value chain and conducted a field trip to one nearby province to collect data on production, collection and processing of milk. The team tested and refined its survey instruments. Week 2. Traveled to the field to conduct focus group discussions with stakeholders in the dairy value chain in the provinces and districts, e.g., government agency representatives, producers, collection center and chilling center operators, milk collectors and processors, with the purpose to collect data specific to the indicators; and Week 3. Held debriefing meeting with MOD staff and prepare draft report in Colombo. Step 4. In preparation of the field investigations, the team prepared field interview guides for key stakeholders (see Annex 7.3.). The evaluation team prepared interview guides employing a mixed methods approach for collection of both quantitative and qualitative data. Step 5. The team started the field trip to Pannala in the Kurunegala district of the North Western Province, followed with Kandy and Matale districts in the Central Province, and proceeded to Anuradhapura district in the North Central Province and to Kilinochchi, Jaffna, and Mullaitivu in the Northern Province. The team returned from the Northern Province through Trincomalee and Batticaloa in the Eastern Province and visited dairy stakeholders in Ampara district. The team interviewed stakeholders in the plantation areas of Badulla district of the Uva Province and the Nuwara Eliya District. The team spent nine days in the field in target zones. The international consultant had regular phone conversations during the field trip with the Chief of Party (COP) of MOD. Step 6. The team held a debrief with the MOD staff and presented findings in a PowerPoint presentation after the team’s return from the field interviews (see Annex 7.11). Step 7. The team met with the Monitoring and Evaluation Specialist of MOD to review aspects of the monitoring of indicators. An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 14 Step 8. The team presented a draft report to MOD staff before departure of the international dairy consultant. Step 9. The evaluation team received and reviewed the comments from the IESC team. Step 10. The MOD staff received the final report after making necessary corrections. 2.2. Methodology and Tools The baseline evaluation team started the field work on January 26 and concluded interviews on Saturday, February 3, 2018. The MOD project is based on a close working relationship with the commercial dairy processors. Consultations were on-going between MOD staff with six to eight major processors on how best to target MOD activities to support the increased commercialization of the dairy industry. The team’s selection of stakeholders to interview was predicated on the team’s initial meetings with the processors. The processors identified their zones of activities, collection points and chilling centers and key staff in the field that would be knowledgeable about the area to assist the team. In addition, the national consultant was instrumental in setting up contacts in the government sector with Department of Animal Health and Production (DAHP) and with project leaders of dairy improvement projects on-going in Sri Lanka. Working with processors with their links to chilling centers and collection points, the evaluation team was able to focus on key dairy areas and stakeholders which helped the evaluation to be more effective. Some dairy areas have more than one processor operating so the team tried to interview dairy producers aligned to several processors’ collection points. The numbers of interviews by types and locations are presented in Table 2.1. Table 2.1. Numbers of people interviewed by type and geographic locations Location/type Producers Collecting￾Chilling Processors Government representatives Others - type Number Colombo 16 Central 15 8 3 1 Donor project rep 2 North Central 2 1 1 1 Chamber of Commerce 1 Northern 6 2 4 2 North West 2 1 6 3 Eastern 3 3 8 3 Uva 2 2 4 Total Persons 30 15 43 14 The evaluation team engaged in a participatory approach with stakeholders by holding both key informant interviews (KIIs) and focus group discussions (FGDs) with a number of stakeholders to get their comments on the dairy value chain and the issues regarding the indicators of importance to the evaluation. Focus group discussions were held with groups of producers either at the collection points or on farms with one or more producers in attendance. The evaluation team applied a systems approach to the dairy value chain to better address the specific indicators for the baseline analysis (see PowerPoint presentation in Annex 7.7.). A systems approach is important in an evaluation to understand how market forces and changes in input costs and market prices An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 15 impact on productive herd life (number of lactations per cow) and milk production.2 The impacts of global dairy prices has “knock-on” effects on the herd because of the large dependence of milk powder for both milk processing, food manufacturing, as well as for the consumer market. 2.2.1. Mixed methods The evaluation team used mixed methods, collection of both quantitative and qualitative date on the current state of the dairy industry. 2.2.1.1. Quantitative data The team collected quantitative data from secondary sources on the production of milk and dairy products (see Table 2.2 and sources of data in Annex 7.1.) The data provided a time series on changes in herd sizes and composition of the dairy herds by provinces and districts. The team conducted interviews with stakeholders (public and private sectors) in the dairy value chain to collect data on production, costs, and prices. Table 2.2. Cows in milk for cattle and buffaloes and collection centers by top six dairy districts, 2016 Region District Cows in Milk Buff in milk Collection Centers (numbers) Chilling Centers and Chilling Capacity heads heads 2015 2016 2017 North West Kurunegala 33,526 6,929 786 870 791 #36 ; 101,750 liters North Central Anuradhapura 26,524 10,973 526 576 619 #29 ; 89,600 liters Central Kandy 12,020 1,751 326 284 429 #16 ; 72,300 liters Central Nuwara-Eliya 14,638 501 200 207 233 #42 ;142,940 liters North Central Polonaruwa 17,985 6,894 216 206 227 #5 ; 50,400 liters North West Puttalam 17,121 942 188 220 203 #16 ; 54,950 liters Other 245,310 90,115 1,188 1,138 1,135 #140 ; 396,503 liters TOTAL 367,124 118,105 3,430 3,503 3,637 #284 ; 908,443 liters Source: http://www.statistics.gov.lk/agriculture/Livestock/MilkCollectingProducionCenters.html The total number of milk collection centers points increased each year from 2015 to 2017. There are 3,637 collection points in 2017. Overall there was a net gain in collection centers during the 2015 to 2017 time period. An increase in collection points means that producers have more access to collection points and more milk can enter the formal milk channel which is important to the modernization of the dairy industry. Cooperatives and milk processors have chilling centers, and there are 284 chilling centers in 2016 and 287 chilling centers in 2017. Dairy processors carry out dairy extension activities from these chilling centers. 2.2.1.2. Qualitative data The team interviewed both key informants (KIs) and stakeholders in focus group discussion (FGDs) in the dairy value chain (see Annex 7.4. of meeting notes.) The qualitative data allowed for understanding of the issues around dairy herd management not revealed in the annual statistics. 2Scholtz, M.M. and S.M. Grobler. (2009). “A systems approach to the South African dairy industry.” South African Journal of Animal Science. 39 (Supplement 1). An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 16 2.2.2. Data collection tools The team prepared interview guides to assist in directing the interviews to include necessary information (see Annex 7.3.) The team used both close-ended questions and open-ended questions to explore behavioral and cultural factors in producing, keeping, and marketing milk to better understand issues affecting baseline numbers. 2.3. Lessons Learned/Key Things to Monitor Going Forward As part of the evaluation, the team examined some of the factors that would impact the evaluation going forward for the MOD project team. These factors are presented in the PowerPoint presentation materials prepared for the team’s debrief after the conclusion of the field work, and in the observations in Annex 7.8. An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 17 3. INDICATOR #1. VOLUME COMMODITIES SOLD BY PRODUCERS 3.1. Indicator and Major Assumptions This first indicator to evaluate was the baseline estimate of current level of milk production for 5,400 producers expected in the MOD program. The project activities plan are to increase milk production above this baseline figure. The evaluation team chose to address production from three different perspectives to triangulate across different assumptions and based on observations from the field trip. By triangulating using three scenarios a more accurate measure of the current level of milk production accounting for the variation in beneficiary practices and in the type of production practices encountered by the MOD project. Several assumptions are important in the calculation: the provinces, districts within, and the area within the districts as likely target areas for MOD (Table 3.1.) Each area in a province had estimates of production coefficients based on whether the dominant practices were intensive, semi-intensive and extensive (Table 3.2.). The production coefficients were number of milk cows per herd, milk yield per cow, and number of days of lactation. Table 3.1. Provinces, districts and percent of area coverage for commercial production Province District - % of area District District Central Kandy – 100% Matale – 100% Nuwara Eliya – 100% North Central Anuradhapura – 40% Polonnaruwa – 40% North Western Kurunegala – 90% Northern Jaffna – 75% Kilinochchi – 50% Vavuniya – 50% Eastern Ampara – 50% Uva Badulla – 90% Source: selection of districts and percentages of area within a district by the national dairy consultant 3.2. Target Milk Production Areas The team chose to provide in-depth analysis by identifying those provinces and districts representing commercial milk production. Commercial milk production are producers raising dairy cows to produce milk for daily sales. Production is assumed under three production systems and there are two areas for each type of herd management set-up: • Intensive – dairy animals restricted in movement to the milking barn and possibly to a small loafing area near the barn for resting when not milking. The producer provides feed and forage to the animal. Intensive production areas are Central and Uva Provinces. • Semi-intensive – dairy cows are released for grazing outside the barn and loafing area. Animal graze grass and forbes and then return to the barn in the evening. Semi-intensive areas would be North Central and North Eastern Provinces. Cows would be milked at the homestead. • Extensive – dairy animals graze on pasture for extended periods of time to meet nutritional requirements. These areas are drier and would be Northern and Eastern Provinces. Cows may be milked while on continuous pasture. Some districts in the provinces are not intensive dairy production areas so the specialists. The evaluation team chose to focus on those areas with higher potential for dairy producers. The MOD project staff may choose to include other districts not visited by the survey team. Table 3.1. has a list of provinces and those districts which would be likely target areas for the MOD project and its beneficiaries. The evaluation team choose these districts as being commercial dairy production areas for estimating the current level of milk production for the baseline. The second adjust was that in a district not all the area would be suitable for An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 18 commercial production because of agro-climatic conditions. A percentage of area was assigned to each district (see Table 3.1.) 3.3. Production Coefficients to Estimate the Baseline Volume of Milk The targeted provinces and districts have varying agro-ecological conditions, dairy production systems, and composition of cattle types. The key technical coefficients for a district were: average number of dairy cows per herd, milk production per day (l/d), and length of the lactation period (See Table 3.2.) These production coefficients vary based on whether the production system is intensive, semi-intensive or extension production systems. Table 3.2. Technical coefficients for milk production by province and production system Province Production Systems milk cows/herd yield l/h/d Lactation period Description Head liters days Central Intensive 6 6 240 North Central Semi-intensive 5 5 200 North Western Semi-intensive 6 6 200 Northern Extensive 6 3 200 Eastern Extensive 10 2 180 Uva Intensive 5 6 240 Source: Production systems are from the Livestock Statistical Bulletin – 2016. 3.4. Three Dairy Production Models Three models are developed to triangulate on what is the high and low range for milk production to corroborate the estimate of the baseline estimate. The types of production systems (intensive, semi-intensive and extensive) and producers with exotic and cross-bred dairy cattle will vary across the MOD’s beneficiaries. The evaluation team presented three scenarios for beneficiaries to provide scenarios in estimating milk production. 3.4.1. Milk production (Model 1) The total gross volume of milk produced in the six target provinces gives an indication of the current level of milk production in these priority areas. The milk production estimate is 227.3 million liters per year for the target areas and data is from the Livestock Statistical Bureau of the DAPH for the priority districts. The amount of cow’s milk produced in each province is used as the first filter for estimating the allocation of the 5,400 producers that MOD will target. The number of targeted producers by MOD in each province is based on the gross milk produced in that province compared to the total for the six provinces (see Table 1.2. and Table 3.1.). The relative gross levels of milk production give an estimate of the existing conditions affecting milk production. Producers were allocated to each province (see 3.3.) and milk production estimated based on producers’ technical coefficient (Table 3.2.). For Model 1, the baseline estimate of milk production is 40,762,000 liters (See Table 3.3.). Because this model is based on total volumes of milk by provinces and districts, then more producers are allocated into the Central and North Western Provinces which have higher milk production based on the target area so combined with production coefficients on yield and lactation length. An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 19 3.4.2. Intensive dairy production (Model 2) MOD may prefer to target producers who are currently practicing intensive dairy production, e.g., stall fed with cut and carry of forages plus concentrate. In the six provinces and their selected districts, the percentage of producers operating intensive operations varies. The data is from the Statistical Bulletin. The result is that the number of producers selected in each province vary. Based on different levels of intensification, the team’s estimate of the baseline of current production is 35,800,000 liters (see Table 3.4.) The description of intensification in this model is for dairy cows that are residence in the barn or an adjacent loafing area, and dairy cows do not go out to graze. Feed is brought daily to the cow. All three districts in the Central Region are targeted as intensive dairy production so the largest amount of milk would be sourced from this province. Table 3.3. Model 1. Estimate milk production based on technical coefficients and number of dairy cows in province Dairy Producers Cow Milk Production % of Tot Producers Est. Baseline Production Farm Price of milk Value of Milk LKR Value of milk in USD Province No. liters % No. mt l/d Rp US$ Central 38,940 121,557,887 0.5347 2887 24946 63 1,571,614,009 10,339,566 North Central 32,069 10,683,494 0.0470 254 1,269 63 79,934,146 525,883 North Western 45,890 47,912,706 0.2107 1,138 8,194 63 516,216,882 3,396,164 Northern 48,656 12,371,445 0.0544 294 1,058 63 66,645,668 438,458 Eastern 68,228 7,717,965 0.0339 183 660 63 41,577,111 273,534 Uva 43,265 27,101,448 0.1192 644 4,635 63 291,994,048 1,921,013 277,048 227,344,945 100 5,400 40,762 2,567,981,864 16,894,618 Notes: MOD target producers allotted to six provinces based on the ratio of milk produced to the total milk Table 3.4. Model 2. Milk production estimated based on the number of farms that are intensively managed Farm Price of milk % No. mt LKR/l LKR US$ Central 0.39 2106 18,196 63 1,146,337,920 7,541,697 North Central 0.04 216 1,080 63 68,040,000 447,632 North Western 0.08 432 3,110 63 195,955,200 1,289,179 Northern 0.13 702 2,527 63 159,213,600 1,047,458 Eastern 0.16 864 3,110 63 195,955,200 1,289,179 Uva 0.20 1080 7,776 63 489,888,000 3,222,947 1.00 5400 35,800 2,255,389,920 14,838,092 Note: Weight % is the percentage of farms with intensive management system, Livestock Stat. Bull, 2016 Province % of farms with intensive system # of farms in MOD target practicing intensification Est. Baseline based on # farms Farm Value of milk Farm Value of milk An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 20 3.4.3. Farms with improved European and Cross-Bred Dairy Cattle (Model 3) The producers with the best opportunity for transitioning to commercial dairy operations during the MOD project are those producers currently with European and cross-bred dairy cattle. These producers will have greater quantities of surplus milk to sell in the formal milk channels and set up a regular sales program with processors and received needed technical assistance. Based on interviews with producers at their dairy barns, those producers with European and cross-breed cattle exhibited more progressive management of their dairy cattle. Producers with higher yielding cattle seem better aware of the care that these higher producing dairy cattle require. Using these estimates for dairy cows in the herd, the estimate of the baseline milk production for the 5,400 producers allocated to the six provinces is 35,105,000 liters of milk (Table 3.5.) 3.4.4. Estimate of the baseline milk production Three models were developed to better estimate the current baseline for milk production: allocation of producers to provinces by milk production (model 1), allocation of producers by level of intensification (model 2), and allocation of producers by prevalence of exotic and cross-bred dairy cattle (model 3). The average based on the three models for the current baseline of milk volumes is an estimated 37,222,000 liters (Table 3.6.) Table 3.6. Weighted calculations of milk volume in the baseline Model 1–Milk produces Model 2 – Intensive Model 3- Exotic and Cross-Bred Cattle Average of the Three Models Volume of milk (L) produced 40,762,000 35,800,000 35,105,000 37,222,000 Value of milk produced (LKR) 2,567,981,864 2,255,389,000 2,211,597,000 2,344,989,288 Value of milk - USD US$16,894,618 US $ 14,838,092 US $14,549,980 US$15,427,561 Note: exchange rate for conversion was LKR152 per USD at the time of the evaluation for converting LKR to USD. Table 3.5. Model 3 - milk production based on farms with European and Cross-Breed Cows No. % no. % no. liters ('000) LKR/l LKR ('000) US$ Central 38,940 0.76 29,594 0.236 1,275 11,020 63 694,268 4,567,554 North Central 32069 0.36 11,545 0.092 2,488 498 63 156,734 1,031,142 North Western 45890 0.53 24,322 0.194 1,048 7,547 63 475,478 3,128,145 Northern 48656 0.36 17,516 0.140 2,718 755 63 171,216 1,126,424 Eastern 68228 0.17 11,599 0.093 1,800 500 63 113,375 745,889 Uva 43265 0.71 30,718 0.245 1,324 9,532 63 600,526 3,950,827 Total 277,048 125,294 1 5,400 35,105 2,211,597 14,549,980 Note: Weight % is the percentage of farms with intensive management system, Livestock Stat. Bull, 2016 Farm Price of milk Farm Value of milk Farm Value of Province milk Registered Farms Farms with Eur Breed x cross Registered farms with Improved stock Percent of Farms in the populatio n by province Project farms by Province Baseline milk production An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 21 4. INDICATOR #2. VALUE OF SALES BY PROJECT BENEFICIARIES 4.1. Indicator and Major Assumptions The second indicator for the evaluation team was the value of sales by project beneficiaries in USD. This indicator builds upon the estimate of the volume of milk currently produced by the 5,400 beneficiaries for Indicator #1. The assumption for this indicator is that beneficiaries of MOD interventions currently sell their milk in the formal milk supply chain or will sell their milk in the formal milk supply chain. This supply chain is different than the sales through the informal system which would include sales to independent bulk milk collectors who collect and hawk milk to processors and others. This would also exclude producers who sell surplus milk to neighbors or direct to other outlets. This excludes producers who produce milk just for home or neighbor consumption. The Government of Sri Lanka (GOSL) establishes a suggested price for producers’ milk based on the fat percentage, milk’s solid non-fats (SNF) and Total Platelet Count. The price schedule sets a minimum price (floor price) for milk measured on fat and SNF. Generally, all processors follow the government’s recommended pricing chart at minimum. They may pay extra to attract farmers into the network and tailor based on their specific quality needs (low bacteria, high fat, etc.) (personal communication with COP of MOD.) The conversion of value of milk in LKR to USD was based on the current open market exchange rate at the time of the evaluation. 4.2. Findings Based on data collected from key informant interviews (KIIs) and focus group discussions (FGDs), the price of milk received by most producers in the districts visited ranged from LKR 60 to LKR 66 per liter based on interviews conducted with producers intercepted at collection points and chilling centers in the formal market channel. Producers who sell outside the formal channel could get lower prices because milk is bought by collectors without being tested. The producer accepts a lower price because bulk collector will accept milk of lower quality without testing and hawk the milk to its network of buyers – which may or may not be commercial processors. In the formal supply channel, the benchmark price is based on the GOSL price sheet. The price is set on the levels of fat and Solids-not-fat (SNF), which when added together gives the total solids (TS) in the milk. For a TS value of 12.5% the Milco price is LKR 66 per liter. The average price received by producers is because most dairy cattle do not receive the required amount of nutrients for their age, lactation period, and milk production. In field interviews, producers said the major constraint was the availability of affordable feed. Some producers of cow’s milk will receive higher prices, even in the range of LKR 70 to LKR 75 per liter because they have retained long standing as good farmers because of their quality. This was not the general case for most producers interviewed. Reasons producers receive prices over the LKR 66 per liter was because they have milk that exceeds the minimum quality because of better facilities, better feeding of cows, closer to the chilling center so milk is delivered in a timely manner with lower bacteria counts and have higher volumes of milk because of the larger herd sizes. Over the lactation period, the TS of the milk will change and the price per liter will vary for an individual cow as it reaches the end of its lactation curve. In addition, producers who sell to bulk collectors receive lower prices, LKR 60 per liter or less. Bulk collectors accept farmers’ milk without testing so quality can be lower compared to milk sold through the formal collection points – collection-chilling center system set-up by processors. The percent of milk collected by bulk collectors from farmers and sold to processors varies by processors. The MOD staff An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 22 estimated that 20% of milk goes through the informal supply chain. In interviews with some processors they said they have stopped purchasing milk from bulk collectors because of the milk’s lower quality and higher rates of adulteration. Producers not selling currently in the formal market system would not likely be in the target population of MOD beneficiaries. In Table 3.3. above, the range in the baseline value for raw milk at the farm gate varies from LKR 2.211 billion to LKR 2.567 billion for 5,400 producers in the baseline estimate. The average value of the baseline milk production for the 5,400 project producers is LKR 2.344 billion (US$ 15,427,561) for a baseline volume of milk of 37,222,000 liters of milk. The preliminary project baseline estimate was US$ 21,926,979. The difference in the baseline value of milk at the farm level is because of the lower average price of LKR 63 per liter. The evaluation team used an exchange rate at the time of the evaluation of LKR to USD of LKR152 per USD. An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 23 5. INDICATOR #3. REDUCTION IN REJECTIONS FOR QUALITY AT COLLECTION POINTS 5.1. Indicator and Major Assumptions The baseline indicator for measurement by the evaluation team is: ➢ Percent reduction in rejections for quality at collection points and at milk chilling centers from beneficiary farmers. The current proposed MOD indicator for quality improvement is the percent reduction in rejections by dairy processors. The team found that the rejection rate is very low in the formal system for milk passing through collection points and chilling centers due in part to processors’ extension programs to its producers. Processors will accept milk of varying quality because they manufacture cultured products where the quality expectation is less than for milk used for UHT. The evaluation team suggests another choice of measurement of the indicator for the improvement in quality of milk. Processors realize that milk quality can be improved from its current condition which would mean better quality finished products. There is room for improvement of milk quality and there is a way to monitor the change in the improvement in milk quality. Milk quality can be defined from the perspective of different stakeholders in the dairy value chain (FAO). i) Milk Producer. The milk producer expects a fair price in accordance with the quality of milk she/he produces. ii) Milk Processor. The milk processor who pays the producer must assure himself/herself that the milk received for processing is of normal composition and is suitable for processing into various dairy products. iii) Consumer. The consumer expects to pay a fair price for milk and milk products of acceptable to excellent quality. The baseline indicator needs to reflect a measurement that is going to reflect the need for milk quality starting at the producer-first handler exchange, e.g., collection points and at the milk chilling centers. 5.2. Current Situation In interviews with processors, chilling center operators, and producers, the evaluation team found a low rate of rejection of milk except if bulk collectors purchased milk from producers and delivered to processors or from cooperative societies operating on behalf of their members. Several processors stated that in recent years they have worked with producers to improve their herd management practices with dairy extension￾development officers. Because milk quality has improved through their outreach, some processors have stopped purchasing from bulk collectors altogether because of the low quality and adulteration of their milk. The processor’s representatives said the rejection rate is low for milk in their formal system of collection points and chilling centers. The opinion of the evaluation team is that the initial indicator, as it is stated, is not a meaningful measure of quality for MOD. An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 24 Because processors do not pay on a quality basis (just TS), there is no incentive for producers to improve quality to further reduce the rate of rejections. Some processors are testing for “keeping quality”, but it is not done by all processors at the same point of milk testing. Tests done by most processors at the collection point or chilling center are the organoleptic tests, lactometer reading, Gerber test for fat, the keeping quality test with Resazurin Test (RT), and the alcohol test (at varying levels of concentration) for adulteration. Processors that produce UHT milk have certain collection locations where they receive higher quality milk and the alcohol level will be set at 75% or higher at these locations. The ideal situation for producers and processors is for the quality of milk to be coupled with the price paid for fat and SNF. GOSL has definitions and standards for Cow’s Milk under the Food Act. The requirements for additives and labeling are included in the regulations.3 Following categories of Cow’s Milk are included: 1. Milk/Liquid Milk 2. Raw or Fresh Liquid Milk 3. Standardized Milk 4. Semi-skimmed Milk or Low Fat Milk 5. Skimmed Milk or Non Fat Milk 6. Pasteurized Milk 7. Sterilized Milk 8. Ultra-Heat-Treated Milk 9. Flavored Milk 10. Recombined Milk 11. Reconstituted Milk 12. Reconstituted Skimmed Milk 13. Toned Milk 14. Lactose Hydrolyzed Milk The reduction in the rejection rate of milk by processors is not an indicator that will be useful for measuring quality improvement made by MOD in the milk supply chain because the current rejection rate is below 1% due to milk being directed for other use without rejection, such as cultured products which does not have the same quality standards. 5.3. Possible Measure of Improvement in Milk Quality 5.3.1. Test for quality The current level of milk quality needs improvement. The industry needs better quality milk to have the opportunity to develop new dairy products with good flavor and taste and which have longer shelf life. These are necessary conditions if the industry is going to compete with the imports of powder milk. There are chilling centers where processors use the alcohol test at 75% concentration for testing adulteration of the milk. Farmers supply these centers with milk to be purchased for the manufacture of UHT milk. There are other collection centers by companies that use a rate of 68% alcohol and use that milk for manufacturing cultured products. With improved quality of milk, processors would be able to use higher alcohol test so that consistent quality raw milk can be used to produce better cultured products, new products requiring higher quality, and ultimately compete with imported powdered milk. 3 See Food Act. http://203.94.76.60/FOODWEB/files/regulations/draft/milk_milk_products_regulations.pdf An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 25 It is important that milk quality improve at the farm level by demonstrating to farmers what is their milk quality. Producers can expect to receive a fair price for the quality of the milk they sell (See FAO definition above.) For this, the processing companies use a variety of tests for measuring milk quality (see Table 5.1.) The industry does not have a standardized protocol of tests due to the fact that the quality needs of each processor varies. For this reason, processors will conduct different tests depending on the milk products they produce and the cost of administering a test. The price the producers receive will be indicative of improved quality milk. In addition, MOD has not yet identified the farmers they plan to work with in outreach and training. It is therefore difficult to come up with a general baseline for the quality tests. Table 5.1. Quality tests performed by processors at different stages in the formal milk supply chain Processors Collection Point Chilling Center Processing Plant Processor #1 Lactometer – YES Alcohol Test – YES, 75% Resazurin - NO Gerber Test - NO Organoleptic - YES Lactometer – YES Alcohol Test – YES, 75% Resazurin - YES Gerber Test - YES Clot-on-Boiling - varied Lactometer – YES Alcohol Test – YES, 75% Resazurin - NO Gerber Test - NO Random tests for specific adulterants Processor #2 Lactometer – YES Alcohol Test – NO Resazurin - NO Gerber Test - NO Organoleptic - YES Lactometer – YES Alcohol Test – YES, 85% Resazurin - NO Gerber Test - YES Lactometer – YES Alcohol Test – YES, 85% Resazurin - NO Gerber Test - YES Random tests for specific adulterants Processor #3 Lactometer – YES Alcohol Test – NO Resazurin - NO Gerber Test - NO Organoleptic - YES Lactometer – YES Alcohol Test – YES, 75% Resazurin - NO Gerber Test - YES Lactometer – YES Alcohol Test – YES, 75% Resazurin - NO Gerber Test - YES Random tests for specific adulterants Processor #4 Lactometer – YES Alcohol Test – NO Resazurin - NO Gerber Test - NO Organoleptic - YES Lactometer – YES Alcohol Test – YES, 75% Resazurin - YES Gerber Test – YES Clot on boiling – YES (for suspected milk) Lactometer – YES Alcohol Test – YES, 75% Resazurin - YES Gerber Test - YES Random tests for specific adulterants Processor #5 Lactometer – YES Alcohol Test – NO Resazurin - NO Gerber Test - NO Organoleptic - YES Lactometer – YES Alcohol Test – YES, 75% Resazurin - YES (every can) Gerber Test – YES Clot on boiling – YES (for suspected milk) Lactometer – YES Alcohol Test – YES, 75% Resazurin - YES (every truck) Gerber Test - YES Random tests for specific adulterants Processor #6 Lactometer – YES Alcohol Test – NO Resazurin - NO Gerber Test - NO Organoleptic - YES Lactometer – YES Alcohol Test – YES, 68% Resazurin - NO Gerber Test – YES Clot on boiling – varies (for suspected milk) Lactometer – YES Alcohol Test – YES, 68% Resazurin - NO Gerber Test - YES An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 26 Processor #7 Lactometer – YES Alcohol Test – NO Resazurin - YES -each can Gerber Test - NO Organoleptic - YES Lactometer – YES Alcohol Test – YES, 70% Resazurin - YES ea. can Gerber Test – YES Infra-red spectrometry Lactometer – YES Alcohol Test – YES, 70% Resazurin - YES (truck) Gerber Test - YES Test: antibiotics, aflatoxins and adulterants Source: Data collected in phone interviews conducted by national consultant, Dr. S. Daniels, March 2018 Milk quality at the MCC is a function of milk quality at the farm level and milk transport efficiency by the milk collecting system and the hygienic conditions, handling and storage of milk in transit. Having improved milk quality at the time of acceptance at the milk collecting point, the processing companies are then compelled to improve the milk transport times from milk collecting points to the milk chilling centers and prevent milk quality losses in the milk transport. The incentive for doing this is processors will receive milk with lower bacteria counts and based on testing of milk at the first point of exchange will pay a higher price for producer’s milk. Processors could educate producers to adjust their milking time of their cows and the milk delivery times to the milk collecting point. Most of milk collected at collection centers is morning milk (early morning is best) with some evening milk from producers close to a chilling center. The processor can then harmonize milk acceptance times at the milk collecting points and pick up time of this milk by the milk transporter to the milk chilling center. 5.3.2. Resazurin Test The evaluation team suggests for MOD to encourage the industry to use of the Resazurin test for measuring quality that would be low cost for all the processors. This test requires a tablet of Resazurin dissolved in 50 ml of sterilized water. One ml of liquid is added to ten ml of milk sample at the milk chilling center. The cost of each test sample would be less than LKR.018 per milk sample tested (Table 5.2.). Based on the bacterial content in the milk, this test produces a color change in the dye added to the milk sample. Milk is graded using the color disk for quality after insertion of the Resazurin solution. The test would result in percentage of milk samples being poor, fair, good, or excellent. The project would strive to have a certain percent of samples tested to be in the acceptable range of 4 and above (see Table 5.3.). The testing could be done at the collection point or at the chilling center. One processor is using the test at collection points and four are using the RT at the chilling centers. In areas where MOD conducts trainings on good milking practices (GMP), testing could validate producers’ adoption of GMP. The three key tests for processors (Table 5.1.) (though not used by all or consistent in measurements) are: • Lactometer test – test the purity of the milk by measuring the density – how much water and if the milk has been adulterated (water added). This test is done by all seven processors at each stage of the supply chain – collection point, chilling center, processing plant. • Alcohol test – test if the milk will coagulate on thermal processing – especially for UHT milk, evaporated milk and milk powders. Test is performed by one processor at the collection point and then by all processors at the chilling center and processing plant. • Resazurin test – test the hygiene of the milk and the keeping quality of the milk. Test is performed by one processor at the collection point, four processors at the chilling center, and three processors at the processing plant. (See the glossary for purpose of the RT.) Table 5.2. Steps taken to administer the Resazurin test An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 27 Step 1. Solution of Resazurin is prepared by dissolving 1 tablet in 50ml sterile glass-distilled water. Step 2. Pour 10 ml of milk sample into sterilized test tube. Step 3. Add quickly 1 ml of Resazurin solution in the test tube. Step 4. Mix the milk and dye thoroughly by inverting 2-3 times. Step 5. Place the tubes in the water bath at the temperature of 37.5 degrees C for two minutes. Step 6. Remove the tubes from the water bath. Step 7. Compare the color of milk in the test tube with standard disk until the colors match. Step 8. Record the color code from the disk. If color falls between two-disc numbers record half value – below 4 would reject (see Table 5.3.). The objective would be to have processors use RT and thus no additional investment equipment costs would be necessary. MOD can then work with processors to track the results of the RT results in those areas (clusters) where the project is carrying out interventions. MOD can then collect, track, and report on the improvements in the quality. MOD monitoring and evaluation (M&E) staff can then compare results to those areas where MOD is not working. These areas would serve as the counterfactual control in any experimental design by MOD’s M&E plan. Since we are monitoring the quality of milk received at the chilling center based on pooled volume of milk in a milk can (sizes of 20l or 40l) it is good to measure the changes in quality for each collecting point supplying to the MCC that falls within the cluster of producers targeted by MOD. Any milk having a persistently lower score from the RT can then be traced back to the individual farmers. In processor interviews (see Table 5.1.), only Processor #7 tests producers’ cans of milk at the collection point, and four processors reported doing RT testing at the chilling center. MOD can encourage wider use of RT testing and assist with the remedial training or refresher trainer as appropriate. Because the quality of the milk depends on time lapse until milk is cooled down to maintain the quality, processors will need to reduce the milk transport time from farm to milk chilling center. The processor could consider adding mini-coolers at strategic locations and these investments can be part of the industry’s investments which can be a key performance indicator in the PMP. The evaluation team saw a few mini￾coolers at collection points in villages that were used for collection and storage of evening milk. Farmers who hold evening milk lack proper cooling equipment and therefore hold milk and mix with next day morning milk which reduces the quality of their morning milk, and they get a lower price. Table 5.3. Reading the results of Resazurin test (10-minute Resazurin test) Resazurin disc number Color Grade of milk Action 6 Blue Excellent Accept 5 Light blue Very Good Accept 4 Purple Good Accept 3 Purple pink Fair Separate 2 Light pink Poor Separate 1 Pink Bad Reject 0 white Very Bad Reject 5.3.3. Farm level An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 28 Another quality indicator would be at the farm level with the simple testing of cows for sub-clinical mastitis which infects the teats of the cow and reduces the quality of the milk. The testing can be part of a training module for the target farmers so that the adoption can be measured against the level of sub-clinical mastitis occurring in the producers’ cows. Sub-clinical mastitis is an issue in dairy operations based on interviews with managers of chilling centers. Processors do have trained staff to visit farms and work with farmers on treating their dairy cows and improving conditions in their dairy barns. More efforts are needed, and MOD staff can train producers to conduct the test and record and report to the MOD field project officer. The producer can obtain the reagent for the California Mastitis Test for a nominal fee at the veterinary investigation center of the DAPH in the district. The measure of the important in herd health and milk quality would be in the price received for the producer’s milk. 5.4. Conclusion The dairy processing industry does not systematically and consistently use the same number of tests to measure quality. A quote from the MOD team is worthwhile to report here. “Quality standards amongst the dairy processors, as well as the type of quality tests they use across the supply chain vary based on individual processors’ product requirements. Therefore, it is impractical to use one common test type or a standard value within a test type across the board to measure quality given the realities in the current industry practices. Therefore, we do not propose to use a parameter (value) based indicator that measures only bacteria (Resazurin Test) and/or any animal issue, such as mastitis.” The GOSL has standards for milk and milk products. However, beyond the minimum requirements, the quality needs of each processor vary so there is not a consistent standard. There are national standards for milk for milk hygiene, but each processor chooses what quality level is required in producers’ milk for the products it manufactures. Furthermore, MOD has not yet identified the farmers they plan to work with. It is difficult to come up with a general baseline for these quality tests. It is important that MOD and its partners (such as DAPH) encourage processors to measure quality at the point of first producer-first handler exchange and prices paid are aligned with the quality of the milk. MOD can use the price paid for milk as a measure to evaluate the effectiveness of the trainings provided to dairy farmers to determine the rate of adoption by producers of best practices on the quality of their milk. 5.5. Recommendation The evaluation team supports the recommendation by the MOD staff to improve the quality of milk rather than further lower the rejection rate since the rate of rejection is already very low (below 1%). The recommendation of the project staff is: ‘The project proposes to train selected beneficiaries on hygienic milking practices including improved nutritious feeding and transport practices and will evaluate improvements based on premium prices paid for quality improvements. As premium pricing is a composite of many quality attributes, it will serve as a practical quality indicator across the industry. Price premiums when selling their milk are a strong financial incentive for farmers to modify and improve handling and management practices. The baseline for the proposed indicator is zero as interventions, and activities have not begun. The project will obtain each beneficiary farmer’s current price and track pricing changes over time. By the end of the project, 80% of beneficiary farmers will be getting a An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 29 quality-based price premium from their initial baseline values. Price premium is defined as the differences (increase) between the price per liter of milk sold during the reporting period by a project beneficiary (farmer) against his/her baseline price before project interventions begin. This indicator is (1) pragmatic, (2) cost effective, and (3) most importantly, in-line with current industry practices capturing a composite of quality attributes.” MOD staff memorandum, May 2018 An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 30 6. CONCLUSIONS, LESSONS LEARNED AND RECOMMENDATIONS FOR M&E FOR THREE BASELINE INDICATORS 6.1. Conclusions 1. MOD has set a target to work with 5,400 dairy producers to become commercial operations. The conclusion of the evaluation team is this number of producers is reasonable. There will be some districts in the target provinces which will be more attractive for commercial dairy investments and receptive to receiving project interventions because of the characteristics of the farming systems, the agro-ecological conditions, and the type of breeds of cattle and the presence of other species, such as buffaloes. The provinces and the districts were identified in Table 3.1. and confirmed through KIIs with key stakeholders during the field reconnaissance trip by the evaluation team. There are sufficient numbers of producers to meet IESC’s target of 5,400 producers who are currently in FMS in the Milco system or just farmers’ societies as in the case of other processors. Producers supply milk to processors through chilling centers to be a ready target for MOD to commercialize. These producers are in several agro-ecological zones (e.g., hill areas and low dry land areas), offering the MOD project staff a wide selection of locations from which to choose project beneficiaries. This diversity of locations for MOD activities will also help to minimize the possibility of potential conflicts that could occur among processors in procuring milk in a specific project area from producer groups. 2. The volume of milk initially estimated for the baseline by MOD is realistic and should be kept as the baseline for the project. The evaluation team’s estimate of annual volume of milk by the target group was 37,338,000 liters (within the range of 35,452,000 to 40,762,000 liters, which matches closely to the project’s baseline estimate, so no adjustment is needed in this indicator). 3. The value of the milk produced currently, given the prices received by farmers at the farm gate, is lower than what was estimated in the performance monitoring plan (PMP), LKR 70 per liter. The evaluation team estimated the current average price ranges from LKR 60 to LKR 66 per liter. The evaluation team used an average of LKR 63 per liter for all producers. The estimated value of the milk at the farm level is US$ 15.43 million rather than the higher amount of $21.9 million estimated by the design team. The producers’ milk price was determined from interviews with stakeholders and is in the range estimated of LKR 60 to LKR 66 per liter. The evaluation team recommends an adjustment in the value of the milk at the farm gate. The team believes that there is adequate space for the milk price to increase for those producers to reward them for TS and better quality of the milk, and also keep domestic production competitive with imports if productivity increases and there are price premiums for quality. Processors have been shown to pay higher prices for milk from producers who provide consistent quality milk. Over the LOP it may be possible to have companies include quality (reduction in bacteria and somatic cell counts) in setting producer price. This would incentivize producers to improve the quality of their milk. However, the team believes that including these measures of bacteria and somatic cell counts in a new comprehensive pricing formula by the dairy industry is not a feasible deliverable within the time period of the MOD project. 4. The evaluation team found in interviews with stakeholders that the rejection rate of milk is low for milk sourced through the formal supply chain. The focus of MOD is on those 5,400 producers who are now engaged or could enter the formal supply chain through the FMS or cooperative. Because of the low rejection rate through the formal collection centers (collection points and chilling centers), the lowering of the rejection rate by 80% is unlikely to be a satisfactory indicator for the MOD project. The evaluation team recommends an alternative measure proposed in this report focused on producers’ receiving a higher price An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 31 for their milk compared to the baseline value. It is important that MOD work closely with its partners, e.g., DAPH and dairy processors, to get their buy-in. 6.2. Lessons Learned Though the project is in its initial stage, there are relevant lessons learned from past dairy development programs in Sri Lanka which the consultants are familiar and from desk research. The addition of this Lessons Learned section is to advise the MOD staff the need to convey to producers and businesses in the dairy value chain the importance to both modernize and commercialize their dairy enterprises. 1. GOSL will have to be proactive in promotion of milk consumption, especially regarding attracting new milk drinkers to fresh fluid milk and milk products (cultured and other products). The promotion of drinking milk for health reasons is an area of public benefit. The dairy industry would benefit from generic advertising promoting fresh milk as experienced in other countries. Consumption of fresh, pasteurized milk requires good quality milk. 2. Stakeholders in the dairy value chain require the best, modern, cost-effective technologies which can boost the transition to commercial dairy businesses. The size of the individual dairy operation is less important than dairy operations that are managed and profitable based on the context of Sri Lanka where the average dairy operation is under five milk cows per family and larger herds will face difficulty during times of the year when forage based feed can be limited. MOD’s target is for 5,400 dairy producers to adopt “dairy as a business” and this is done in collaboration with the public (Milco) and private sector processors. The major difference between Milco and the private processing companies is that Milco prefers to purchase its milk through Farmer Managed Societies (FMS) to support dairy cooperatives. 3. Achieving the target in increased milk volumes proposed by MOD must be balanced with the market demand for milk. Sri Lanka imports large amounts of dairy products. Milk production will need to be cost competitive with imports of powder milk. Increases in producer prices of milk can stymie the development of new products which can increase milk consumption. The focus needs to on improving productivity of dairy cows and quality of milk. 6.3. Recommendations In evaluating the three-key baseline estimate, the team makes the following recommendations: • The reported producer price is between LKR 60 and LKR 66 per liter which is lower than the GOSL’s suggested price for cow’s milk with TS of 12.5% of LKR 67. The price difference could be because of low fat or SNF because a decline in these two measures for milk. The GOSL price does not change between flush and lean season of milk supply. This can be addressed with: ➢ Stakeholders in the Sri Lanka need to recognize the interdependence among themselves if a strong and sustainable dairy industry is to prosper. There is the need to support the emerging and small-scale dairy sector to be part of the national enterprise. ➢ Producers engaged in recording keeping of their dairy cows can achieve a higher productive herd life and increased milk production (Smoltz and Grobler). ➢ The inclusion of improved quality forages in dairy ration to increase solids-not-fat (SNF). Quality forage must be accessible and affordable to the producers. An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 32 ➢ The MOD staff needs to monitor nutrient composition of the dairy ration that the farmers make on-farm, and this can be done with the assistance of the veterinary investigation center staff. MOD staff and its partners can train farmers in feed management, and it is important to assess the rate of sustained adoption in improved feeding practices by those who attend the MOD trainings. ➢ Artificial insemination (AI) is important for sustaining and improving the present genetic potential of dairy cows to produce milk with higher TS. MOD can support more trained private sector inseminators. MOD staff and its partners can monitor AI services over time on it effectiveness to improve herd productivity. Too many producers reported dissatisfaction with current AI services, e.g., too many repeated trips to obtain a successful insemination. • Farmers need training in GMP. The collaboration between the MOD project and the private sector processors offers the chance for producers to transition to commercial dairy production and become incentivized to produce quality milk. MOD can: ➢ Train producers in GMP from initial “let down” of milk to first point of collection; ➢ Assist processors to better collect milk from first point of collection to chilling of milk, e.g., management of transportation and logistics to reduce time in transit before chilling; ➢ Encourage processors to adopt and conduct inexpensive tests on milk quality, e.g., such as the Resazurin Test, and report this information to producers; and ➢ Advise processors on a payment system that will reward producers that supply quality milk (low bacteria and somatic cell counts) to incentivize them to improve better on-farm practices. An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 33 7. ANNEX 7.1. Data Sources Department of Animal Production and Health. 2014. “Livestock Statistical Bulletin – 2013.” Peradeniya Department of Animal Production and Health. 2016. “Livestock Statistical Bulletin – 2015.” Peradeniya. Department of Animal Production and Health. 2017. “Livestock Statistical Bulletin – 2016.” Peradeniya. Food and Agriculture Organization. (n.d.) “Milk Testing and Quality Control.” Milk Processing Guide Series, Vol.2. www.fao.org/ag/aginfo/resources/’documents/MPGuide/mpguide2.htm Le Gal, Pierre-Yves, et al. (2013). “Supporting strategic thinking of smallholder dairy farmers using a whole farm simulation tool.” Tropical Animal Health and Production. 45 (5), p. 1119-1129. Mendes, S.S. et al. (2014). “Milk Powder Imports and government policies: the case of Sri Lanka.” Journal of Agricultural Economics and Rural Development. 2 (3), p. 86-91. Pathumsha, H.M.A.V. (2016). “Dairy Industry Trends in Sri Lanka.” Colombo. Scholtz, M.M. and S.M. Grobler. (2009). “A systems approach to the South African dairy industry.” South African Journal of Animal Science. 39 (Supplement 1). Sri Lanka Dairy Excellence Training Initiative. (2016). Current Situation of the Dairy Sector in five districts of Sri Lanka – Report on the Baseline Survey. September 2016. United States Department of Agriculture (USDA). (2013) “ Monitoring and Evaluation Policy.” Food Assistance Division, Office of Capacity Building. Foreign Agricultural Service. Washington, D.C. United States Department of Agriculture (USDA). (2013) “ Food for Progress and McGovern-Dole Indicators and Definitions.” Food Assistance Division, Office of Capacity Building. Foreign Agricultural Service. Washington, D.C. 7.2. Scope of Work for Baseline Consultants Background and Justification IESC – the International Executive Service Corps – is a U.S.-based nonprofit that seeks private enterprise solutions to problems of poverty in developing countries and around the world. By providing a mix of technical assistance and volunteer expert support, IESC works with emerging industries, financial institutions, and governments to stabilize economic environments, increase opportunity, and promote growth. On September 5, 2017, IESC was awarded the Market-Oriented Dairy project under USDA’s Food for Progress program. This 5-year project, implemented between September 2017 and September 2022, will help farmers and enterprises to meet commercially-sustainable market demands of the dairy value chain. The project addresses two of the dairy sector’s greatest needs: sustainable access to and availability of inputs and quality and safety of milk and dairy products at the local level. The project will improve the availability and access to quality cattle for dairy farming and to quality artificial insemination and veterinary services and to high quality fodder for dairy cattle, while increasing adherence to proper feeding regimens and management practices and technology. An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 34 IESC will conduct six distinct agricultural development activities in coordination with the private sector, the Department of Animal Production & Health (DAPH), and the National Livestock Development Board (NLDB). Activities include the following: • Activity 1 – Capacity Building: Agricultural Extension Agents/Services o Train extension officers and beneficiary farmers in best practices for productivity o Improve artificial insemination services o Develop and roll-out mobile extension services o Conduct training on dairy farming as a business to develop capacity of dairy entrepreneurs • Activity 2 – Inputs: Develop Agrodealers and/or Input Suppliers o Assist formalization of breeder programs targeting medium and large-scale farms o Develop capacity of private input retail operations o Develop fodder and silage enterprise • Activity 3 – Financial Services: Leverage Private and/or Public Sector Investment o Set up an Overseas Private Investment Corporation (OPIC) – USDA investment fund o Increase local financial institutions interest to lend to dairy sector • Activity 4 – Market Access: Facilitate Buyer-Seller Relationships o Develop and implement mobile applications to facilitate exchange of inputs o Connect informal producers to formal markets • Activity 5 – Training: Sanitary and Phytosanitary Standards o Train farmers on best practices in milk quality o Quality and safety campaigns for consumers o Promote quality-based payments for milk producers • Activity 6 – Capacity Building: Trade Associations o Strengthen National Dairy Association Priority regions to be reached by the project include Sri Lanka’s northern, northwestern, northcentral, eastern, Uva, and central regions. These regions are targeted due to the prevalence of poverty and potential for contributing to the growth of the country’s dairy sector. While this project indirectly benefits all actors within the dairy value chain, the targeted direct beneficiaries are dairy producers at multiple levels of production, input suppliers, collection center managers and employees, farmer cooperatives, artificial insemination technicians and veterinary students, farmers with potential to become medium-scale fodder producers, SMEs seeking to invest in the dairy value chain, as well as the staff of banks, microfinance institutions (MFIs), and other financial institutions. Baseline Evaluation As part of project inception, IESC will undertake an evaluation of the initial baseline data, which formed a basis for the original proposal. The baseline evaluation team will include an international agricultural economist with a background in dairy and a Sri Lankan dairy value chain specialist. The team will follow USDA Monitoring and Evaluation Policy located here. IESC is responsible for establishing indicator baseline information and targets for which the project will regularly measure performance against (Information on indicators can be found here: USDA/FAS Guidance on Standard Indicators for Food Aid Programs). The baseline information for indicators must be measured and established prior to the start of program activities. The baseline evaluation team will review the initial annual targets IESC established for all project indicators to evaluate, which will be reviewed as realistic and ambitious. An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 35 The baseline study will lay the groundwork for all future monitoring and evaluations by establishing a) baseline values and targets for project indicators, and b) evaluation methods for assessing project outcomes and impacts. USDA/FAS has allowed for baseline studies to be conducted in-house, however, to ensure objectivity and a solid foundation for ongoing project monitoring and evaluation, IESC will retain an independent consultant to ensure the baseline study properly addresses performance indicators and other elements of the project’s official Performance Monitoring Plan (PMP). The selected evaluator will determine the methodology of the baseline study and the focus of quantitative and qualitative baseline data collection will be on the following stakeholders: dairy producers at multiple levels of production, collection center managers and employees, farmer cooperatives, artificial insemination technicians and vet students, farmers with potential to become medium-scale fodder producers, Sri Lankan enterprises seeking to invest in the dairy value chain, as well as banks, MFIs, and other financial institutions. The evaluation process will be managed by the independent evaluator with support from IESC’s monitoring and evaluation staff (non-project management) in both Sri Lanka and Washington, DC. The selected candidate will be provided a brief upon arrival to the MOD Project office in Colombo, Sri Lanka. This short-term assignment is intended for an expatriate with experience in dairy value chain project design, monitoring, and evaluation, and is anticipated to start on or about January 16, 2018, not to exceed 25 working days (including report writing). Duties and Responsibilities • Lead review and test of IESC’s proposed methodologies for collecting, compiling, and reporting data; • Supervise and collaborate with Sri Lankan local dairy value chain specialist to support technical inputs and analysis; • Lead review of data sources, and testing of the initial data sets themselves, to confirm data quality; • Develop a set of evaluation questions to be addressed in study • Serve as the lead report writer, responsible for outline, assigning input to local specialist, and finalizing written, high-quality product; • Verify estimated baseline numbers for 1) value; and 2) volume of sales; and, 3) percentage of rejections at collection points of dairy based commodities using desk review, interviews, and direct observation. • Analyze applicable market data sources related to domestic dairy value chain inputs and end￾market products; • Design, facilitate, and analyze a limited number of focus group discussions, including dairy￾focused entrepreneurs, producers, that serve as qualitative research; • Produce thorough documentation for each data collection method used to ensure the same can be applied throughout the project (data reliability); • Maintain regular updates to chief of party, home office project staff, and IESC M&E director; • Ensure the baseline study properly addresses performance indicators and other elements of the project’s official Performance Monitoring Plan (PMP); and • Determine whether the project’s established M&E protocols are sufficiently well-designed and implemented, and if there are adequate controls to ensure that the information collected is accurate and reliable. An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 36 Deliverables • Evaluation workplan • Clean data sets • Draft evaluation report and performance indicators annex o Final evaluation report that includes the following (A baseline study report outline in IESC template will be discussed and finalized during the first week of the assignment): o Proposed baseline values for three program indicators and any applicable disaggregation o A detailed description of the entire data collection process (methodologies and tools) o Lessons learned/key things to monitor going forward • Transcripts of any focus group discussions conducted and meeting notes. • 2-3 Page stand-alone brief (Nontechnical summary written in plain language) to include the following: o Description of evaluation design o Key findings o Other relevant considerations Payment and Resources In addition to the fee paid to the selected evaluator for services rendered, IESC will cover the following costs: 1. Airfare to and from Colombo, Sri Lanka 2. Assignment related transportation costs 3. Accommodations while on field assignment 4. Meals and incidentals 5. Costs related to the implementation of the baseline – IESC’s project team will work with the selected evaluator to formulate an assignment budget. Information and communication technologies (computers, mobile devices, etc.) will be available for the evaluators use upon arrival to Sri Lanka. Qualifications • B.A. in agriculture science, economics, agribusiness, statistics, or related field; M.S. preferred, but not required; • Minimum of 15 years of experience working on agricultural value chains, including considerable experience with dairy and five years of which working on project design, measurement, analysis; • Extensive experience with quantitative and qualitative data collection techniques required; • Baseline report generation and analysis experience related to agriculture sectors required; • Experience with USDA, USAID, or other donor programs preferred; • Ability to prioritize among multiple competing requests; • Self-starter, detail-oriented, thorough, and well organized; • Well-developed interpersonal and cross-cultural communication skills; • Excellent written and oral communication and presentation skills; and • Ability to travel when in-country. An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 37 7.3. Interview guides 7.3.1. Artificial Inseminator (AI) • How many times on average do you have to inseminate a cow? • How quickly after receiving call from producer can you be at the farm? • Do you keep records on the breed history you use for dairy cows inseminated? 7.3.2. Producer (individual and group at a chilling center) • Major issue facing your dairy operation? • Do you belong to a farmer association? • If no, would you like to form a farmer association? • How do you rate the AI service you receive from the government? Good, Fair, Poor • Do you use concentrate feed in your daily ration? Who supplies the feed to you – processor, collector, buy from an agribusiness? 7.3.3. Bulk Collector • Describe the nature of your business as a private collector of producers’ milk? • How many years have you collected milk? • How many staff and vehicles do you have? • How many milk producers do you collect milk? • Do you collect producers’ morning and evening milk? • To how many outlets to you sell your milk? • How often do you pay your producers in a month? • Do you provide any other services to your producers? 7.3.4. Milk Collection Points • How many people supply this collection point? • Average volume of milk collected daily? • Processing company responsible for collection point? • Price received from processing company? • What tests are performed on the milk received? 7.3.5. Processors • How and from where do you source your milk from producers? Collection points, chilling centers, bulk collectors? • Number of collection points? Number of chilling centers? • Number of milk producers supplying your plant? Out-growers supplying milk? • What is the average size of the dairy operations you source your milk? • How many producers can be labeled as conducting “dairy as a business” in your collection area? • Volume of milk received from the average producer? Peak season and lean season? • Where does the raw milk go to and into what products? • What is your rejection rate of milk? • What services do you provide to your milk producers? An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 38 7.4. Milksheds for the major processing companies Annex Figure 7.4. Milkshed for the major processing companies An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 39 7.5. Review of proposed methodology for collecting, compiling and reporting data 7.5.1. Overview MOD has three areas of interventions that require monitoring: training, technical assistance and investments with investment fund (US$4.0 million). Training of trainers (TOT). The project will train Master Trainers (+100) who will train farmers in production, quality and conducting “dairy as a business.” Technical assistance (TA). The project will conduct technical assistance to fill the gaps in the supply chain on a “demand” driven basis. Staff will identify gaps during their interactions with stakeholders and make continuous visits to businesses. Sarvodaya, partner in MOD, will have staff in the regional offices to deliver technical assistance. Investments. MOD will facilitate financial needs of farmers, processors and other stakeholders to obtain financing for improvement of the value chain. Farmers will seek smaller loans, and processors and others obtain larger loans. MOD has a $4 million fund to facilitate the investments. The target is for the $4 million to leverage $24 million. MOD will provide TA for firms to access loans. 7.5.2. Collecting and Compiling 7.5.2.1. Profiling of target beneficiaries (farmers) Farmers to participate in monitoring will be identified in the training programs. The MOD staff will administer questionnaires to collect information (demographic, behavioral, etc.) on producers at the first intervention (training) they undergo and will serve as the baseline information for that producer. The database will grow over time and eventually have details on all 5,400 farmers for their profiles. The first survey will be done soon. The steps to be taken in the profiling are: 1. selection of target areas 2. develop map of clusters of producers that includes numbers of small, medium and large units 3. location of key facilities around the target beneficiaries, e.g. chilling centers and processors 7.5.2.2 Collecting data from target beneficiaries (farmers) after interventions From the population of project farmers, a sample will be selected using scientific sampling methods (stratified random sampling) for tracking and projection to the larger population. Approximately 700 farmers will represent the target population to evaluate outcomes for the interventions. Sample will be stratified so that representation of farmers in a cluster can be representative of the larger population. MOD project will also make use of the processor’s information on producers to triangulate the survey findings. Observation #1. The selection of producers from attendance at training is being considered for selecting farmers. To have attribution of trainings to changes in outcomes, then this is one criteria to consider. Others include being a member of a FMS and selling to a processors chilling center and not to a bulk collector. Observation #2. Any survey in a province and district needs to have prior commitment from the provincial, district and veterinary range livestock officers. Theses stakeholders need not to be engaged in the survey directly but certainly must be aware of the activity and general aggregate results. Where feasible it is important to engage provincial, district and veterinary range specialists in reviewing the collection An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 40 process for data. This will help in getting their support when the project staff. Sharing results from data would build team building with the government agencies. Observation #3. An issue of importance is the cross-influence of other donor programs on-going or recently completed. Producers have not been in a vacuum when it comes to donor programs. It is important to the best of the project capacity to segregate producers into cluster areas (where practical) to minimize cross-influence and carry out a mapping exercise to identify areas based on the above criteria so that direct attribution results for the training offered can be measured. To have a plausible attribution from MOD project one critical criteria may be to select farmers and areas where heavy interventions are not carried out in the recent past, on-going or is planned to take place in terms of training etc., specially by non￾government organizations. The regular government interventions theoretically are supposed to apply to all so that we can discount the factor across all areas and farmers. Observation #4. Where possible, the field survey teams use of the latest information technology (IT) in collecting data. This will improve efficiency in collecting and analyzing data. Several software for use on smart phones can improve the collecting, uploading and analyzing field data. This can save costs of using paper questionnaires and improve the accuracy of compiling the data for analysis. 7.5.3. Reporting The MOD M&E expert will submit reports twice a year to USDA. The reporting requirement does not appear to be unreasonable. The project will triangulate the findings with data from other sources, e.g. processor payment records to producers. The MOD office staff will be regularly collecting data on predefined formats whenever they are in the field and will serve as an effective feedback mechanism for corrective action on a timely basis. Observation #5. It would be important from a project development approach to have an adaptive management strategy which follow the methods of the Collaboration, Learning and Adaption (CLA) approach. This approach requires using the field data collected to make necessary decisions about modification necessary to the project based on information from the field. MOD offers the opportunity to be flexible within bounds of the project design to address necessary changes in project activities. An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 41 7.6. PowerPoint presentation describing approach, methodology and findings to MOD staff An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 42 An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 43 An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 44 An Evaluation of Three Baseline Indicators for the Market-Oriented Dairy (MOD) Project 45