i November 2015 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Social Impact, Inc. Final Performance Evaluation of USAID’s Developing Agricultural Communities (DAC) Project (2010 to 2015) ii Final Performance Evaluation of USAID’s Developing Agricultural Communities (DAC) Project (2010 to 2015) AVANSA M&E TASK ORDER 2015-2020 November 25, 2015 USAID Contract AID-486-I-14-00001 Task Order AID-472-TO-15-00003 Evaluation Team: SI-Timor-Leste Peter Jarvis Team Leader Alberto Correia National M&E Specialist Henrique Ximenes Operations Specialist Regional Support Amanda Stek Data Analyst (Jakarta, Indonesia) Short-Term Technical Assistance Acacio Amaral Senior M&E Specialist Eugenia Correia Research Assistant DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. iii CONTENTS Acronyms...................................................................................................................................................................... 1 Executive Summary .................................................................................................................................................... 2 Background ................................................................................................................................................................... 6 Context .................................................................................................................................................................... 6 Developing Agricultural Communities Project ............................................................................................... 6 Evaluation Purpose & Evaluation Questions......................................................................................................... 7 Evaluation Purpose ................................................................................................................................................. 7 Evaluation Questions ............................................................................................................................................. 7 Methodology ................................................................................................................................................................ 8 Desk Review ............................................................................................................................................................ 8 Qualitative Data Collection ................................................................................................................................. 9 Focus Group Discussions ............................................................................................................................... 9 Key Informants ................................................................................................................................................... 9 Quantitative Data Collection ............................................................................................................................ 11 Data Synthesis ....................................................................................................................................................... 12 Limitations.............................................................................................................................................................. 12 Findings ........................................................................................................................................................................ 13 Conclusions ................................................................................................................................................................ 41 Evaluation Summary against the Five Standard Evaluation Criteria ......................................................... 41 Conclusions for Specific Themes ..................................................................................................................... 42 Recommendations .................................................................................................................................................... 47 Annex I: Data Collection Instruments ............................................................................................................ 51 Group Information .................................................................................................................................................. 51 Introductions ............................................................................................................................................................ 51 Participant List and Consent Sheet ................................................................................................................ 52 A. Group Information ....................................................................................................................................... 52 B. Working with DAC ....................................................................................................................................... 53 C. Marketing Produce ...................................................................................................................................... 53 D. Future Plans .................................................................................................................................................. 54 Questions for Other Key Informants .................................................................................................................. 54 ANNEX II: Results for the quantitative household survey ........................................................................ 60 ANNEX III: Evaluation scope of work ............................................................................................................ 85 ANNEX IV: Disclosure of any conflicts of interest ..................................................................................... 87 ANNEX V: Sources of information / Contacts ............................................................................................. 90 ANNEX VI: Matrix of key findings and recommendations ........................................................................ 93 1 ACRONYMS AHDISTAL District Horticulture Association ALME Asia Learning Monitoring and Evaluation BNCTL National Commercial Bank of Timor-Leste CDCS Country Development Cooperation Strategy COP Chief of Party COR Contracting Officer’s Representative DAC Development Agriculture Communities DAI DO Development Alternatives Incorporated Development Objective DQA Data Quality Assessment DSP Developing Private Sector (Dezenvolve Setór Privadu) EG Economic Growth ET Evaluation Team FGD Focus Group Discussion FTF Feed the Future GDA Global Development Alliance GIS Geographic Information System GoTL Government of Timor-Leste HH Household HQ Headquarters HWG Horticulture Working Group ILO International Labour Organization IP Implementing Partner IPM Integrated Pest Management IR Intermediate Result KII Key Informant Interviews MAF Ministry of Agriculture and Fishery M&E Monitoring and Evaluation NGO Non-Governmental Organization PE Performance Evaluation PMEP Project Monitoring and Evaluation Plan PMP Performance Management Plan QA Quality Assessment RDMA Regional Development Mission for Asia SHA Special Horticultural Area SI Social Impact SOW Statement of Work TNS Taylor Nelson Sofres TO Task Order US United State USD United States Dollar USAID United States Agency for International Development 2 EXECUTIVE SUMMARY The USAID Developing Agricultural Communities (DAC) project was a 4.5 year project costing USD 8.3 million and implemented by Development Alternatives Incorporated (DAI). A private sector partner, under the Conoco Phillips Global Development Alliance/Partnership (GDA) agreement, contributed $0.95 million of this amount in order to promote agricultural sector employment. The project ran initially from 2010 until 2013 but was extended until February 2015 as a result of the additional funding contributed jointly by USAID and Conoco Phillips. The project linked farmers primarily with supermarket buyers in Dili. The buyers provided the farmers with seeds for the vegetable crops they wanted and attempted to manage production volumes by controlling the amount of seed distributed to farmers. The project initially focused on farmers in Aileu Municipality, in which a predecessor project, Dezenvolve Setor Privadu (DSP), had supported vegetable farming, but eventually expanded to Ermera, Dili, Bobonaro and Liquica. DAC supported 37 groups of the farmers (Special Horticultural Areas – SHAs) by providing technical training, the materials and farm inputs required to set up demonstration plots, and a limited amount of material free to farmers to set up their own individual vegetable gardens. Supermarkets paid higher than average market prices, but in return expected higher quality produce and a greater range of products that normally found in the local markets. This higher quality produce was intended to substitute for imported vegetables. One of the objectives of the project was to extend the growing season by using plasticulture, mainly plastic growing tunnels, which protect the young vegetable crops during heavy rains. Most vegetables are traditionally produced during the start of the dry season when there is still water in irrigation canals and ponds; at this time plants are less susceptible to pests and diseases. USAID posed 5 evaluation questions, being: 1. Do target beneficiaries report an increase in income as a result of project activities? If so, by how much? How was income used? Has income resulted in increased benefit to all members of the household? 2. Were there any unintended outcomes (positive and/or negative) that resulted from project activities (i.e. child labour, increased school attendance, increased household burden for women)? 3. Are linkages between farmers and buyers likely to be sustainable? 4. Did technical assistance to farmers reach all intended beneficiaries (i.e., women and youth)? 5. What variations were there in implementation approaches or strategies and what was most effective/efficient in achieving results? Social Impact answered these questions by carrying out a review of project documents, and through focus group discussions with representatives of 12 horticultural groups supported by the project, key informant interviews with ex project staff, supermarket buyers, farmer leaders and others associated with the project. A survey of 182 beneficiaries was also carried out using a pretested questionnaire. 3 Main Findings Farmers were linked to supermarkets and in 2013 and 2014 the two main supermarkets purchased $278,000 and $373,000 worth of vegetables from DAC supported farmers, with an increase in value of 34% between 2013 and 20141. From our beneficiary survey, farmers reported an average weekly income of $70 from the sale of vegetables, selling on average for 46 weeks in the years. Median income was $50; there was a large variation in income per beneficiary with 86% of farmers earning less than $2,700 per year ($60 per week). The more diligent and committed farmers earned higher incomes. This income data was supported by project data which suggests that farmers’ sales to Kmanek (the largest supermarket buyer) were on average around $1,000 in 2014. However, farmers also sell vegetables2 in the local market, and sometimes to other buyers, not just to the supermarkets. This additional income was spent mainly on school fees and daily living expenses, traditional ceremonies, home improvements, and to a lesser extent on asset purchases, savings, and business activities. Most members of the family benefit from the additional income. Women, who were 32% of project beneficiaries, have the greatest control over money although decisions about spending money are commonly made by the husband and wife together. In our survey, if the beneficiary was a man 43% of them kept the money, but if a woman beneficiary 91% of them kept the money. In the case of male beneficiaries, 53% of them gave the money to their wives. The market for fresh vegetables is now becoming saturated with supermarkets reporting that there is too much production at certain times of the year, particularly for the easier to grow green leafy vegetables. However, there are still shortages of some product lines. Prices paid by supermarkets for produce on the domestic market are far higher than can be justified if the produce were to be targeted for the export market. Informants reported that if produce were to be targeted to the export markets, prices paid to farmers would have to be about half what they are now. The vegetable value chain is one of the most challenging. The technologies used need to be quite advanced, quality standards should be high, fresh produce needs to be available in sufficient volumes throughout the year, and shelf-life is short in the absence of processing options. The project had a demonstration effect in that both the two main supermarkets started additional groups not supported directly by DAC. This was mainly for outdoor vegetable production. They did this because they were unable to buy enough produce of the types they wanted from the existing DAC groups. However, this has exacerbated logistical problems because they now need to make pickups from more places that are more geographically dispersed. There are signs of unsustainability in some aspects of the project. Whilst the supermarkets are continuing to support farmers through their field staff, they do not have the resources or the technical skills to conduct horticultural training for their own field staff. So the viability of private sector extension efforts is uncertain. DAC provided some farm inputs free of charge to group members, but the members must now purchase on credit from supermarkets, although the distribution of seeds is effectively free to farmers. 1 Project data records total sales to all 3 supermarkets in 2014 at 408,000 USD including indoor and outdoor production. 2 Grown from their own seeds, or vegetables that do not meet the required supermarket grading standards 4 The horticultural association started by DAC is not sustainable. The Aileu District Horticultural Association (AHDISTAL) was started quite late and does in any case not support the 11 groups outside Aileu Municipality3. AHDISTAL does not have a sufficient revenue base to be sustainable, relies on volunteer staff, and does not have the resources to continue to train farmer members or to replace assets. There is insufficient reinvestment by farmers in productive assess, and limited ability for farmers to access credit, even though the project did link farmers to credit sources. The supermarkets say that they initially made losses, and that now they hardly make any profit but continue to support farmers because they feel they have a social responsibility to the rural community. The project supported women and youth. Thirty-two percent (32%) of the project direct beneficiaries were women and they had the same training opportunities as men. Nine percent of direct beneficiaries were under 25 years old; children helped their families in the vegetable gardens and were in some cases able to earn money for themselves. The DAC training was particularly valued by beneficiaries. Most reported that they were very satisfied with the technical horticultural training. The most valued training was the agricultural training, marketing, and the linkages to the value chain and buyers. However, some farmers still do not comply with the quality and grading standards that the supermarkets would like. Important criteria for training success is that it should be of sufficient duration (DAC training lasted for at least 3 months) and that it should include both theory and practice. The local cross-visits were appreciated particularly by farmers in the districts outside Aileu; the one month training in Indonesia, where farmers lived with Indonesian farmers, was especially valuable. Almost half of beneficiaries had not attended school, so this created problems for some of the more theoretical training in particular training such as book keeping and farming as a business. Some of the initial beneficiaries that were selected did eventually drop out of the groups. In our focus group discussion this drop-out rate appeared to be 12% of the group members4. The business model adopted by DAC has risks because farmers are very dependent on the supermarkets to continue support. If this support were to stop, farmers reported that they would still continue to grow vegetables but would sell in the local market and use local seeds. High quality seeds are not available locally and are supplied by supermarkets who import the seeds. Irrigating crops is very labour intensive; however, very few farmers have drip irrigation or hoses for watering crops by gravity feed or from a pumped water supply. Farmers are not investing, either individually or in groups, in irrigation/watering infrastructure. Group cooperation for investment is not yet evident. Key Recommendations If intending to support the commercialization of agriculture, projects should focus more on more commercially minded farmers and in strategic locations. Whilst this approach may not provide benefits to so many farmers, it offers the most chance of sustainability. Projects should also focus on producing crops that have some chance of being competitive in international markets because the domestic market is unable to absorb substantial amounts of additional production. Timor-Leste’s domestic market is relatively small, with little economic activity outside Dili that can justify higher prices. New projects will need to focus on more diversified products 3 Previously called districts 4 Compared to the number of members reported by DAC 5 than vegetables and start moving towards exports of crops that are less perishable and where Timor￾Leste may have comparative advantage and some chance to be competitive. Timor-Leste is a high cost economy with a high value currency. Any target markets are likely to be high value, niche markets. Focusing on more commercially minded farmers in strategic locations will also make projects more cost effective, make logistics easier, and will have more chance of sustainability. The DAC training model was a good one and should be adopted by other projects. Farmers really gained benefits from the training, which was sufficiently long and included sufficient practical training using qualified trainers, as well as opportunities to observe technologies in other locations. Whilst farmers do report making additional income, there is no real data that shows the profitability of enterprises or the profitability across the value chain. Neither are there any financial models that demonstrate this. There is a need for better financial and economic analysis of enterprises, and across the value chain, so that donors, projects and the Government of Timor-Leste can make good investment decisions and promote the crops that have a sustainable market potential. The ability of farmers to represent themselves through farmer organizations is very limited, and capacity building will take time. AHDISATL needs continued support if it will have the capacity to support farmers and to act as a model for future capacity building of farmer organizations. The new Avansa Agrikultura project will need to continue to support the capacity building of AHDISTAL and to expand the coverage of farmer organizations to the groups that were not covered by AHDISTAL in other districts. Avansa should also continue to support the farmer groups supported by DAC, particularly the groups that were established late in the project cycle. Furthermore, the Avansa project needs to continue to support the training of the supermarkets’ field and extension staff if they are to continue to support farmers. A market review needs to be undertaken to identify what opportunities there are for continued import substitution by vegetable product lines, and by type of crops. The possibility and viability of crop export needs to be investigated so that new opportunities are identified. Evaluation Criteria Against the five evaluation criteria, the project scores well against relevance and effectiveness. At the household level it had a significant impact on beneficiary incomes, but not very much impact nationally. It did not score so well against the criteria of efficiency and sustainability. 6 BACKGROUND CONTEXT Agriculture is the primary source of livelihood for more than 80% of Timor-Leste population5. However, the majority of farmers are subsistence farmers and do not adopt the technologies and practices required for commercial and sustainable agricultural production. DEVELOPING AGRICULTURAL COMMUNITIES PROJECT To support Timor-Leste to reduce poverty and under-nutrition, USAID funded the Developing Agricultural Communities (DAC) project, implemented by DAI. The DAC project began in 2010, and aimed to help Timorese farmers move from subsistence agriculture towards more commercially oriented income-generating farming, and to link them with higher value domestic markets. This project purpose aligns with the Government of Timor-Leste’s objective to support the transition from subsistence to commercial farming and to increase the production and productivity of key agricultural commodities. Developing Agricultural Communities/Desenvolve Agricultura Comunitária (DAC) followed on from another USAID funded project, Dezenvolve Setor Privadu (DSP) project which ran from 2005 to 2010. This project was also implemented by DAI as the managing contractor. The project also assisted a local company, Zero Star, to establish trading links to farmers to enable them to sell to the higher value markets. DSP developed modern horticultural technology by building a hydroponic greenhouse in Lequitura in 2007, which enabled Timorese farmers to grow high-value quality vegetables such as capsicum, tomatoes, and aubergines. These were marketed to local supermarkets and replaced vegetables which were previously imported. This stimulated interest in improved systems for horticultural production in Timore-Leste and enabled farmers to earn cash income. When the new DAC project was implemented, it continued to work with Lequitura growers and established two more greenhouses in Sarin and Liurai, also in Aileu Municipality. DAC was initially intended as a 3-year project, but was extended for another one and a half years as a result of receiving additional funding from Conoco Phillips to a total of four and a half years. The project officially ended on February 2015. The objective of DAC was to implement a development model that improved the economic and social livelihoods of household members in poor communities, in a way that enabled men, women, and youth to benefit. The project aimed to achieve this through the introduction of new technology and by providing technical and management training to horticulture group members. These groups were initially 8, but later expanded to a total of 37 Special Horticultural Areas (SHAs), or horticultural groups, consisting of around 20 farmer beneficiary members per group6. The project’s targeted beneficiaries included poor traditional farmers from relatively remote locations. Criteria for beneficiary selection included suitable land, motivation to move to more commercially orientated farming, and an adequate water supply for vegetable production. Selected farmers were mostly already growing vegetables in traditional7 ways and selling vegetables in local markets, including Dili. 5 USAID Avansa M&E Statement of Work, pg. 10. 6 The number of groups was expanded through an additional USD 950,000 of funding, which was provided to USAID by Conoco Phillips. 7 Traditional means outdoors, without using inputs such as new varieties of seeds, plasticulture, agricultural chemicals and fertilisers, and by selling in local markets without grading produce. 7 DAC established groups in three phases. There were 8 original groups all in Aileu, then a GDA18 expansion to another 19 farmer groups based mainly in Aileu but also in Dili. In late 2013, a further GD2 expansion of another 10 group occurred, now in Ermera, Liquica and Bobonaro municipalities. Over the course of four and half years, the project supported 547 direct beneficiary farmers (32% of them women) in the municipalities of Aileu, Bobonaro, Dili, Ermera and Liquica. The total project funding was USD 8.3 million, being $7.35 million from USAID and $950,000 from Conoco Phillips. The project trained farmers in horticultural technologies and business management skills. Other activities included: assisting farmers organize and manage shared production facilities, to negotiate selling terms and conditions with supermarkets, conducting fresh vegetable value chain analysis to identify opportunities and constraints, to collect and analyze data about vegetable prices and quantity, to facilitate credit links between the farmers and the National Commercial Bank of Timor-Leste (BNCTL), to assist the supermarket buyers establish a production-management database, and to support supermarkets to develop and implement vegetable grading standards. The project established farmer trading arrangements principally with three supermarkets, Kmanek, Dilimart and W4. EVALUATION PURPOSE & EVALUATION QUESTIONS EVALUATION PURPOSE The purpose of the DAC final performance evaluation is to extract lessons learned and to generate recommendations which USAID/Timor-Leste and implementing partners can incorporate into the Avansa Agrikultura project. Findings from the evaluation report will be disseminated by Mission staff to key stakeholders, project beneficiaries, and project implementers as appropriate. EVALUATION QUESTIONS The evaluation explored the following five evaluation questions: 1. Do target beneficiaries report an increase in income as a result of project activities? If so, by how much? How was income used? Has income resulted in increased benefit to all members of the household? 2. Were there any unintended outcomes (positive and/or negative) that resulted from project activities (i.e. child labour, increased school attendance, increased household burden for women)? 3. Are linkages between farmers and buyers likely to be sustainable? 4. Did technical assistance to farmers reach all intended beneficiaries (i.e., women and youth)? 5. What variations were there in implementation approaches or strategies and what was most effective/efficient in achieving results? In order to answer the five key questions above, the evaluation team (ET) developed sub-evaluation questions for the quantitative and qualitative data collection. The survey instruments that were used are include in Annex 2. 8 GDA refers to the Conoco Phillips Global Development Alliance a Development Partnership between USAID and Conoco Phillips. 8 METHODOLOGY The Evaluation Team comprised three staff from Social Impact’s Timor-Leste (SI TL) Avansa M&E field office, two short-term national staff, and support for data analysis from the Social Impact’s Jakarta Office. The Avansa M&E staff were the Chief of Party/Team Leader, National M&E Specialist, and Operations Specialist. The two nationally recruited staff were the Senior M&E Specialist (as a sector specialist) and a female Research Assistant. In addition, an SI Program Manager from SI Jakarta provided data analysis support. The evaluation used a multi-level, mixed-methods design to triangulate information; this included: a) document review of secondary sources from DAC9 ; b) primary qualitative data collection through focus group discussions (FGDs) and key informant interviews (KIIs), and c) primary quantitative data collection via a household b e n e f ic i a r y survey of 182 respondents. The team also field tested the survey instruments and the beneficiary questionnaire prior to implementation and made modifications for improvements. The first field work was the qualitative data collection through focus group discussions (FGD) in 12 communities; this included two women’s groups that benefited from the project activities. The FGD participants were primarily the groups’ elected leaders and farmer members of the Special Horticulture Areas (SHA groups) established by DAC. The team also conducted key informant interviews (KIIs) with several stakeholder groups. These included: Government staff such as local government employees, the Ministry of Agriculture (MAF) as well as the Municipal Department of Horticulture; the supermarket/buyers who were the prime private sector partners of DAC; influential farmers; and DAC Implementing Partner staff10. In addition, the evaluation team contacted former DAC project staff (some of whom now work with the supermarkets or with the new Avansa Agrikultura project), input suppliers, as well as Conoco Phillips, which was a private sector donor to the project. TNS Global, an international market research social survey company, was subcontracted to conduct the quantitative data collection from 182 farmer beneficiaries. DESK REVIEW The main components of the desk review of project documents included:  Review of project SOW, indicators, annual plan, mid-term evaluation report and project final report to examine DAC progress made against the project target objectives according to each annual work plan.  Review of project quarterly progress reports and financial reports and other key documents to assess project quarterly activities, achievements, lessons learned and challenges including project operational expenses.  Email correspondence with the former DAC COP to get a better sense of the project data as recorded during the life of the project. To share out the work, different members of the team reviewed different documents and important issues from the document reviews were discussed at team planning, data collection and data review meetings. 9 Documents included the Statement of Work, DAC deliverables, the project work plan, and quarterly progress reports. 10 Implementing partners include local NGOs Empresa Diak, HIAM Health, and Timor Aid 9 QUALITATIVE DATA COLLECTION Focus Group Discussions DAC worked with 37 farmer groups; 12 groups were selected (32% of groups) for FGDs. Four groups were selected from the original groups that worked with DAC in Aileu, including the greenhouse groups; 4 groups from the GDA1 expansion, and another 4 groups from the GDA2 expansion. The groups were selected purposively, to include communities t h a t worked with the various su pe rmarket sales outlets, and to include communities that joined the project at different stages. The total number of beneficiaries that participated in the 12 FGDs was 83, of which 29 were women (35%) and 54 were men (65%) – refer Table 1. Two of the beneficiary focus groups were women-only groups. We also used a female research assistant for the women only FGDs, and for some of the other interviews with women, so that female beneficiaries would be comfortable with the interviews. TABLE 1: DISTRIBUTION OF SHAS AND PARTICIPANTS FOR THE FOCUS GROUP DISCUSSIONS Municipality Group name Participants F M Totals Aileu Original Sites Liurai (Greenhouse) 1 2 3 Foin-Kaman (women’s group) 5 0 5 Hakiak Moris 3 4 7 Tasonih 2 5 7 GDA 1 Expansion Sites Fatubosa 1&2 3 8 11 Fo Liman ba Malu (women’s group) 5 0 5 Remexio (in-active) 3 4 7 Sarlala 2 5 7 GDA 2 Expansion Sites Bobonaro Miggir (In-active group) 0 6 6 Atabae 1 (Madatameta Haburas) 2 7 9 Ermera Gleno 1 & 2 1 7 8 Liquica Loes – Haburas Fini (H-4) 2 6 8 TOTAL 12 Groups 29 54 83 Key Informants District Horticultural Association (AHDISTAL) The DAC final report describes a regional association that supports the individual SHAs. The president of the regional association, and farmer association members, were interviewed. The DAC final report also mentions a training activity in Seloi which, according to the project, would guarantee the sustainability of the project through support to and building the capacity of farmers. Regional leaders in other municipalities were also interviewed. Supermarkets / Buyers The project primarily marketed vegetable crops through Kmanek, Dilimart and W-Four, in Dili. The owner of these supermarkets were interviewed to gain their perspective of DAC activities and the business model. The informants were asked about the market for farm produce, the ability of the 10 market to absorb more produce (by type of produce), as well as about perceived opportunities for exports. Josefina Farms is another project supported by the International Labour Organization (ILO). The ILO management staff were interviewed to gain a different perspective of the business model, and to compare it to their approach. Staff of the supermarkets were also interviewed, including 2 ex-DAC team leaders and 3 of the supermarkets’ technical staff. These supermarket buyers now support the farmers and the extension/ field staff now that DAC has ended. Some of the current extension staff were DAC employees under the project. Training Providers working with DAC Also interviewed were the training providers under DAC including Timor Aid, Empresa Diak, HIAM Health, and FarmPro. This was to understand the approach, effectiveness and sustainability of the training. The findings provided insights on the ability of farmers to continue with the technologies in which they were trained, and on the management of the project. Other Key Informants Other key informants interviewed included the Director of Horticulture in the Ministry of Agriculture and the Aileu MAF Chief of Department, as well as 3 input suppliers that provide inputs to the farming groups. Senior staff from ConocoPhillips, which was a DAC private sector donor, were also interviewed. 11 TABLE 2: DISTRIBUTION OF KEY INFORMANT INTERVIEWS (KIIS) Organization Name Position District Horticultural Association (AHDISTAL) AHDISTAL Silveiro Amaral B. Felix President of AHDISTAL Supermarkets / Buyers Kmanek Trading and Supermarket Clarence Lim H.M. Director of Kmanek Trading W-Four Supermarket Adi Setiadi Staff (use to work with DAC) Dilimart Supermarket Manuel Pereira Horticulture Manager FarmPro Peter Dougan Director (owner) ILO - Josephina Farm Jenny Ikelberg Value Chain Development Expert Kmanek Trading and Supermarket Antonio Field Staff Kmanek Trading and Supermarket Mercelino Field Staff Kmanek Trading and Supermarket Cipriano Field Staff Training Providers working with DAC Timor Aid Anina Bareto Project Officer HIAM Health Natalino Galoso and Sabino Mendes Trainer Empresa Diak Agostinho Sena de Jesus Staff of Empresa Diak Other Key Informants MAF Octavio da Costa Monteiro de Almeida Amaro Ximenes Inspector General of Policy, Planning and Monitoring Director of Agriculture, Horticulture and Extension MAF Aileu Municipality Galeni Galios Director ConocoPhillips Jose Lobato David de Araujo Country Manager External Relations Officer Ex DAC Staff Cesaltino Lopes Team Leader, Agribusiness Ex DAC Staff Bruno Benevente Operations Manager TOTAL 20 QUANTITATIVE DATA COLLECTION The evaluation team designed and tested the beneficiary household survey. Social Impact’s partner, TNS Global, then administered the survey to a total of a 182 households (HH) (33%) out of a total of 547 beneficiary households in the five municipalities of Aileu, Bobonaro, Dili, Ermera and Liquica. Prior to survey implementation, TNS conducted enumerator and supervisor training and final testing of the questionnaire in Liquica municipality. The data collection took a period of 2 weeks. The Social Impact Timor-Leste team provided technical assistance and field monitoring of TNS. At the end of the field work, data was formatted into an electronic data table by TNS and analysed by Social Impact. 12 TABLE 3: DISTRIBUTION OF BENEFICIARIES FOR THE QUANTITATIVE DATA COLLECTION Beneficiary Gender and HH Status Male Female Total Sample % Head of Household 113 10 123 18.2% Spouse of Head of Household 1 43 44 78.2% Other 13 2 15 3.6% Totals 127 55 182 100% Percent 69.8% 30.2% 100% Beneficiary by Municipality and Gender Male Female Total Sample % Aileu 87 41 128 70% Bobonaro 12 5 17 9% Dili 5 0 5 3% Ermera 14 2 16 9% Liquica 9 7 16 9% Total 127 55 182 100% Beneficiary by Project Phase Percent Original 31.8 GDA 1 40.3 GDA 2 27.8 Total 100 DATA SYNTHESIS The evaluation team hand-coded all qualitative data from the FGD and KIIs and inserted the results into a standardized matrix to identify common themes, frequency of responses, and distinctions in the findings between stakeholder groups. Quantitative data was primarily from the beneficiary survey, but also includes data on incomes and purchased from the DAC documents. LIMITATIONS As in any evaluation there are limitations imposed by the time and resources available, and by the recall ability of informants. The DAC project finished in February 2015 and field operations were wound down at the end of 2014. As a consequence, DAC staffs were no longer available to assist with organizing evaluation activities. H o w e v e r , t he evaluation team did manage to find the location of the SHA groups and the contact lists, and were able to conduct the focus group discussions (FGDs), KIIs and the survey of 182 beneficiaries. Ex DAC staff have been included in the KII but not as existing DAC employees. Because beneficiaries do not normally keep written farm records, they only gave their best estimates of historical crop sales and incomes. The data from the individual interviews with beneficiaries are therefore only an approximation to their true values. However, information provided 13 by other source was checked against the evaluation data and project information. Data on incomes and sales were cross-checked with the DAC / buyer crop purchasing data sheets. Beneficiary survey data is compared and contrasted with FDG information. Another challenge is that, there was no baseline survey done for DAC, so no comparison can be made for end line data, except for some production and income information. FINDINGS Q1. Do target beneficiaries report an increase in income as a result of project activities? If so, by how much? How was income used? Has income resulted in increased benefit to all members of the household? Incomes Beneficiaries report a substantial increase11 in income as a result of the DAC project activities. From the FGDs informants reported a typical weekly income from the sale of vegetables from $40 to $75 per week, selling to the supermarkets with whom DAC linked beneficiaries.12 Based on the sample survey of beneficiaries, for 95.6% of beneficiaries’ vegetable growing is the main source of income. Our sample revealed a mean income from selling vegetables or $71 per week, with respondents selling vegetables for 46 weeks in the year. The median income was $50, which corresponds to the information from the FGDs. Ninety-five percent (173 beneficiaries) of beneficiaries in the sample survey reported that working with DAC had increased their overall income, with no difference by gender. This increase in income was reported even for those whose main source of income was not growing vegetables. TABLE 4: INCOME REPORTED BY BENEFICIARY SURVEY RESPONDENTS (USD PER WEEK) Mean Income N Median All respondents 70.55 182 50 Male respondents 71.15 127 Female respondents 69.16 55 There were 8 respondents (4.4%) who reported that working with DAC had not increased their incomes. However, the amount of the sales varied significantly between beneficiaries in the same group and between SHA groups. For example, in The FGDs in Fatubosa and Sarlala (both GDA 1 groups) farmers reported income of only $10-$15 and $25 per week per person respectively; whereas in Madameta and Loes farmers reported sales to be typically $80 and $150 per person per week (both GDA 2 groups). In Tasonih farmers reported the range of income to be $10 to $180 per week per beneficiary; in Hakiak Moris from $25 to $100 per week per beneficiary. 11 Baseline income data is limited; data suggest an average monthly income of $26 per month from selling vegetable but this is only for one month, July. The project also extended the number of months that farmers were able to sell vegetables. 12 This amount is the gross income from the sale of vegetables, not net income. However, cash costs for inputs may be minimal because most farmers received free seeds and have few cash production costs. 14 Figure 1 uses a proxy indicator13 to estimate the total annual income per year from selling vegetables. This is the aggregated total for the annual crop income by type of crop reported by respondents, based on recall. This data shows that the distribution of income by farmer is heavily skewed to the right; 86% of respondents have estimated annual incomes of less than $2700 from selling vegetables, or less than about $60 per week for the weeks when they sell vegetables. FIGURE 1: PERCENTAGE OF RESPONDENTS, BY ANNUAL INCOME BRACKET FROM SELLING VEGETABLES Source: Beneficiary survey of 182 respondents For the 174 respondents that reported that working with DAC had increased their overall income, 81% of these said this increase in income was just sufficient; only 16% said their income had increased by a lot more.14 FIGURE 2 13 The aggregation of annual incomes reported for each crop type sold, for each survey respondent. 14 In Tetun: Naton, Uitoan, Barak Liu 58.79 27.47 8.79 1.1 1.1 1.65 1.1 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 USD 1349 and less USD 1350 - 2699 USD 2700 - 4049 USD 4050 - 5399 USD 5400 - 6749 USD 6750 - 8099 USD 8100 and more Percent Income Range Total Income from Vegetable Sales Just sufficient; 80% By a little more income; 5% By a lot more income; 15% By how much DAC increased incomes (Males) Just sufficient; 83% By a little more income; 0% By a lot more income; 17% By how much DAC increased incomes (Females) 15 Table 5 shows the percentage of respondents in our survey (182 beneficiaries) who grow vegetable by types of vegetable, the average income earned by respondents growing each type of vegetable, and the average price they received. TABLE 5: PERCENTAGE OF RESPONDENTS GROWING VEGETABLES BY TYPE, THE AVERAGE INCOME EARNED IN A YEAR, AND AVERAGE PRICES RECEIVED Crop % of Respondents Income USD (average) Price USD/kg (average) Bean-red (koto mean) 9% 114 2.5 Bean-snap (buncis) 33% 126 2.408 Bell pepper-capsicum (ai manas boot) 27% 210 2.94 Bok choy (modo mutin cina) 85% 176 1.38 Bombay onion (lis bombay) 16% 113 1.017 Broccoli (brocolis/kobi funan) 67% 163 1.64 Carrots (senoura/wortel) 25% 235 2.007 Cauliflower (Kobi funan) 70% 146 1.319 Coriander (koentru/ketumbar) 25% 188 1.86 Cucumber (Pepinu) 62% 111 1.509 Egg plant (berinjela) 65% 98 1.707 Kailan 58% 64 1.67 Kale (kangkung) 21% 107 0.464 Leek (olho poro/lis tahan boot) 16% 37 1.003 Lettuce (alfase) 84% 131 1.65 Long chili (ai manas naruk) 45% 68 2.02 Mustard (mostarda) 61% 138 2.32 Onion small (lis mean/ mutin ki'ik) 12% 110 0.89 Pechay 63% 121 2.15 Pineapple (ananas) 2% 197 0.78 Potatoes (fehuk ropa) 1% 30 3 Radish (rabanete/lobak) 21% 52 0.56 Red cabbage (repollumean) 40% 99 2.55 Rockmelon 10% 114 1.07 Round cabbage (repollu) 49% 113 1.38 Snow peas (ervilla) 13% 76 0.77 Spring onion (listahan) 16% 49 0.97 String beans (koto nurak) 34% 84 0.88 Sweet potatoes (fehuk midar) 7% 28 1.47 Tangerine (tanjerina/sabraka) 1% 785 38 Tomatoes (small) 46% 176 1.94 Tomatoes (big) 64% 193 0.98 Zucchini (abobriha/pipinu naruk jepang) 43% 225 0.94 Other specify 1 (Lainnya sebutkan) 9% 164 5.46 Other specify 2 (Lainnya sebutkan) 2% 185 0.64 16 Photo: Farmers aggregating and weighing crops for pickup by Kmanek in Tasonih, Aileu DAC Income Records To complement the evaluation data, the evaluation team also examined the income data provided by DAC. The source of this data is the SHA/group income data from the sales to supermarkets. Figure 3 shows average monthly income per beneficiary from sales to Kmanek supermarket for outdoor vegetable production for Sarin and Liurai groups in Aileu. These two groups were originally started by DSP but continued under DAC. However, the main intervention under DSP were the greenhouses which earn $15,000 to $20,000 per year for group members. The graph shows the seasonal peak in vegetable production during the dry season (Jun-Nov). For the groups of Sarin and Liurai, average income from growing outdoor vegetables is around $1,000 per year or $83 monthly, in 2013. FIGURE 3 Jan. Feb. Mar. April May June July Aug. Sept. Oct. Nov. Dec. Sarin $42. $24. $21. $18. $57. $36. $51. $95. $55. $99. $102 $53. Liurai $39. $47. $21. $43. $61. $84. $50. $116 $62. $115 $129 $37. 0 50 100 150 Income ($) Average Monthly Income / Beneficiary Sarin & Liurai Groups 2013 17 Figure 4 shows average annual income per beneficiary in 2014 for the Kmanek groups. The original groups15 worked with DAC from the 2010 and the GDA1 groups were started in 2012. The graph represents income only from the sales to Kmanek supermarket because beneficiaries may also sell some of their vegetables in the local markets16. FIGURE 4 Note: Averages are across groups In contrast, incomes tend to be less for groups which sold to Dilimart in 2014. For example, Kmanek groups earned on average $950 per year, whereas Dilimart groups $550 form sales to the supermarkets. These groups include GDA1 groups which are in Aileu and two GDA2 groups which were established in 2013 and only sold their first vegetables under project support in June 2014 (Figure 5 and Figure 6). 15 The original groups were Sarin, Liurai, Lequitura (started under the DSP project), plus Foin Kaman, Hakiak Moris, Moris Foun, Mudansa, and Tasonih 16 In our beneficiary survey 94% of farmers grew vegetables before joining with DAC; of these 96% sold in the local market, 41% to traders, and just 2.4 % to supermarkets. $- $0.5 $1.0 $1.5 $2.0 $2.5 Thousands 2014 Annual Income / Beneficiary for Kmanek Groups Average for Original Groups that worked with DAC since 2010 $ 998 (8 Groups) Average for GDA1 groups which were established in 2012 $ 899 (10 Groups) 18 FIGURE 5 FIGURE 6 Boklelo Sarlala Casnafar Gleno Railaco Total ($) $388.3 $536.8 $764.8 $204.4 $861.9 0 100 200 300 400 500 600 700 800 900 1000 Income ($) 2014 Average Annual Income / Beneficiary for Dilimart Groups GDA1 groups selling to Dilimart $ 563 (Boklelo, Sarlala, Casnafar - All in Aileu) GDA2 groups selling to Dilimart $ 533 (Gleno, Railaco - In Ermera) Jan. Feb. Mar. April May June July Aug. Sept. Oct. Nov. Dec. Boklelo 1514 0 255 461 373 372 442 475 478 653 235 177 Sarlala 1515 1180 846 1741 1517 1804 1507 1086 1371 1045 1250 1782 Casnafar 873 144 245 293 413 715 394 549 1443 581 284 184 Gleno 0 0 0 0 0 557 0 424 988 885 680 350 Railaco 0 0 0 0 0 439 2035 1814 2169 1921 1564 400 0 500 1000 1500 2000 2500 Income ($) 2014 Average Monthly Income / Benficiary for Dilimart Groups 19 Figure 7 shows total purchase by Kmanek and Dilimart from DAC beneficiary farmers for 2013 and 2014. These two main supermarkets accounted for most of the sales to supermarkets in 2013/2014. This shows an incremental increase in sales to these two supermarkets by 34% from 2013 to 2014. FIGURE 7 Table 6 is the income data recorded by the DAC project for the sales made by farmers to the supermarkets. This data was compiled by DAC from the weekly recording sheets used by farmers and supermarkets on the collection days to record the individual sales by farmers for each SHA group. In 2014 the total sales recorded are USD 408,000. This is equivalent to average sales per farmer of USD 85017 per farmer in 2014 to the DAC-linked supermarkets. TABLE 6: TOTAL INCOME RECORDED BY DAC FOR BENEFICIARY FARMERS (USD) Year Indoor Production Outdoor Production Totals 2010 3,046 12,954 16,000 3 months only 2011 23,768 67,491 91,259 2012 16,324 88,003 104,327 2013 33,122 290,516 323,638 2014 29,585 378,360 407,945 $105,845 $837,324 $943,169 11% 89% 100% 17 Assuming 481 active farmers, based on a dropout rate of 12% of direct beneficiaries (547 listed beneficiaries x 0.88). - 50,000 100,000 150,000 200,000 250,000 300,000 350,000 400,000 Jan-March April-June July-Sept Oct-Dec Annual Totals USD Dollars Quarterly and Annual Sales by DAC Beneficiaries to Kmanek and Dilimart Supermarkets 2013/2014 2013 2014 20 TABLE 7: LIST OF DAC SUPPORTED HORTICULTURAL GROUPS (SHAS) Source: Developing Agricultural Communities (DAC) Final Report, February 2015 21 Who Benefited Farmers18 told us that all the members of the household benefit from the additional income and this is supported by the fact that the income is mainly spent on daily living expenses. Children also benefited because money was reported to be spent on school fees as a high priority. From the sample survey, with the exception of one male beneficiary, all respondents reported that all members of the household benefited from the extra income generated by the DAC project. Over 90% of respondents reported spending the addition income on school fees and daily living expenses. Fifty-five percent (55%) reported spending money on traditional ceremonies (adat), fifty three percent (53%) on home improvements, followed by asset purchases (23%), savings (18%) and business (only 5%). FIGURE 8 A common complaint from farmers in FGDs was that the amount of additional income they earned was not sufficient for reinvestment; most of the money went on daily living expenses. Reinvestment on the farm was a lower priority than other uses mentioned by beneficiaries. Some groups reported that children were given their own small plots in which to grow vegetables and in some cases the children are allowed to keep the income they earn from growing vegetables (e.g. Seloi, Liurai, Gleno groups). Decisions about Money In the Timor-Leste context it is common for women to keep the money. However, spending decisions are usually made by the husband and wife together. 18 No differences between male and female respondents School fees House improving Purchase asset (eg TV, radio, motorbike, etc.) Saving Business Traditional Ceremony (Adat) Daily living expenses Other (specify) 0% 20% 40% 60% 80% 100% 120% Percent of Respondents How the Additional Income was Spent 22 Women in FGDs reported that growing vegetables for the DAC project really benefited women because the buyers came to the village to collect the produce from village aggregation points. The most successful beneficiaries were the most diligent ones. Even though the women help increase the household income, their family responsibilities do not change. The domestic work of the family is still the responsibility of the women. Women in the focus groups say they are interested to implement more activities to improve their knowledge and skills, in addition to increasing the family income. Activities mentioned were not only planting more vegetables but also tailoring and making fruit juices. When asked in the survey of beneficiaries about who keeps the money, if the beneficiary was male 54% gave the money to their wives and 43% kept it themselves. For women, 91% of beneficiaries kept the money themselves and only 4% gave it to their husbands. If the household head was male, slightly less of the men gave the money to their wives, 43% compared to 54%. If the household head was a woman they were also more likely to keep the money themselves, 94% compared to 91%. More than half of both male and female respondents reported that they would make decisions together about how to spend the money. Q2. Were there any unintended outcomes (positive and/or negative) that resulted from project activities? (i.e., child labour, increased school attendance, increased household burden for women) Market Supply One unintended consequence is that farmers and supermarket buyers now report that the market is oversupplied with some types of vegetables promoted by the project; particularly for the most commonly grown vegetables such as tomatoes, bok choy, lettuce, and cabbages. In fact, Kmanek supermarket is sometimes offering a “buy one get one free” special promotion for some produce lines in its stores. Oversupply is greatest during May to July19, in particular for the green leafy vegetables which are fast growing. One supermarket reported there has been too much capsicum and it could not all be sold. A full container of lettuce went rotten and had to be thrown away. Some produce lines did not match the required grade. 19 Key informant, Dilimart 23 FIGURE 9 However, the situation with regard to oversupply can be hard to interpret because the main growing period for outdoor vegetable production is the start of the dry season when there is still water available in irrigation canals. Growing outdoor vegetables during the rainy season in Timor-Leste is a problem because of heavy rains which damage unprotected crops and the high humidity which increases the likelihood of diseases (such as mildew and other fungal diseases). DAC promoted plasticulture (mainly plastic tunnels) with pest and disease management and this makes it more feasible to grow vegetables during the rainy season. At the same time, during KIIs, the buyers report that there is insufficient production of certain types of vegetables that the supermarkets want; these crops include cauliflower and broccoli, carrots and chili; another problem is obtaining a constant supply of produce throughout the year. What is certain, however, is that the market for an ever expanding supply of vegetables for the Dili market is very limited and the existing producers, as well as new projects, will need to focus on other markets and on processing surplus vegetables. There is also very limited purchasing capacity for higher priced vegetable crops in the districts outside Dili due to the limited amount of economic activity compared to Dili. DAC aimed to improve the quality of vegetable production and to offer premium prices to farmers for this better quality produce. The DAC project put in place a fixed price structure for vegetable crops which meet the supermarkets’ required quality standards. This is supported by the supermarket buyers which fix the price with farmers for a 6 – 12 month period20. This provides farmers with certainly with regard to prices and encourages farmers to grow commercially for the market. This fixed price arrangement does not allow prices to fluctuate in response to supply and demand. However, it works because the production of the higher quality produce that the supermarkets want is a relatively small percentage of the total domestic vegetable market. Since DAC ended, supermarkets have negotiated both price increases and price decreases for certain produce lines and farmers report that they are discouraged when the prices agreed are less than they 20 In some examples informants reported the period to be 6 months and in some 12 months. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Percent Crops Top 10 Crops Grown in DAC % of respondents that grow... 24 received previously. Farmers do not yet accept that when producing for the commercial market, price fluctuations are normal. Cost Effectiveness Although unlikely to have been envisaged, one consequence of the project was that the cost of the project per beneficiary farmer was very high. The DAC project was USD 8.3 million; this included USD 0.95 million provided by Conoco Philips for the GDA 1 and GDA 2 expansion groups.21 If this total sum is divided by the total number of beneficiaries (547 direct beneficiaries in all) then the cost of the project per beneficiary household is over $15,000 over four years. This compares to $2,630 per beneficiary household for the Avansa Agrikultura project (USD 19.2 million divided by 7,300 HHs over five years). So DAC was a very expensive project when considering the cost of the project and the number of beneficiary households involved in the project.22 The cost of the DAC project per beneficiary farmer would have been even greater if it had not been for the extension of time granted to the project and the additional farmers involved in GDA 1 and GDA 2 expansion because the number of farmers involved in the first phase of the project was only 203 compared to 547 overall with the two expansion phases supported by Conoco Philips. Production Supermarkets planned to control the supply of vegetables by controlling the amount of seed they distributed to farmers. However, farmers do not always plant all the seeds they are provided with by the supermarket suppliers and this can result in production being less than expected. Farmers also do not always plant the required area. This was confirmed by farmers in FGDs and supermarket buyers in KIIs. It proved to be ineffective trying to control the supply of vegetables through the control of seed distribution; for vegetable production to be managed well it is important to plan the number of seedlings to be planted out and the area to be planted. Demonstration Effect One of DAC’s successes was to develop an approach that was widely copied. There has been an expansion of the number of groups over and above the groups supported by DAC. For example Dilimart has supported 7 additional groups (compared to 6 under DAC, now making 13 groups) and Kmanek 3 or 4 additional groups (making a total of about 25 groups). This expansion of groups by Kmanek and Dilimart was in part motivated by the difficultly they faced making sufficient collection of vegetables from the DAC established groups alone. However, the buyers have found that increasing the number of groups has increased the transport costs, because supermarkets must now pick-up produce from more groups, further afield. Not only did the supermarkets support more groups than were supported originally by DAC, but MAF Aileu also copied the approach to 3 new groups covering 76 households; MAF also have a plan to establish yet more groups. According to MAF, the focus of these new groups will be cauliflower and broccoli production as priority crops under the MAF strategic plan. Sales from these groups are to the existing supermarket buyers supported under DAC. 21 Conoco Phillips support agricultural sector projects because they believe they have a social responsibility, and that there is a misconception of where jobs will come from in future. The oil and gas sector will create relatively few jobs, whereas the agricultural sector currently employs the most Timorese and has the potential to offer more people employment. 22 In our sample survey, there were 6.8 members in the average household; based on this estimate, the total number of people in DAC beneficiary household would be around 3,700. 25 Women and Children During FGDs women did not mention that being involved in the DAC project created additional work that was burdensome. They reported that they were willing to make extra effort in order to earn additional income. However, from the sample survey, 23% of respondents say that the DAC project did create extra work for women that was difficult to manage. Two thirds of these respondents were men and one third were women, so men do recognize that women’s roles can be too demanding when they also have to cope with project activities. Only a few respondents (10%) reported employing labour as a result of DAC activities. Photo: DAC outdoor vegetable production with plastic tunnels (greenhouse behind) in Sarin, Aileu TABLE 8: NUMBER AND PERCENTAGE OF BENEFICIARIES WHO REPORTED EMPLOYING LABOUR AS A RESULT OF DAC ACTIVITIES Frequency Percent (%) Did Not Hire Labour 163 89.56 Hired Labour 19 10.44 100% Whilst the project did benefit children in a number of ways (money was available for school fees, and children were able to earn their own pocket money or income), we found through FGDs that there were some instances of children dropping out of school because they wanted to earn money from vegetable farming. Children do spend time working in vegetable gardens but in Timor-Leste culture (mostly in the rural area) it is normal for children to work on the farm and this is to assist their parents rather than to work for others. 26 The observed cases of children dropping out of school early were in Loes (2 children aged 13 – 14) and in some cases children complained that they had to spend too much of their spare time working in vegetable gardens. Nineteen respondents (10%) reported hiring additional labor because of DAC activities. Over 50% of these respondents hired adults full-time and hired, on average, 2.7 adults. Respondents that hired youth (20 years of age or less), either part-time or full-time, hired more youth than respondents that hired adults. In total, full-time jobs were created for 27 adults and 34 youth. In total, part time jobs were created for 7 adults and 90 youth. Aggregated over the entire 37 beneficiary groups (547 beneficiaries) this would be equivalent to full-time jobs being created for 81 adults and 102 youth. In total, part time jobs would have been created for 21 adults and 270 youth23. One positive consequences is a success story told by Maria Mendonca Carvalho, a group member from Remexio. Maria reported that she did not expect to send her husband to a university. However, after joining with DAC horticulture group in 2013 – 2014 and with the sufficient income they received selling vegetables weekly, she was able to send her husband to continue studying in the National University (UNTL) under the faculty of agriculture. Farm Inputs DAC intended to support the establishment of input supply shops and five in Dili were assisted by DAC. However, farmers complain that farm inputs, including the plastic required for tunnels and plastic mulch, is not available in the districts. FarmPro (an independent agribusiness social entrepreneur) attempted to establish input supply shops in districts but found that there was insufficient demand. Kmanek also confirmed that there was insufficient demand for local input supply shops; they are able to supply their farmers with seeds and other inputs through their field workers from Dili. Thirty-eight percent (38%) of survey respondents reported that there were inputs they wanted but which were not available. Most commonly mentioned items include plastic for tunnels, small tools, and irrigation items such as water tanks, hoses, water pumps. Also mentioned was spare parts including for backpack sprayers (spray nozzles). Farm mechanization equipment for cultivation was mentioned by just a few farmers. DAC originally provided inputs to farmers free, in particular for the demonstration sites and for the initial establishment of one tunnel per farmers for SHA group members. The supermarkets import the seeds and distribute these to farmers in a controlled manner in an attempt to manage the supply of vegetables of the type and quantity they want. Because of this, at least some of the market for inputs was controlled by DAC and subsequently by the supermarkets, in particular for seeds which farmers complain they cannot buy locally. Seeds are provided mostly by the supermarket buyers for the types of vegetables they want to purchase; 31% of respondents also reported receiving seeds directly from DAC staff24. Eighty seven percent (87%) of farmers reported using plastic growing tunnels25. However, only 67% of respondents used agricultural chemicals and only 57% inorganic fertilizer. If using chemical fertilizer, agricultural chemicals, and small tools, the source of supply is mainly from the local market (33% - 49%) with input supply shops supplying 11% - 15%, with the balance coming directly from DAC staff or the supermarket buyers. 23 Based on 547 direct beneficiaries being three times the number of our beneficiary respondents in the survey. 24 Farmers may not always make a correct distinction between DAC and Supermarket staff. Some ex-DAC staff now work for the Supermarkets. 25 One key informant also reported that not all DAC beneficiaries did receive plastic for growing tunnels 27 With DAC most respondents report receiving vegetable seeds and plastic free (83% and 92% of respondents respectively). However, agricultural chemicals and chemical fertilizer is mainly purchased for cash. Very few respondents report getting credit for input purchases although 11% of respondents do report using credit to purchase vegetable seeds. Interestingly, a significant number of respondents report receiving agricultural chemicals, chemical fertilizer, and small tools free of charge. TABLE 9: SOURCE OF INPUTS BY METHOD OF PAYMENT Input Type Free Cash Credit Seeds 83% 6% 11% Chemical fertilizer 25% 73% 2% Agricultural chemicals 36% 63% 1% Small tools 46% 51% 3% Plastic 92% 5% 3% Other 47% 53% 0% Association Office MAF gave AHDISTAL the old MAF office site in Aileu to use as an office but a subsequent dispute over the land ownership meant this site was unable to be used by AHDISTAL. Currently Kmanek pay the $300 a month rent for the alternative ADHISTAL office in Aileu. Q3. Are linkages between farmers and buyers likely to be sustainable? Timeline DAC closed in February 2015 after four years. Whilst the three supermarkets assisted by DAC are still buying from DAC groups, there are signs of a lack of sustainability. For some the GDA 2 groups26 the project closed too soon after the groups were started and the groups were marketing vegetables for less than a year after they began working with DAC.27 This is particularly the case for Miggir, where implementation was not properly completed.28 Three groups are listed by DAC as having been started but not functioning at the end of the project. These are Remexio and Talitu (GDA 1 groups) and Miggir from GDA 2. This represents 8 percent of the 37 listed DAC groups but less than 3% of the listed farmers. In Remexio production was insufficient for Dilimart to continue making weekly collections from the group, the road is bad and the location relatively remote. This group was originally supported by the NGO Child Fund and has continued with fewer groups members (only 4 member from the original 12 members) selling to the local and Dili markets. In Miggir training was carried out by DAC but then DAC intervention stopped. The Miggir farmers did not know why DAC stopped supporting them but the location is relatively remote and there were reports that the demonstration plot may not have been successful. Talitu never really got going and the reason appears to be that the site was poorly selected with insufficient water supply. A higher percentage of respondents (51%, 25 beneficiaries) in the GDA2 group reported having expanded their vegetable production beyond what they had achieved under DAC than in the original DAC groups (34%, 19 beneficiaries). This is likely to have been because DAC only operated in the GDA 2 group quite late and not all activities were completed under DAC. 26 As listed as GDA2 groups in the DAC Final Report 27 The DAC project intended to start working with GDA 2 groups in August 2013 but this was postponed until May 2014 because of delays in the funding agreement with Conoco Phillips, MAF and USAID. 28 Miggir is reported in the DAC final report as an Inactive Group. 28 AHDISTAL (Aileu District Horticultural Association) AHDISTAL was only formed in late 201429, which was too late for it to become a sustainable organization capable of supporting farmers effectively30. The project started with a concept of having five regional groups as part of a horticultural federation, or possibly a cooperative, but this idea was later changed to the concept of AHDISTAL. As its name suggests, AHDISTAL only supports the SHA groups in Aileu and there is no representation through AHDISTAL or any other district grouping for the groups that are not in Aileu District. AHDISTAL has 20 affiliated groups31 which means that there are another 17 groups (14 if no longer functional groups are excluded) that have no farmer organization to represent them. Kmanek currently pays the office rent of $300 a month on behalf of AHDISTAL in Aileu; some of the Kmanek supermarket staff work to support the association and are based at the AHDISTAL offices. The staff of AHDISTAL including the chairman, are all unpaid volunteers and there are no professional staff. Only two thirds of the members of the association pay their membership fees ($1.00 per month per member) on time and so there are no regular funds to ensure that AHDISTAL can continue operations, even with Kmanek support to pay the monthly office rental costs. Ninety-two percent (92%) of survey respondents reported that they did not pay any membership fee to their own horticultural group; only 8% paid fees to their own group. The association complained that they have no training infrastructure in Seloi (which was mentioned in the DAC Final Report as a location for training that would ensure DAC sustainability) in the form of a training room. Training is conducted by the farmers themselves using their own plots as demonstration sites. The association has only two motor bike which were provided by DSP (Dezenvolve Setor Privadu / Private Sector Development project, a USAID funded project which preceded DAC) and no bus or truck transport for transporting farmers or equipment. The computer they use was provided by DAC and they proudly related how they have paid for the repair of the computer and printer. The two containers that are now used as storage space were donated by DAC. When interviewed, the chairman of AHDISTAL requested the government to provide training facilities, for transport including new motorbikes and a bus, and for extension and training support. Horticultural Working Group (HWG) DAC established a Horticultural Working Group (HWG) with the Ministry of Agriculture and Fisheries (MAF) and other stakeholders. The HWG rarely met in the last two years of the project. It has recently been reactivated by the Avansa Agrikultura project and has met twice under the auspices of the new project, and a third meeting is planned. One issue mentioned by the current Director of Horticulture, is that projects frequently follow individuals with which they have established links within MAF; when these individuals move due to staff changes or restructuring, projects can fail to work with new staff or with the appropriate Directorate. DAC worked originally with the Director of Horticulture whose responsibility was changed to Food Security. This appears to be the time that the activity of the HWG declined and the 29 According to the DAC Final report the Associacao Hortikultura Distrito Aileu (AHDISTAL) was formally registered with the Ministry of Justice on August 18, 2014. 30 An Ex-DAC key informant advised that there was an association(s) formed during 2012 for the greenhouses established by DSP previously. Horticultural groups do exist for both Sarin and Liurai greenhouse groups but these groups do not appear to be integrated into AHDISTAL which cover the other SHAs that DAC established in Aileu. 31 The DAC final report mentions 26 groups, but the Chairman of AHDISTAL mentioned 20 groups are now active. 29 Directorate of Horticulture became disassociated with the HWG. Another issue is that MAF has been recently restructured and further restructuring may occur. Maintenance Farmers in FGDs mentioned that they don’t have enough money from vegetable growing to be able to invest in on-farm maintenance and in infrastructure. This is true for many farmers who spend most of the income from selling vegetables on daily necessities. There may also be problems for farmers to effectively cooperate to share resources for jointly owned infrastructure such as water pipes and water tanks which can service more than one farmer. When DAC started working with farmers, the project supplied all the inputs for the demonstration sites free of charge. They also provided all SHA group members with one 25 m tunnel and other basic tools for vegetable farming. For some SHAs, they also provided drip irrigation and water tanks, at least for the demonstration sites. Most farmers told us they received all the inputs from DAC free of charge. However, what happened in reality seems to be that there was a basic distribution of inputs to establish and operate one 25- meter-long tunnel for all group members; however, if farmer members wanted to expand production to a larger area this was to be at their own cost. For many groups visited for the evaluation, there was insufficient water reticulation in the form of water pipes, drip irrigation, and water storage tanks. This means that farmers have to water their crops by hand from the river or from the irrigation canal. This is extremely time consuming and the farmers really require drip irrigation, if feasible, or hoses supplied by gravity feed to water their vegetable crops. In Lequitura there is a greenhouse but it is damaged and has not been repaired even though this was one of the original DAC groups, in fact this group was also supported by DSP (a previous USAID project) which built the greenhouse and provided training and market linkages. In Sarlarla the group leader had a damaged water tank which was unrepaired and the water supply piping was made from bamboo rather than permanent piping. In Atabai there were damaged tunnels which had not been repaired; however, it is possible that these could be repaired before the start of the rainy season. Photo: Broken gravity water storage tank in Sarlala, Aileu 30 However, from the respondents’ survey most beneficiaries reported that they were able to maintain the greenhouse or plasticulture. TABLE 10: PERCENTAGE OF BENEFICIARIES REPORTING WHETHER OR NOT THEY ARE ABLE TO MAINTAIN THEIR GREENHOUSE (2 GROUPS ONLY) OR PLASTIC TUNNELS Male Female Overall No 12.6% 20% 14.84% Yes 87.4% 80% 85.16% The beneficiaries who earn more from selling vegetables were also more likely to maintain their greenhouses or plastic tunnels (80% for those earning less than $1350 per year compared to 100% for those earning more than $4050 per year). Lequitura Group Three groups which became part of the original DAC groups in Aileu are Sarin, Liurai and Lequitura. These three groups were started by DAC’s predecessor DSP; as part of this support three greenhouse were constructed (one at each site) and farmers were linked to the market through the Zero Star trader who received a cool store truck from DSP. Through a combination of ineffective management by some of the group’s elected representatives and as a result of a disagreement32 with Zero Star, the Lequitura greenhouse is no longer functional and Zero Star has ceased to operate. Photo: DAC Greenhouse in Liuria, Aileu 32 Informants reported that the disagreement was about payment for crops sold and the ownership of the greenhouse. 31 Extension Eighty-one percent (81%) of survey respondents have been recipients of extension services, with the most frequently cited provider of extension services being the DAC project (97% of respondents who received extension services received services from DAC). However, 47% of those who received extension services under DAC also said they received services from the supermarket buyers (not just from DAC staff). FIGURE 10 Ninety-four percent (94%) of those receiving DAC extension services were either very satisfied (83%) or satisfied (11%) with the services, according to the survey results. By contrast 90% of respondents expressed they were either very satisfied (74%) or satisfied (16%) with the extension services from supermarket buyers. Now that DAC has ended, 11% of respondents reported that extension services have declined, 10% say they have improved, and 80% say they are the same now as with DAC. Whilst supermarkets do continue to support farmers now that DAC has ended, in particular Kmanek which has 14 horticultural staff, the supermarkets now rely on recruiting already trained staff and on the staff they took on after DAC closed. They do not have the ability to train their own staff in technical horticultural topics, and cannot provide the training and other resources that DAC were able to offer. Dilimart have 4 field staff working with farmers; 2 in Aileu and 1 each in Ermera and Loes. One existing staff member previously worked with DAC. Dilimart mentioned that DAC did not provide sufficient training to their horticultural staff. Credit DAC did introduce farmers to lines of credit other than for seeds. DAC originally liaised with the Government bank (BNCTL), the NGOs Moris Rasik and Tuba Rai Metin. The BNCTL visited the DAC 0 20 40 60 80 100 120 140 160 Supermarket Buyer MAF Extension staff NGO DAC project Others # of respondents Extension Service Type Provider of Extension Services # of respondents that 'receive extension services' (n=147) 32 groups in Aileu District. The field visit and the book keeping system established by DAC was persuasive enough for BNCTL to agree to provide credit to farmers33. BNCTL first agreed to provide farmers with loans of $150 to $300; then later increased this amount to $500 as the first loans were repaid34. The most recently recalled information from our key informant was that BNCTL was now advancing loans up to $2,000 and about 120 loans of such loan may have been advanced to farmers35. However, in our beneficiary survey, only 8% of respondents (equivalent to about 40 farmers across all DAC beneficiaries) reported having access to any form of formal credit and most of these accessed credit through the supermarket buyers who later recover the money in stages from the farmers’ sale income. This trade credit is normally for farm inputs, including plastic for the growing tunnels. Only 8 farmers in our sample survey of 182 farmers (4.4% of beneficiaries) reported having access to credit from the BNCTL. This would equate to about 24 DAC beneficiaries36. DAC introduced farmers in Aileu to the Government bank; however this was done in 2014 and was too late to develop sustainable relationship with the bank and a proper understanding of credit commitments by farmers. In our survey of beneficiaries, a third of respondents say that the main use of credit is for daily living expenses, farm crop and livestock inputs, and to buy farm equipment. Very few respondents report borrowing money for other uses, although some do borrow money for house repairs. TABLE 11: NUMBER OF BENEFICIARIES REPORTING ACCESS TO FORMAL CREDIT BY GENDER Male Female Overall No 91% 85% 90% Yes 9% 15% 10% 100% 100% 100% TABLE 12: PERCENTAGE OF BENEFICIARIES WITH ACCESS TO CREDIT, BY SOURCE OF CREDIT % of Beneficiaries with Access to Credit % of Those with Access Supermarket 7% 63% NGO 1% 11% Farm input supplier 0% 0% Crop trader 1% 5% Bank 4% 42% Note: Beneficiaries may have more than one source of credit Business Profitability Both Kmanek and Dilimart representative stated in KIIs that they supported DAC primarily because they wanted to support Timor-Leste’s farmers and that their relationship had a social aspect. Both 33 Key informant: Mr. Bruno Benavente, Ex DAC Operations Manager 34 The Quarterly Report for Jan-Mar 2014 mentions total loans from BNCTL of $11,300 to 35 farmers – average therefore $323 per farmer as at March 2014. 35 Ibid 36 Possibly BNCTL also advanced loans to non-DAC farmers; this would explain the discrepancy between the key informant lending figures and the figures from the sample survey of DAC beneficiaries. 33 claimed that the business was hardly profitable and made large losses in the early stages. Without commercially sensitive operations data it is hard to categorically state that there is no profit in the business for the supermarkets. However, the vegetable trade is only a small part of the supermarkets’ overall business and they are not reliant on this trade to contribute significantly to profits. There is no data available about the profitability of the business model for farmers, or for the value chain overall37. So there is no real data or even model data to demonstrate the profitability of enterprise to farmers, despite the “farming as a business” training provided to farmers. Q4. Did technical assistance to farmers reach all intended beneficiaries (i.e., women and youth)? In FGDs, respondents reported that some of the original beneficiaries selected by the project as group members later dropped out because they found the work too hard, or the time commitment to be a problem. This was reflected in a decline in some of the group membership numbers reported to us (Table 13). Another problems was a lack of education which meant some of the training was difficult to follow and training could be conducted in Indonesian, a language some farmers do not understand. TABLE 13: MEMBERSHIP OF FOCUS GROUP DISCUSSION GROUPS COMPARED TO DAC MEMBERSHIP LISTING Group Name from FGDs Original Members from the DAC List Current Members from FGDs No. of Beneficiaries Dropped Out Liurai 22 23 1 Foin Kaman 22 22 0 Hakiak Moris 17 15 -2 Tasonih 28 23 -5 Fatubosa 1&2 36 15 -21 Fo Liman ba Malu 19 19 0 Remixio (inactive) 5 4 -1 Sarlala 31 24 -7 Gleno 1&2 19 21 2 Loes 8 12 4 Atabae (Madameta) 10 12 2 Miggir (inactive) 10 10 0 Total 227 200 -27 Dropout Rate Overall 12% All the original DAC groups with whom the project started working since 2010 received assistance. However, the GDA 2 expansion groups were started too late38 and the project finished before the DAC intervention was able to be consolidated. Groups reported that they lacked some of the training materials provided to prior groups and the group registration with the Ministry of Justice was not completed. Furthermore, the GDA 2 groups being in Ermera, Liquica and Bobonaro are outside of Aileu District and are not part of AHDISTAL. 37 There is only the record of the gross sales made by farmers to supermarkets and this does not include the sales farmer make to other buyers and in the local markets. 38 DAC planned to start the GDA2 groups in August 2013 but the agreement between MAF, USAID and Conoco Phillips was signed later than expected; consequently the groups did not start until May 2014. 34 FIGURE 11 Women and youth were included in the training and the women who did join as beneficiary group members reported that they benefited from project activities in the same manner as men. In the list of DAC beneficiaries provided by the project 32% of group members are women. Those women we interviewed in the FGDs reported that there were no barriers for them to participate in project activities including attending the one month training in Indonesia. TABLE 14: PERCENTAGE OF MALE AND FEMALE RESPONDENTS WHO WERE RECIPIENTS OF DIFFERENT TYPES OF TRAINING Type of Training Males Females Environmental Mgmt 61% 55% Accounting/Recordkeeping 65% 56% Farming as a Business 64% 53% Nutrition 65% 45% Agriculture 98% 100% Management 61% 62% Inventory Mgmt 47% 38% Produce Supply Mgmt 49% 42% Grading System 55% 64% Improved Market Information 87% 76% Market Linkages 92% 84% Value Chain Introduction 81% 75% Some groups have elected women to official positions. For example in Gleno 2 (Ermera), and Foin Kaman and Fo Liman in Seloi have elected women officials. However, the women did state that there is still some discrimination in the opportunity to be elected to the group leader position. In particular women were entrusted to the position of treasurer. In fact, in Foin Kaman there are 16 members (11 women and 5 men) but all the elected committee members are women. In Tasonih, Foin Kaman and Gleno 1, there were youth members who are still at school. However, not all groups did include youth as full members. Youth benefited more from their parent’s activities 54% 58% 55% 46% 42% 45% 0% 20% 40% 60% 80% Male Female Overall Percent Gender % of Beneficiaries who have used DAC Farm Taining Manuals, by Gender No Yes 35 with the group rather than as full members of the group. Only 9% of beneficiaries in our respondent sample were under 25 years of age. Photo: Children helping in the vegetable gardens in Remexio, Aileu FIGURE 12 From the sample survey, seventy six percent (76%) of respondents reported that there was nothing that stopped or limited them attending the DAC training. Twenty four percent (24%) said that there 0.6 8.2 28.6 24.7 22.5 9.9 5.5 0 5 10 15 20 25 30 Under 18 years 18 to 24 years 25 to 34 years 35 to 44 years 45 to 54 years 55 to 64 years 65 years or older Percent Beneficiary Age 36 were things that stopped them attending the training, with a slightly higher percentage for men than women. Over half of these reported that they had not attended primary school. Q5. What variations were there in implementation approaches or strategies and what was most effective/efficient in achieving results? From the beneficiary survey, the most commonly stated benefit of participation in DAC activities is access to training (90% of respondents), access to new markets (83%), and increased income and savings (46%). Only 6% of respondents mentioned access to credit as being a main benefit. There was no observable difference between men and women. Training A highlight of DAC, consistently commented on favourably by farmer beneficiaries was the technical horticultural training39. This training was for one full day every week over a three month period. At the start of the DAC this training could last for a longer period. Some of the beneficiaries were also selected to travel to Bandung Indonesia for a one-month long training which was valuable as it allowed farmers to see modern horticultural practices used in Indonesia. Participants lived with Indonesian farm families. When responding about the technologies they have learned under DAC, the 5 most useful technologies are composting, making organic fertilizer, plastic tunnels, seedling nursery and using raised beds to grow vegetables – in that order. Most of the technologies had been learned and applied by beneficiaries and they stated that they will continue to use the technologies. FIGURE 13 However, only a few of the respondents reported having learned and applied irrigation and integrated pest management (IPM). About one third of respondents had both learned and applied these techniques. Fifty five percent (55%) of respondents had learned about drip irrigation and of these only 39 Our beneficiary survey confirmed that all farmers had received this training. 0 20 40 60 80 100 120 140 160 180 200 Number of Respondents Technology Type Technologies Learned / Will Continue to Use Learned Will continue to use 37 half reported that they will continue to use it. Across all respondents, only 2 reported drip irrigation to be the most useful technology. In the FGDs, farmers said that they did not have drip irrigation and watering hoses because they lack gravity feed for water and water storage tanks due to the high cost of the tanks and plastic pipes. If water is to be pumped then there is a cost for electricity. DAC did install some demonstration sites for drip irrigation, but the technology and infrastructure was not distributed free to all farmers. Fewer survey respondents report receiving the non-agricultural topics such as environmental management, nutrition training, produce grading and the areas involving financial and business management and inventory and supply management. FIGURE 14 Generally respondents report high levels of satisfaction with the training. For the agricultural training and the training related to market information and market linkages 96% of respondents say they were either satisfied or very satisfied with the training; 72% being very satisfied with the training. The most relevant training was the agricultural training; 65% of respondents said this was the single most valuable training. The agricultural training was by far the most relevant training. Only 10% of respondents by contrast listed market linkages/ partnerships training, or farming as a business training, as the most useful training offered by DAC. In some cases informants said they did not understand the language used in the training (if in Bahasa Indonesia and delivered by an Indonesian trainer). Some of the written manuals are also in Indonesian, and in addition many beneficiaries are illiterate and so cannot understand the manuals. Of the 51 respondents in the beneficiary survey who made suggestions as to how the training could be improved, 21 (or 12% of respondents surveyed) mentioned that they were unable to understand the language of the training. 0% 20% 40% 60% 80% 100% 120% Percent of Respondents Training Type Training Received by Beneficiaries 38 DAC used Indonesian consultants to manage the training in Timor-Leste. Some of the groups mentioned that the consultants lived in their community and were able to provide support to farmers more frequently than for only the once-a-week training. DAC produced two training manuals in Tetun:40  “Manual Identifikausaun” in colour on pets and diseases identification.  “Matadalan Kuda Modo Timor-Leste” a horticultural crop growing manual. Manuals were not given to every farmers but rather to the group leaders. The group leaders keep the manual as a reference for other farmers. However, most farmers told us they already felt confident in the knowledge contained in the manual and did not need to refer to them often. The Indonesian consultants working for DAC also produced a manual “Sistem Produksi Sayuran Indoor Dan Outdoor” in Bahasa Indonesia. Farmers who traveled to Indonesia also collected a range of local training materials in Bahasa Indonesia from their time in Indonesia (For example: Teknik Budidaya Zucchini – a technical growing manual for Zucchini). Farmers also reported that in Indonesia they were able to work in teams to focus on specific vegetable crops and this gave them confidence in the skills gained. After the training in Indonesia farmers reported that they made presentations on what they had learned in Indonesia to other farmers, to MAF staff, and to local Government officials on their return to Timor-Leste. DAC also provided two other trainings mentioned by farmers. These were:  “Book-keeping Training” delivered by Empresa Diak  “Farming as a Business” delivered by Timor Aid. Both of these short trainings only lasted for one day. Book-keeping covered the record keeping required to track farmers’ group vegetable sale on an individual basis so that on the weekly pick-ups by supermarkets a record could be validated of the farmers’ individual sales to the supermarket buyers. This training has delivered the results needed and the evaluation team were shown the completed weekly record sheets kept by the farmers which recorded the volume and value of individual sales. However some of the farmers are illiterate and found the training hard to follow. The group leaders are mostly responsible for keeping the sales records on behalf of other farmers. 40 The DAC Final Report also mentions other manuals or guides: Pesticide Label Guide; Pesticide Safer Use Training guide; Water Use and Irrigation Training guide. 39 FIGURE 15 The Farming as a Business training was of insufficient duration and farmers found it hard to follow. In theory it included how to calculate income and expenses for vegetable production and how to calculate the profit from vegetable production. However, most farmers found the duration of the training too short for the topics covered. This shortcoming was recognized by Timor Aid; however, they delivered what they were contracted to provide on behalf of DAC. Some technical training proved inappropriate to specific locations. For example, potato cultivation training was provided in lowland areas which were not suitable for growing potatoes. In Miggir farmers reported that the raised bed system of vegetable production was not suitable for the soil and water conditions there. In windy locations in Aileu, damage to greenhouses and tunnels is common and the quality of the plastic used needs to be tested before being widely adopted. Cross-visits were included in the training, not just to Indonesia. Farmers in Ermera, Dili, Bobonaro and Liquica particularly appreciated the opportunity to visit the established vegetable growing groups already established in Aileu. Farmers consistently mentioned that the most effective intervention by DAC was linking them to the supermarket buyers who pay higher than local market prices and pick up from their village aggregation point. Farmers are paid in the week following pick up and there were no reports of any problems with being paid. GDA2 groups, in Ermera, Bobonaro, and Liquica, were started too late for beneficiaries to benefit fully from the DAC interventions. Farmers were initially provided with free inputs and if farmers did not expand beyond their first 25 meter tunnel then all the inputs were provided free. Most farmers perceive the distribution of seeds to be at no cost even though in reality this might be taken into consideration by supermarkets when setting the price paid for vegetables. Business Models Four business models have been tried in Timor-Leste for the promotion of commercial vegetable production: 0% 10% 20% 30% 40% 50% 60% No School Primary (SD) Pre-secondary (SMP) Secondary/Vocational School Percent Beneficiary Education, by Gender % of male and female, for education levels with n > 5 Male Female Overall 40 Dezenvolve Setor Privadu / Private Sector Development project (DSP) was the first USAID project to support commercial vegetable growing in Timor-Leste. It was started in 2005 and included support for vegetable growers. The support included providing three greenhouses to farmers in Sarin, Liurai and Lequitura, which were groups that continued to be supported under the DAC project. The project also supported a private sector entrepreneur, Zero Star, to establish a trading business to buy from the farmers for resale to the domestic markets, including hotels and supermarkets in Dili. The project assisted Zero Star to acquire an insulated cold truck for vegetable collections from farmers. The Lequitura greenhouse has not been productive since 2011 because there was a disagreement with the farmers and Zero Star about payments for vegetable purchases, as well as a dispute over ownership of the greenhouse41. Eventually Zero Star also ceased operation. However, in 2011 the DAC project was already established and Kmanek continued to buy produce from all three groups (including outdoor vegetable production from the Lequitura group). The DAC approach has linked beneficiary farmers directly with supermarket buyers in Dili. The buyers collect produce directly from farmers and provide seeds to farmers. DAC provided training and other inputs to farmers (such as plastic, water tanks, and pipes for drip irrigation). Now that DAC has ended some of the support provided by DAC has been taken over by the supermarket buyers. An important component of DAC was the concept of a farmer organization to advocate on behalf of farmers, and the DAC legacy is AHDISTAL. This is a risky model because farmers are dependent on one buyer and if the relationship with the buyer fails the marketing chain is broken, including the farmers’ access to seeds and inputs. Josephina Farm (supported by ILO) has been a lower cost intervention (compared to DAC) where a donor has supported a private entrepreneur to make contract farming agreements with farmers; this entrepreneur acts as the commercial trader to sell to a number of different outlets including the supermarkets in Dili. ILO argue that this is a better model because an entrepreneur buys from multiple farmers and sells to multiple buyers and that this should be more sustainable because the entrepreneur supports the farmers rather than the buyer. This model is also risky if there is only one entrepreneur arranging farmer contracts and sales, but becomes less risky and potentially more sustainable if there will eventually be several independent entrepreneurs servicing farmers and the market. FarmPro in an independent social entrepreneur who works with lead farmers to motivate farmers to produce vegetables and to aggregate crops for the market. FarmPro buys from farmers and sells to supermarkets and other outlets, such hotels, in Dili. There are some similarities in this model and Josephina Farm in that FarmPro does provide some technical assistance to farmers, but the model is less formalized and there may be no formal contracts with farmers. FarmPro initially envisaged setting up input supply shops as a trading base but these proved to be non-viable due to lack of demand from farmers for inputs. Grading Farmers know that they need to grade their produce for it to reach the standard required by the supermarkets. However, in reality they are still reluctant to accept the grades agreed by the buyers and to accept that produce may be rejected. Supermarkets reported that they tend to accept under grade or rejected produce from farmers to avoid arguments and to avoid disappointing farmers. Therefore the commercial criteria for crop quality are not being followed. 41 Farmer reported that the greenhouse was handed over to Zero Star (as the owner) but the land belonged to one of the farmers. 41 Photo: Farmer watering by hand in Gleno, Ermera CONCLUSIONS EVALUATION SUMMARY AGAINST THE FIVE STANDARD EVALUATION CRITERIA Relevance The project was relevant because it provided training in improved technologies to farmers, and increased their incomes by linking them to higher value markets. Although farmers traditionally grow vegetables, DAC introduced them to new types of vegetables and enabled them to extend their growing season. Efficiency The DAC project was relatively high cost as it benefited only 547 direct beneficiaries for a total project cost of USD 8.3 million; however, there was a demonstration effect with supermarkets extending similar technologies to new groups of farmers; this was done without project support. Supermarkets did this because their buyers were unable to source all the vegetables they wanted from the DAC groups. Effectiveness DAC’s training was particularly effective as it was of sufficient duration and combined theory and practice. The farmers reported that they were confident to apply, on their own, the skills they had learned. For those farmers in GDA2 groups, which were started late, not all activities were completed fully. Watering crops by hand is very laborious and there needs to be more implementation of less labour intensive watering systems. 42 Sustainability There are issues of lack of sustainability. If supermarkets cease to support farmers the value chain is only sustainable if another buyer will fill the gap. Supermarkets do not have the ability to train their own staff in horticultural technologies. Improved seeds are not available locally so the farmers are dependent on the seed imports of the buyers. The farmer organization that was set up to support farmers is not sustainable without further project support. Farmers are not sufficiently able to reinvest in their businesses and not many of them have access to credit institutions. This lack of reinvestment also affects access to water for crop irrigation. Impact The project’s impact is limited in relation to cost. On average, farmers earned $1,604 per year42 from selling vegetables to the supermarkets, as well as to other buyers and in the local markets. This is 1.25 times the current GDP per person43; with an average of 6.8 persons per family44, total vegetable sales are equivalent to about 20% of GDP per person.45 At the beneficiary household level the project had significant impact but very little at a national level. If we assume that the average income can be applied across all beneficiaries of the project, then total income from vegetable sales would be USD 0.775 million annually46, allowing for a 12% beneficiary dropout rate but without counting any demonstration effects. CONCLUSIONS FOR SPECIFIC THEMES Profitability Although DAC did an excellent job of linking farmers directly to supermarket buyers the commercial profitability of the business does not appear to be demonstrated and supermarkets report that the relationship in not profitable; however, they continue because they have a social responsibility to farmers. Supermarkets also report that farmers are not sufficiently commercially orientated and that they do not accept the quality guidelines for produce to meet the market standards they require. This suggests that the model is not yet fully commercial. Given that domestic prices paid are relatively high (only competing with fresh imports), it will be extremely challenging for farmers to move from the local domestic market to one orientated for exports. In order for farmers’ enterprises to be sustainable they need to generate sufficient income to allow for reinvestment. This requires the scale to generate more income than is required by families for daily living expenses. Working with fewer more commercially minded farmers would increase the effectiveness of training, encourage a more commercial scale of operation, and allow for on farm reinvestment in productive assets. Working with fewer groups of more commercially minded farmers would reduce the cost of freight and cost of administration (few sellers to track and fewer payments to be made for the same amount 42 Based on the survey data, using the average of the proxy indicator for income. This indicator aggregates the annual revenues reported by beneficiaries for all the vegetable crops they report selling. 43 Calculated at USD 1,280 per person in 2014, from World Bank data 44 Based on our beneficiary survey average household size 45 Calculated at USD (1604 / 6.8)/$1280 46 Compared to the $408,000 total sales to supermarkets reported by DAC in 2014 43 of produce). Logistically, trading routes for supply and pick-up need to be rationalized, if the model is to be commercial. Training Many farmers are illiterate. Training that requires numeracy skills, such as the farming as a business training, requires more time to be allocated than the single day of training. Supermarkets do not have the resources to train their own staff in technical horticulture. Consequently ongoing extension capability is at risk once the existing staff resources become overstretched or leave to work elsewhere. For technical training to be effective it needs to be of sufficient duration for farmers to gain competency in both the practical skills and the theory underpinning for those skills. The DAC project did demonstrate the commitment to provide training of sufficient length and combined theoretical and practical training by using well qualified technical consultants from Indonesia. These Indonesian consultants mainly lived locally and were able to provide weekly or more frequent training to farmers over a period of three months or longer. This training was reported to be very effective by farmers. The training was supplemented by cross-visits to other locations (particularly for those farmers outside of Aileu district), and to Indonesia, and this complemented the local training and added value for farmers. For training to be relevant, it needs to be adapted to farmers’ education background and literacy. In some cases the training was above the comprehension level of farmers in the time allocated to the training. For some farmers the cost and the travel documentation meant they were unable to participate in the training in Indonesia. DAC manuals tended to be mainly kept by the group leaders. Some of the manuals were in Indonesian and not all farmers understand Bahasa Indonesia. Manuals need to be translated into Tetun and all farmers need to be given copies. And more emphasis needs to be made on providing visual materials that can be understood by illiterate farmers. Technologies Not all technologies are applicable to all locations. In particular technologies need to be tested before widespread adoption everywhere. The training for potato cultivation in an unsuitable lowland environment, raised beds on unsuitable soil types, and the use of plasticulture in locations easy damaged by wind are examples of unsuitable technologies in inappropriate situations. Infrastructure Cold chain infrastructure is not well developed in Aileu or in other districts and also not sufficient in Dili. To reduce wastage and increase the shelf life of produce, cold storage capacity needs to be increased. Because there are few buyers with the capability to invest in cold storage; a public private partnership may be appropriate to achieve this. Credit Whilst Kmanek and Dilimart do report assisting farmers with access to inputs such as plastic, not only seeds, they now require farmers to pay for inputs. Kmanek and Dilimart have both provided inputs to farmers and deduct payment from future crop sales. 44 Farmers were introduced too late to credit institutions and because of the initial distribution of free inputs, the farmers still have a mentality that inputs should be provided free by donors and are not well sensitized to credit application and management. Women Women were fully integrated into DAC activities and the project activities did not cause significant problems for women’s participation. However, because of traditional cultural roles, women were seldom elected to the Group Leader role in groups where women did not dominate the number of beneficiaries in the group. Where women were elected to leadership positions they were mainly for the position of secretary or treasurer rather than as the group leader, except for groups in which women were the majority of the membership. Traditionally, women do not normally do heavy work such as cultivation. However, because of the income benefits women working under the DAC project reported that they were willing to do such work in order to reap the benefits of the additional income. Youth Whilst youth are still at school, it is not realistic for school-aged youth to become fully participating group members. The fact that some youth did drop out of school early to become vegetable farmers indicates that farming can be an attractive option for youth, if they are able to observe farming as a profitable business. By helping their parents, children benefited from DAC project activities. They benefited because families had more money to spend on their children, including their children’s education. However, working on the farm after school can become a burden for children. In some groups there were no active youth members; although youth do benefit as a result of being involved in family activities. Time horizon Some of DAC’s famers successfully made the transition from subsistence to more commercially orientated farmers. This process takes time and a lot of training; this mindset change is more difficult when farmers are not numerate or literate. The DSP supported vegetable farmers in three Aileu suco since 2007 and only two of the three groups supported by DSP can be classified as sustainable in 2015. Josephina farms has been supporting a sole entrepreneur for seven years and still doesn’t believe what they have done is sustainable47. Many informants talk of the need for a 20 year time horizon for transformational projects to be sustainable in Timor-Leste. The fact that many aspects of DAC are not sustainable after almost four and a half years and also show signs of not being sustainable in suco supported by the DSP projects since 2007 (a nine year time horizon) tends to support this view. A generation is required to change from a subsistence to commercial mindset. 47 The Josephina Farm was working with 150 farmers using contract farming arrangements, but this number has now reduced to 50 farmers. 45 DSP, the predecessor to DAC, supported Zero Star to buy from the famers the DSP project supported in Sarin, Liurai and Lequitura. The fact that this trading relationship subsequently collapsed, and DAC had to step in to fill the gap, is evidence of the need for long-term support to farmers. In the GDA2 communities DAC started work too late in the project for there to be effective completion of project activities. Logistics A significant cost for fresh produce is regular fresh vegetable collections, these were weekly from farmers under DAC. The Dili market is limited in size and not so many commercial farmers are required to produce the quality of produce that can attract a premium in the market to justify the additional costs of production. The most logical way to produce fresh produce for the Dili market is to grow in three zones on one transport route as close to Dili as possible in order to reduce collection time and transport costs. The route to Maubise through Aileu covers all zones from lowland, mid and high altitude where most of the crops required by the Dili market can be grown. So the most economic production would be to select only commercially orientated farmers on this route at all the required zones. So expanding to more farmer groups and more districts is not commercially viable if long-term viability without donor support is envisaged for the domestic fresh vegetable market. FarmPro currently collect produce from Bobonaro to sell in Dili but because of the cost of transport they considered stopping such a remote collection point. They were persuaded by farmers to continue collections on the basis that they would increase the amount of crops aggregated for pick up and sale by FarmPro. Control of seed distribution is an ineffective way of controlling the quantity of vegetables produced for the market. To better control supply to match demand requires better control of the number of seedlings planted at the correct time. For this to happen, supermarket buyers need to have more coordination and control of field activities. Business Model AHDISATL is not a sustainable farmer organization which will be able to advocate successfully on behalf of farmers in the long term. AHDISTAL is heavily reliant on a few committed farmers who volunteer their time, so once these current advocates cease to work for free on behalf of farmers it is unlikely they will be replaced. AHDISTAL do not have sufficient funding to pay for current operational cost (for example the rent for the Aileu office space) and cannot replace fixed assets such as motor bikes or buy new assets as their funding base is too low. AHDISTAL does not have sufficient revenue to pay for professional staff and does not have the resources to train new groups of farmers. Because the concept of AHDISTAL changed from a regional federation of farmer organizations and was eventually consolidated into one group for Aileu District only in 2014, the formation of the association occurred too late to build the capacity of its membership and to establish a proper financial base for it to become a sustainable farmer organization doing anything more than lobby supermarket buyers on behalf of farmers. There is no capacity in the organization to be financially sustainable or to train farmers or to manage extension staff. Sarin and Liurai groups (both established under DSP) have large greenhouses, and were supported by DAC and are included on the list of DAC beneficiary groups. However, they do not pay any fees to AHDISTAL and do not appear to accept integration into the association. 46 Due to a lack of education level amongst farmers, AHDISTAL needs to be able to recruit professional managers in order to have the professional skills it needs to support association roles. The current business model of linking each group of farmers to one supermarket is risky because if the buyer fails or loses interest the value chain will cease to function unless another buyer working in another location steps in. The AHDISTAL only supports the Aileu SHA groups and does not cover the groups formed by DAC in Bobonaro, Ermera, Liquica and in Dili. Whilst farmers feel confident that the supermarkets will continue to buy their produce; the model has risks because:  The buyers report that the vegetable marketing is not a profitable activity,  Buyers do not have the capability to provide horticultural training to their own staff,  Farmers resist the quality grading needed to be able to offer premium prices,  Collection points are becoming dispersed rather than consolidated, and  There is no certainty that if one buyer should cease purchasing from farmers that one of the other buyers would step in to fill the gap. Because most farmers are not reinvesting on their farms, there is a risk that what assets DAC provided free will deteriorate and not be replaced. Farmers reported that if the supermarkets would stop buying then they would continue with vegetable production but use what local seeds would be available and sell in the local markets. This would be a reversion to previous practices. Marketing Fresh produce is one of the most challenging value chains. Challenges include perishability and the need for a cold chain to alleviate damage in transit and wastage, produce must be of high quality to attract premium prices, the technologies required for vegetable growing tend to be more demanding than for most field crops, and prices are subject to fluctuation depending on supply and demand for crops with a low storage life. There is only limited demand in Dili and very little purchasing power in the districts. The market in Dili has reached saturation for some crops and for other crops, whilst there are still opportunities to fulfill the supermarkets needs, these can be supplied by a relatively few commercial farmers. Continuing to produce more and more vegetables, in particular the ones already in plentiful supply, is not going to have a significant development impact in Timor-Leste. In fact, it might be counterproductive. To cater for surplus production for the local market, small-scale processing facilities are required, otherwise crops get sold at a discount or are wasted. To expand the market and to make a real difference to Timor-Leste farmers other crops need to be considered which have opportunities for export. Exporting crops will require a completely different mindset to that created by DAC, this is particularly the case for perishable crops with a short shelf life. As an example of the need to be more competitive, Kmanek gave the example of broccoli which they import for $3.60 per kilogram. Because of the high cost of imported product Kmanek can offer farmers relatively high farm gate prices which are now $2.50 per kilogram at farm gate (to this must be added freight costs to Dili, chilled storage, the cost of the seed originally provided free to farmers, and an 47 allowance for wastage). However to export this same crop competitively to Singapore (a high value market) or to Indonesia (which has a large domestic market) would require farmers to produce at half the current price paid to farmers and these prices would be much less than local market prices48. This can only be done by technologically advanced farmers who can produce crops on a commercial scale and who have the resources to take the good times with the bad (to make losses as well as profits). In addition these farmers will need to be able to invest in their own farms; to do this they will need to be able to access credit. Selection Criteria The fact that some farmers dropped out of groups and that some groups became inactive is an indication that more attention is required to select the right location and membership of groups. Projects need to be clear if they have a poverty alleviation focus of are commercially focused as this will affect the type of farmers and the location of the beneficiaries chosen to participate in the project. Free Inputs The distribution of free inputs to famers perpetuates a dependency mentality that is incompatible to a commercially orientated project that will need to transition to commercial practices if it is to be sustainable. RECOMMENDATIONS Based on the findings and conclusions of our final evaluation, the following recommendations are made for future interventions: Beneficiary Selection 1. Donors should be clear whether they want projects to focus on commercial farming activities or on poverty alleviation. To support commercial farming activities and value chain development, projects should select beneficiaries according to commercial criteria to maximize the efficiency of interventions and to have the best chance of sustainability. 2. Project design should take into account that commercial interventions will be more cost effective if fewer farmers are selected who can produce larger volumes of product; this will reduce the number of beneficiaries and increase the commercial scale of enterprises. Whilst this may not benefit so many farmers, it offers more chance of sustainability and establishing a critical mass from which other farmers can later benefit as a result of value chain development. 3. Avansa Agrikultura project should continue to support the DAC legacy if it is to be sustainable. This will involve technical advice to farmers, help with credit access, provision of inputs for cash or on credit, and capacity building of the ADHISTAL. 4. Avansa Agrikultura project should continue to work with the GDA2 groups which DAC started but which received less than one year of project support. 5. If groups will be discontinued, project implementers should make the reasons clear to the farmers. Some DAC farmers felt disenfranchised and did not understand why the project had ceased working with them. Communication with farmer groups should be a two way relationship with projects listening to farmers, and vice versa. 48 Key informants: Kmanek Supermarket; FarmPro. 48 Business Model Sustainability 6. For the enterprises promoted by the project, projects need to track the commercial profitability of enterprises. As a minimum, gross margin models should be developed showing returns to land and labour. Such models will show if enterprises are profitable and where improvements can be made to increase profitability and the sensitivity to changes. Better still, is a financial analysis across the value chain (of which enterprise gross margins are part), this would show whether interventions are profitable for farmers, and the other actors, taking into account capital investment costs, maintenance costs, and any debt servicing. 7. If commercial farming systems are to be adopted, projects should emphasis to farmers the need for reinvestment on-farm and to demonstrate that commercial farming models can be profitable after reinvestment. 8. Farmer groups should establish a maintenance fund of 5% of income in order to have the required capital resources for reinvestment. 9. Whilst the current business model is risky, it currently does work. New projects should consider alternative models so that if one part of the value chain fails, it will not collapse entirely. The previous USAID supported DSP intervention was also a risky model, relying on one trader who eventually ceased trading with farmers. Such an alternative model would include multiple traders and buyers, or a farmers’ organization which has the capability of acting effectively on behalf of farmers. Women and Children 10. If children will participate in project activities, projects should mount an awareness campaign to stress the importance of children completing their education and not dropping out of school. 11. Projects, NGOs and Government programs should encourage youth to take a positive attitude to agriculture; proactive programs should be included for youth to participate in agribusiness activities. 12. Projects should embed gender equity into project activities. However, awareness raising is required for activities in which women should not participate for health reasons. Training should be planned around women’s other activities, if women are the targeted beneficiaries. 13. Because women’s ability to be elected to group leader positions is still affected by cultural norms, Government, projects, as well as other community based organizations, should provide selected women with special leadership training opportunities. Monitoring Technologies 14. Projects should monitor new technological interventions to check that they are suitable from an agronomic and technical standpoint to the sites selected. Crop-related training needs to be adapted, or checked, against local condition to make sure it is relevant to the local farmers. Inputs 15. Distribution of free inputs continues a dependency mentality with farmers and is not a suitable model for commercial value chain activities. Commercially orientated projects should expect some form of contribution from farmers; this contribution can become larger over time. Such an approach needs to be a consistent across donors. 49 Farmer Organizations 16. When establishing farmer organizations, there should be a business plan prepared with a vision for a sustainable organization and for the services it will provide on behalf of its members.49 The plan needs to include a means of raising sufficient revenue from members, or from other sources, to allow for sustainability after the project ends. Sufficient time is required to build the capacity of the leaders and membership. 17. AHDISTAL only services horticultural groups in Aileu district. The organization should be expanded to represent the other groups (established by DAC and the buyers) in districts not covered by AHDISTAL (i.e. in Ermera, Liquica, Dili, and Bobonaro). This can only be accomplished by further donor support. Logistics 18. For commercial vegetable crop production, projects should select locations which are commercially sensible in terms of crop collections, transport and freight costs. With the main market being Dili, it would be better to support farmers in all the production zones (low, medium and high altitude) but to plan an economical route so as to minimize the time and cost for crop collections, rather than to have growers dispersed in many locations. 19. For new vegetable production initiatives, it is more rational to consolidate and expand existing farmer groups rather than to establish more groups. By establishing more groups, projects will only exacerbates collection and logistical problems. 20. Government should consider public-private partnerships to establish cold chain facilities in Aileu and in Dili to allow for greater shelf-life of crops and to allow for any expansion of cropping opportunities. 21. To better control the produce supply for the market, supermarkets should maintain greater control of vegetable planting by monitoring areas planted and seedling density. Management of crop production through the control of seed distribution alone is insufficient and ineffective. Training 22. Training programs should ensure that training is of sufficient duration and that it includes both theoretical and practical aspects in order to be sustainable. Training in book-keeping and farming as a business, which require numeracy skills, should target suitably educated and capable farmers. 23. Avansa Agrikultura should continue to support the technical training of supermarket staff currently working with the DAC farmers. 24. The technical training model developed by DAC for vegetable production is a good one, and should be adopted by Avansa Agrikultura. 25. Training providers should prepare training materials in an appropriate language. The training and the training materials should be tailored to the abilities of the beneficiaries. For illiterate farmers, effort should be made to produce suitable visual materials. 26. Projects should build cross-visits, which allow technologies to be demonstrated to farmers in other locations, into training activities. 27. Projects should ensure that the farmers selected for overseas training are assisted in all aspects of their travel documentation and expenses. Markets 28. USAID should conducted a review of the fresh produce market because there is currently an oversupply of some fresh produce lines in Dili. Additionally, there is a limit to the capacity of 49 Roles may include as a lobby groups for farmers, farmer services and training, advice on credit access, and marketing. 50 the Dili market to absorb more fresh produce. The capacity of districts to absorb more produce at premium prices is also limited by low economic activity and purchasing power in districts. 29. USAID should commission a study to investigate what crops have any export market potential, to what markets and at what prices. Likely markets are Singapore and Indonesia. However, for exports to have any chance of success they need to be competitive, of high quality, and to be in sufficient volume. This means selecting crops of high value for niche markets. There is currently no private sector capacity in Timor-Leste to effectively promote export production, so a public or donor / private partnership may be appropriate. 30. Projects should diversify into products less susceptible to wastage and with less demanding cold chain requirements. Such crops include nuts, some fruits and certain vegetables, legumes and lentils. There may also be opportunities for livestock and fisheries which are not currently the focus of commercial value chain support. 31. The private sector/projects need to establish small-scale processing facilities to cater for the now developing overproduction of fresh produce. 32. When giving farmers fixed price contracts, traders should make sure farmers are aware that prices must reflect market realities and that prices will change over time to reflect market conditions. Relationships with Stakeholders 33. Projects should make sure they understand the organizational structure of MAF and involve the right Directorates and staff in stakeholder groups and activities. 51 ANNEX I: DATA COLLECTION INSTRUMENTS DAC Focus Group Discussion for SHA members GROUP INFORMATION 1 Unique FGD ID Number ___ ___ ___ ___ To be filled by evaluation team 2 Interviewer / Facilitator Name(s) 3 Notes on this form taken by (name) 4 Date of FGD Day: ___ ___ Month: ___ ___ Year: 2015 5 Location: Name of Municipality 6 Location: Name of Sub￾Municipality 7 Location: Suco 8 Location: Aldeia 9 Name of SHS Group INTRODUCTIONS  Facilitator introduces her/himself and the note taker: - Working for Social Impact, conducting the evaluation on behalf of USAID -  Welcome the participants and thanks them for taking part in the discussion, reaffirm how important their views are.  Inform participants that information collected from individuals will not be attributed to them, and their names will not be used.  Describe purpose of evaluation: - To learn lessons from the Developing Agricultural Communities (DAC) project that may make the new Avansa Agrikultura project more effective -  Time for discussion about 1.5 Hours  No right or wrong answers. Everyone’s views are important. Please feel free to ask questions. 52 PARTICIPANT LIST AND CONSENT SHEET Place your signature or mark on this form so signify your consent to participating in this Focus Group Discussion. # Participant Name Consent Sex Signature or Mark Male Female 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Tota l Focus Group Questions A. GROUP INFORMATION 1. When did this group start working with DAC? 2. How many members are in this group? 3. Do they pay membership fees? 53 4. Are there elected leaders? 5. Are you members of the regional horticultural association? 6. If working with the regional horticultural association, what benefits does this provide? B. WORKING WITH DAC 7. What were the main activities you participated in under DAC? For training activities, list topics of the training… 8. What were the main benefits of these activities? 9. What were the most effective DAC activities, and why? 10. What were the least effective activities, and why? 11. Were there any activities that were needed, which were not provided? 12. Are women and youth active within this group? If yes, have they received the same benefits as the men in the group? 13. Have there been any negative consequences as a result of being part of this SHA? In particular, have there been any negative consequences for women and/or youth? 14. Do you have access to any of the training materials and guides that are the DAC legacy? If yes, which ones, and how do they benefit you? C. MARKETING PRODUCE 15. What were the main crops you sold, when working with DAC? 16. Which ones were the most profitable? And why? 17. About how much could your members make every month by selling produce through the DAC project? 18. How does this compare with now? Is it more income now, or less? 19. How about the prices for produce? Are they the same as before, more or less? 54 Can you give examples of any price changes? 20. To whom do you sell your produce now? Is it collected from the village, or do you have to transport it? 21. Do you think these selling arrangements will continue in future? If not, why not? D. FUTURE PLANS 22. What are your expectations for this SHA next year? Will it be the same as now, or are there plans to make changes? 23. What type of support does this SHA continue to need? Or can you now manage independently? Final question: Is there anything else you would like to tell us about your involvement with DAC that we didn’t ask you about? Closing Remarks  Thank everyone for participating, their opinions have been useful and will contribute to making Avansa Agrikultra a better project. QUESTIONS FOR OTHER KEY INFORMANTS Questions for Regional Group Leaders 1. How many SHA groups are there in this regional association? 55 2. How many farmer member do you support through this regional group? 3. What is your relationship with the current supermarket buyers? 4. How do you fund your activities… for example, what contributions do your member groups provide? 5. What services do you now provide to group members? 6. How could these services be improved? 7. Do you provide any training to farmer members, or member group committees? If yes, what type of training? 8. How is your management committee selected? Are committee members elected? 9. Do you employ any professional staff, such as accountants or administrators? 10. What are the main challenges in managing association groups and members? 11. What support did you receive from DAC in the past? … And can you now manage without this support? 12. What other support is available to you now? Do the supermarket buyers assist you in any way? If yes, in what way? 13. In your opinion, is your group now sustainable, and will be it be here to support farmers in another 2 years’ time? If yes, why? If not, why not? 14. Are you developing new, or alternative markets? Questions for Supermarket Informants 1. Box for: Name and contact details of informant Working with DAC: 2. What support have you received whilst working with DAC? 3. What was the most effective support, and were there any issues or problems? Groups: 4. How many groups do you work with now, which ones are they, and where? (Confirm against DAC list) 5. How often do you pick up produce from the groups? 6. About how much would be the volumes and value by crop type? (Note: get the best information they can provide. Note if they have detailed records) 56 7. What are the main challenges when working with farmers (such as crop quality, complying with contract agreements, timing of supply, repayments by farmers, etc.)? Extension services: 8. Do you provide extension services to farmers? If yes, what are these services? 9. How many field staff do you have working with farmers? 10. Are these staff originally taken from the previous DAC staff? 11. What technical support are you able to provide to your field staff? Do you have access to the technical skills you require? If not, what attempts are you making to get access to technical resources? Farm Inputs: 12. Do you provide farm inputs to farmers; if yes what are they? 13. Do you get these inputs locally, or are they imported by your company? 14. Are there any problems with quality and availability of inputs? 15. Do you have a permit for importing seeds? If not, why not? Credit: 16. Do you provide credit to farmers? If yes, what type of credit and for what purposes? And, are there problems with repayments by farmers? If yes, why? Markets: 17. What produce types are most in demand? … And which are the most profitable? 18. Are there opportunities for new crops, not now grown? If yes, which ones? 19. What are the trends in prices paid to farmers, and in farm profits? 20. What are the quality issues due to surplus supply or non-sales? 21 What are your views about the local and export markets for the produce you buy from farmers (is it in deficit, or in surplus)? Business: 22. Is this a profitable business for you? 23. Will you be able to continue in future to support the farmer groups you now work with? 24. Do you see this business expanding, staying the same, or decreasing, in future? 57 Questions for Training Providers 1. What training services did you provide for DAC in the past? 2. What were the challenges for you in training the beneficiaries? 3. What were the main challenges faced by the beneficiaries? 4. What were the most effective methods of training? 5. In your opinion, can farmers continue using the skills you provided without further training and support? 6. Do you now provide any training services to the existing supermarket buyers? If yes, what are these services? 7. In your opinion, was there any training need for beneficiaries which was not met by DAC? 8. What other training would you suggest for beneficiaries? Questions for Women’s Groups 1. Are there any women’s only activities within the SHA groups? If yes, what are they? 2. Are women elected to leadership positions within this SHA group? If yes, what are their positions? 3. Are the interests of women considered and served by the group? If so, how is this done? 4. Did women receive the training for the activities they participate in? If not, for what skills did they not receive the training? 5. Did your workload as women increase with the DAC activities? If yes, is this a problem? 6. Have the DAC activities resulted in any changes in responsibilities between you and your husband(s) either on the farm or in the household? If yes, what are the changes? 7. Do you receive a fair share of the income from the vegetable production? If not, why not? 8. Who keeps the money, and who makes decisions about the money you earn from vegetable growing? 9. Are women’s spending priorities considered equally with the men, and other HH members? 58 10. On what did you spend the extra money which you received from vegetable growing? List… Of these, which is the highest priority? 11. Are there any needs of women that should have been addressed by the DAC project but were omitted? Questions for Field staff that work with Supermarkets 1. Name, current role, qualifications and contact details. 2. Did you previously work with DAC? If yes, what was your job with DAC? 3. Do you received training in your current job? If yes, what type of training? 4. What support do you provide to farmers and farmer groups in this job? 5. What are the main challenges working with the farmers? 6. In your opinion, are the groups now sustainable on their own? If not, why not? 7. Are your current approaches with farmers the right ones? 8. What technical training do you yourself now provide to farmers? 9. Which crops are most successful, and why? 10. Are there any issues related to credit availability for farmers? 11. Do farmers accept commercial farming arrangements; for example, do they abide by contract arrangements made with buyers? 12. What are the technical problems, such as pests and diseases, etc? 13. Do farmers know about calculating profitability for their enterprises, or business? 14. Is there anything which you can suggest which should now be done to make the farmer groups more effective or sustainable? Questions for Ex DAC Field staff 1. Name, qualifications and contact details. 2. When did you work with DAC? 3. What was your job? 59 4. For how long did you work with DAC? 5. Did DAC provide you with any training? If yes, what was it? Was it relevant? Was it effective? How could it have been improved? 6. What support did DAC provide to farmers and farmer groups? 7. What technical training did you yourself provide to farmers? 8. Did women receive the same training as men? 9. What were the problems and issues when working with farmers? 10. Were the DAC approaches the right ones? 11. In your opinion, are the groups now sustainable on their own? 12. Which crops were most successful, and why? 13. Were there any issues related to the availability of credit for farmers? 14. Do farmers accept commercial farming arrangements; for example, do they abide by contract arrangements made with buyers? 15 What are the main technical problems, such as pests and diseases, etc.? 16. Do farmers know about calculating profitability for their enterprises, or business? 17. If there were to be a new project, what would you suggest anything should be done differently? 60 ANNEX II: RESULTS FOR THE QUANTITATIVE HOUSEHOLD SURVEY 4 Beneficiary Gender and Status Male Female Total Head of Household 113 10 123 Spouse of Head of Household 1 43 44 Other 13 2 15 Total 127 55 182 69.8% 30.2% 5.1 Beneficiary District, by gender Male Female Aileu 87 41 128 Bobonaro 12 5 17 Dili 5 0 5 Ermera 14 2 16 Liquica 9 7 16 127 55 182 5.3 Beneficiary by SHA and/or group # Freq Percent Original 56 31.8 GDA 1 71 40.3 GDA 2 49 27.8 # 6 missing respondents (respondents that reported "H4" and "Leo Diak") A2.1 Household Members, by Gender M F TOTAL >65 years 18 15 33 21 - 64 years 263 234 497 15 - 20 years 123 111 234 6 - 14 years 182 145 327 <5 years 74 73 147 TOTAL 660 578 1238 A2.1 Household Members by gender, descriptive statistics Mean Min Max ALL 6.802 1 14 Male 3.62 0 8 Female 3.17 0 10 Note on interpretation: On average, respondents reported 6.8 members in their household with slightly more male members reported than female. 61 A2.3 Beneficiary/Respondent Age Age Freq Percent Under 18 years 1 0.6 18 to 24 years 15 8.2 25 to 34 years 52 28.6 35 to 44 years 45 24.7 45 to 54 years 41 22.5 55 to 64 years 18 9.9 65 years or older 10 5.5 A4.1 Beneficiary Education, by gender Male Female Total No School 58 31 89 Pre-primary (TK) 0 1 1 Primary (SD) 17 12 29 Pre-secondary (SMP) 17 6 23 Secondary/Vocational School 29 5 34 Polytechnic/Diploma 0 0 0 University 2 0 2 Non-formal 4 0 4 Total 127 55 182 A5 Vegetable growing main source cash income, by gender Male Female Total No 5 3 8 Yes 122 52 174 Total 127 55 182 A5.1 Main source of income other than vegetable growing Freq Wages - public servant 1 Business owner/self employed 1 Pension 1 Money transfers from relatives 2 Other 3 Total 8 62 B1 Grow vegetables before joined DAC Freq Percent Yes 170 6.59% No 12 93.41% Total 182 B1.2 Sales outlet before DAC (can select more than one) Freq Local Market 164 Trader 70 Supermarket 4 Other 49 CROPS GROWN IN DAC Crop Frequency Income (average) Price (USD/kg, average) Bean-red (koto mean) 16 114 2.50 Bean-snap (buncis) 60 126 2.41 Bell pepper-capsicum (ai manas boot) 50 210 2.94 Bok choy 155 176 1.38 Bombay onion (lis bombay) 29 113 1.02 Broccoli 122 163 1.64 Carrots (senoura/wortel) 46 235 2.01 Cauliflower 127 146 1.32 Coriander (koentru/ketumbar) 46 188 1.86 Cucumber 113 111 1.51 Egg plant 119 98 1.71 Kailan 106 64 1.67 Kale (kangkung) 38 107 0.46 Leek (olho poro/lis tahan boot) 30 37 1.00 Lettuce 153 131 1.65 Long chili (ai manas naruk) 81 68 2.02 Mustard 111 138 2.32 Onion small (lis mean/ mutin ki'ik) 21 110 0.89 Pechay 114 121 2.15 Pineapple (ananas) 3 197 0.78 Potatoes (fehuk ropa) 1 30 3.00 Radish (rabanete/lobak) 38 52 0.56 Red cabbage (repollumean) 72 99 2.55 Rockmelon 18 114 1.07 Round cabbage (repollu) 90 113 1.38 Snow peas (ervilla) 23 76 0.77 63 Spring onion (listahan) 30 49 0.97 String beans (koto nurak) 62 84 0.88 Sweet potatoes (fehuk midar) 13 28 1.47 Tangerine (tanjerina/sabraka) 2 785 38.00 Tomatoes (small) 83 176 1.94 Tomatoes (big) 116 193 0.98 Zucchini (abobriha/pipinu naruk jepang) 79 225 0.94 Other specify 1 (Lainnya sebutkan) 16 164 5.46 Other specify 2 (Lainnya sebutkan) 4 185 0.64 B2 Total income per respondent, as reported by crop in B2 Freq Percent USD 1349 and less 107 58.79 USD 1350 - 2699 50 27.47 USD 2700 - 4049 16 8.79 USD 4050 - 5399 2 1.1 USD 5400 - 6749 2 1.1 USD 6750 - 8099 3 1.65 USD 8100 and more 2 1.1 182 100 B3 Income from vegetables in week interviewed, by gender Mean income N Median All respondents 70.55 182 50 Male respondents 71.15 127 Female respondents 69.16 55 B3 Income from vegetables in week interviewed, by group Mean income N Original 69.71 56 GDA 1 58.04 71 GDA 2 56.95 49 B4 Total weeks selling vegetables, by gender Mean weeks N Median All respondents 45.91 182 48 Male respondents 46.3 127 Female respondents 45 55 64 B4 Total weeks selling vegetables, by group Mean N Original 46.67 56 GDA 1 47.4 71 GDA 2 46.69 49 B5 Expenditures of respondent, by gender Male % Female % School fees 112 88.2 52 94.55 House improving 62 48.82 31 56.36 Purchase asset (eg TV, radio, motorbike, etc.) 25 19.69 15 27.27 Saving 14 11.02 14 25.45 Business 2 1.57 5 9.09 Traditional Ceremony (Adat) 69 54.33 31 56.36 Daily living expenses 121 95.28 53 96.36 Other (specify) 4 3.15 2 3.64 B6 All members of HH benefitted as a result of DAC project, by gender Male Female No 1 0 Yes 126 55 B7 Expanded vegetable enterprise beyond DAC? By gender Male % Female % No 71 55.91 40 72.73 Yes 56 44.09 15 27.27 65 B7 Expanded vegetable enterprise beyond DAC? By group Original Original Percent GDA 1 GDA 1 Percent GDA 2 GDA 2 Percent Yes 19 33.9 26 36.3 25 51 No 37 66.07 45 63.4 24 48.9 56 71 29 B7 Expanded vegetable enterprise beyond DAC? By income level USD 1349 and less USD 1350 - 2699 USD 2700 - 4049 USD 4050 - 5399 USD 5400 - 6749 USD 6750 - 8099 USD 8100 and more Total No 63 29 12 2 0 3 2 111 Yes 44 21 4 0 2 0 0 71 Total 107 50 16 2 2 3 2 182 Percent Yes 41.1 42 25 0 100 0 0 39 B9 Working with DAC increased income overall, by gender Male % Female % No 6 4.72 2 3.64 Yes 121 95.28 53 96.36 B9 Working with DAC increased income overall, by group Original GDA 1 GDA 2 Yes 54 68 46 168 No 2 3 3 8 Percent Yes 96.43 95.77 93.8 95.45 66 B9 by A5, increased income for those that mainly grow/don't mainly grow vegetables Vegetable growing main source of income Increased income No Yes No 0 8 Yes 8 166 * all respondents that reported vegetable growing as not their main source of income reported increase income as a result of working with DAC (n=8) *95.4% of respondents who reported vegetable growing as their main source of income reported increased income as a result of working with DAC (n=166). B9.1 DAC has increased income by how much, by gender Male Female Just sufficient 97 44 By a little more income 6 0 By a lot more income 18 9 *Approximately 15% of both male and female respondents reported "a lot more income" as a result of working with DAC. *A majority of male and female respondents (80% and 83% respectively) reported "sufficient" income increases as a result of working with DAC. 67 C1 What type of DAC of technologies have you learned, applied/used, and will continue? (frequency of Yes, how many times it was listed as most useful) Learne d Applied / used Will continu e to use Which one is the most useful? Seedling Nursery 176 174 174 30 Counting Seeds / Estimating Production / Spacing Seeds 165 163 162 1 Improved Planting Media 170 170 169 3 F1 Seeds 133 125 121 7 Raised Beds 161 160 160 13 Drip Irrigation 100 58 50 2 Other Irrigation 63 57 57 1 IPM 75 62 57 3 Organic Fertilizer 173 170 163 34 Tunnels 149 134 123 31 Composting 172 168 165 53 Value Chain Participation 117 104 100 0 Managing Group Nursery 139 132 130 2 Sales in kilos 164 163 160 2 Post Harvest Handling 154 153 152 2 Other (specify) 3 2 2 0 (last column adds to 184 not 182. Respondents selected more than one response.) C1 Rank of most top 5 most useful technologies Rank Technology 1 Composting 2 Organic Fertilizer 3 Tunnels 4 Seedling Nursery 5 Raised Beds C2 Can you afford to maintain your greenhouse, tunnels? Male Percentag e Female Percentage No 16 12.6% 11 20.0% Yes 111 87.4% 44 80.0% Total 127 55 Can you afford to maintain your greenhouse, tunnels? , by group Original GDA 1 GDA 2 Yes 47 58 44 No 9 13 5 Total 56 71 49 Percent Yes 83.93 81.69 89.8 68 Can you afford to maintain your greenhouse, tunnels? , by income level USD 1349 and less USD 1350 - 2699 USD 2700 - 4049 USD 4050 - 5399 USD 5400 - 6749 USD 6750 - 8099 USD 8100 and more Total No 21 6 0 0 0 0 0 27 Yes 86 44 16 2 2 3 2 155 Total 107 50 16 2 2 3 2 182 80% 88% 100% 100% 100% 100% 100% C3 If no, why? Frequency Percentage Too expensive 25 78.1% Hard to find the spare/materials in Timor (need to import) 7 21.9% Other 0 0.0% Total 32 69 D1 Have you received these trainings? No Yes Total Males Females Males Females No Yes Environmental Mgmt 50 25 77 30 75 107 Accounting/Recordkeeping 45 24 82 31 69 113 Farming as a Business 46 26 81 29 72 110 Nutrition 44 30 83 25 74 108 Agriculture 3 0 124 55 3 179 Mgmt 49 21 78 34 70 112 Inventory Mgmt 67 34 60 21 101 81 Produce Supply Mgmt 65 32 62 23 97 85 Grading System 57 20 70 35 77 105 Improved Market Information 16 13 111 42 29 153 Market Linkages 10 9 117 46 19 163 Value Chain Introduction 24 14 103 41 38 144 Other 126 55 1 0 181 1 70 D1.1 Level of satisfaction Very satisfied Satisfied Neither Not satisfied Very unsatisfied Males Females Total Males Females Total Males Females Total Males Females Total Males Females Total Environmental Mgmt 42 12 54 32 15 47 3 3 6 0 0 0 0 0 0 Accounting/Recordkeeping 46 11 57 29 17 46 6 3 9 1 0 1 0 0 0 Farming as a Business 51 18 69 24 11 35 5 0 5 1 0 1 0 0 0 Nutrition 57 17 74 26 5 31 0 3 3 0 0 0 0 0 0 Agriculture 93 47 140 29 8 37 2 0 2 0 0 0 0 0 0 Mgmt 41 16 57 31 17 48 5 1 6 0 0 0 1 0 1 Inventory Mgmt 28 8 36 22 8 30 9 5 14 0 0 0 0 0 0 Produce Supply Mgmt 35 10 45 23 11 34 4 2 6 0 0 0 0 0 0 Grading System 44 17 61 20 15 35 5 3 8 1 0 1 0 0 0 Improved Market Information 73 28 101 34 10 44 4 4 8 0 0 0 0 0 0 Market Linkages 82 31 113 30 11 41 5 4 9 0 0 0 0 0 0 Value Chain Introduction 57 23 80 36 14 50 8 3 11 1 1 2 0 0 0 Other 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 71 D1.2 Which one is the most relevant? Males Females Total Top 5 most relevant for males Top 3 most relevant for females Environmental Management 3 0 3 Agriculture training Agriculture training Individual Accounting/Recordkeeping 1 1 2 Farming as a Business Market Linkages / Partnerships Farming as a Business 15 3 18 Market Linkages / Partnerships Farming as a Business Nutrition training 6 0 6 Management training Agriculture training 77 42 119 Nutrition training Management training 7 1 8 Inventory Mgmt. 1 0 1 Produce Supply Mgmt 0 0 0 Grading System 0 1 1 Improved Market Information 3 0 3 Market Linkages / Partnerships 13 6 19 Value Chain Introduction 1 1 2 Other 0 0 0 *top 5 most relevant bolded above for respondents overall 72 D2 Was any training planned by DAC but not actually delivered? By gender Males Freq M Percent Females Freq F Percent Total No 112 88.19 47 85.45 159 Yes 15 11.01 8 14.55 23 D3 Have you used the farm training manuals developed by DAC? By gender Males Freq M Percent Females Freq F Percent Total No 68 53.54 32 58.18 100 Yes 59 46.46 23 41.82 82 *insert by group D5 Did anything stop/limit you to participate in training? By gender Males Females Total No 95 44 139 Yes 32 11 43 127 55 Did anything stop/limit you to participate in training? By education No Yes No school 67 22 Pre-primary 1 0 Primary 24 5 Pre-secondary 17 6 Secondary 26 8 University 0 2 Non-formal 4 0 73 E1 Did you pay membership fees? Total Percentage No 167 91.76 Yes 15 8.24 182 E1.1 If yes, how much did you pay the membership fees? Fee Freq 0.25 1 1 9 2 2 4 1 5 1 20 1 *mean fee: $2.82 E.2 Is the group still active now? Total Percentage No 4 2.2 Yes 178 97.8 E3 Main benefits of DAC participation Total Frequency Male Female Access to Training 164 114 50 Access to new market 151 103 48 Access to credit 11 6 5 Increased income and saving 83 57 26 Other 1 1 0 E4 Sold vegetables as part of group selling Freq Percent Yes 174 95.6 No 8 4.4 E6 Happy with arrangement for being paid by the group Male Female Yes 115 47 *162 got paid through the group (response to E5) 74 F2 If you received extension services (n = 147), who provided it? Frequency Percentage Supermarket Buyer 69 46.9% MAF Extension staff 29 19.7% NGO 11 7.5% Input suppliers 0 0.0% Crop traders 0 0.0% DAC project 143 97.3% Others 2 1.4% F2 If you received extension services (n = 147), how satisfied were you with the services? Level of satisfaction Extension services Very Satisfied Somewhat Satisfied Neutral Somewhat dissatisfied Very unsatisfied Supermarket Buyer 51 11 2 1 4 MAF Extension staff 18 4 4 3 0 NGO 10 1 0 0 0 Input suppliers 0 0 0 0 0 Crop traders 0 0 0 0 0 DAC project 119 15 0 2 7 Others 1 0 0 1 0 F3 Situation of extension services now that DAC is completed (n = 147) Freq Percent About the same 117 79.59 Better now 14 9.52 Worse now 16 10.88 G2 Types of inputs used (n=182) Frequency % Seeds 181 99.5% Chemical fertilizer 104 57.1% Agricultural chemicals 122 67.0% Small tools 149 81.9% Plastic 158 86.8% Other 19 10.4% 75 Types of inputs from… Kmanek/Dilimart/W4 DAC staff directly Other traders Input supply shop Horticulture group Local market Seeds 118 57 0 1 0 5 Chemical fertilizer 17 23 2 11 0 51 Agricultural chemicals 19 37 0 18 1 47 Small tools 26 53 1 18 2 49 Plastic 32 120 1 3 1 1 Other 0 9 0 0 0 10 Types of inputs by payment type Free Cash Credit Seeds 150 11 20 Chemical fertilizer 26 76 2 Agricultural chemicals 44 77 1 Small tools 69 76 4 Plastic 146 8 4 Other 9 10 0 G3 Inputs wanted that were not available Freq Percent No 112 61.54 Yes 70 38.46 76 H1 Sold produce under DAC, frequency, by gender Male Female No Yes No Yes Total Supermarkets 12 115 5 50 165 Local market 24 103 16 39 142 Other farm gate trader 106 21 45 10 31 Others 119 8 49 6 14 H1 Main outlet for selling produce under DAC, frequency, by gender # responded 'yes' Male Female Total reported yes Supermarkets 113 50 163 Local market 10 3 13 Other farm gate trader 3 0 3 Others 1 2 3 127 55 182 H1 Main outlet for selling produce under DAC, frequency, by group # responded 'yes' Original GDA 1 GDA 2 Supermarkets 55 68 35 Local market 1 3 9 Other farm gate trader 0 0 2 Others 0 0 3 Total 56 71 49 H2 How often did you sell your produce? Frequency Percent More than once a week 4 2.2 Weekly 172 94.5 Every 2 weeks 6 3.3 H3 What are your plans for selling in the future? Frequency Percent Continue as usual 124 68.13 Sell more to existing buyers 52 28.13 Look for new buyers 6 3.3 Others 0 0 77 H4 Are you still now linked with the same traders who bought your produce when the DAC project assisted you? Frequency Percent No 17 9.34 Yes 165 90.66 H4 By location Aileu Bobonaro Dili Ermera Liquica No 5 11 0 0 1 Yes 123 6 5 16 15 H4 By group Original GDA 1 GDA 2 Yes 51 71 37 No 5 0 12 Total 56 71 49 Percent yes 91.07 100 75.51 Percent no 8.93 0 24.5 H5 How about the price now compared with DAC? Frequency Percent Still the same as with DAC 94 51.65 Increased 18 9.89 Decreased 68 37.36 Others 2 1.1 H6 Were you happy with the trading arrangements that DAC established for you? Frequency Percent No 18 9.89 Yes 164 90.11 H6 Were you happy with the trading arrangements that DAC established for you? Original GDA 1 GDA 2 Yes 50 69 39 No 6 2 10 Total 56 71 49 Percent Yes 89.3 97.2 79.6 Percent No 10.7 2.8 20.41 78 I1 Do you have access to credit? By gender Male Female No 116 47 Yes 11 8 127 55 I1 Do you have access to credit? By group I2 If yes, who provides the credit? Frequency No Yes Supermarket 7 12 NGO 17 2 Farm input supplier 19 0 Crop trader 18 1 Bank 11 8 Other 19 0 23 I2 Purpose by source of credit Daily living expenses incl health and school fees Farm crop or livestock inputs (e.g. fertiliser, tools, chemicals) Farm machinery / equipment (e.g. hand tractor) House repairs Other large non-farm items (e.g. motor bike, boat engine, etc…) Others Supermarket 1 6 4 0 0 1 NGO 1 1 0 0 0 Farm input supplier 0 0 1 0 0 0 Crop trader 0 0 1 0 0 0 Bank 5 1 1 1 0 0 79 J1 Who keeps the money from the vegetable production? Frequency Percent Interviewee beneficiary 105 57.69 Husband 4 2.2 Wife 71 39.01 Other 2 1.1 J1 Who keeps the money from the vegetable production? By respondent gender Male M Percent Female F percent Interviewee beneficiary 55 43 50 90.9 Husband 2 1.5 2 3.6 Wife 69 54 2 3.6 Other 1 0.79 1 1.8 99.29 J1 Who keeps the money from the vegetable production? By Head of Household Male HH Male HH, percent Female HH Female HH, percent Interviewee beneficiary 87 53.3 18 94.74 Husband 4 2.4 0 0 Wife 70 42.94 1 5.26 Other 2 1.2 0 0 J2 Who makes the decision over how to use the money? Frequency Percent Interviewee beneficiary 58 31.87 Husband 1 0.55 Wife 2 1.1 Together 119 65.38 Other 2 1.1 J2 Who makes the decision over how to use the money? By beneficiary gender Male Male percent Female Female percent Interviewee beneficiary 36 28.35 22 40 Husband 0 0 1 1.82 Wife 2 1.57 0 0 Together 88 69.29 31 56.36 Other 1 0.79 1 1.82 J2 Who makes the decision over how to use the money? By HH Male HH Male HH, percent Female HH Female HH, percent Interviewee beneficiary 46 28.2 12 63.2 Husband 1 0.6 0 0 Wife 2 1.2 0 0 Together 112 68.7 7 36.84 Other 2 1.2 0 0 80 K2 Did working with DAC create more work for women that was difficult to manage? Male Female No 100 41 Yes 27 14 K4 How would you compare the farm situation now (after DAC) as compared to when you were working with DAC? Frequency Percent About the same 119 65.38 Better now 37 20.33 Worse now 26 14.29 Don't know K6 Did you employ any labour as the results of DAC activities? Frequency Percent No 163 89.56 Yes 19 10.44 K6 Did you employ any labour as the results of DAC activities? By income level USD 1349 and less USD 1350 - 2699 USD 2700 - 4049 USD 4050 - 5399 USD 5400 - 6749 USD 6750 - 8099 USD 8100 and more Total No 95 45 14 2 2 3 2 163 Yes 12 5 2 0 0 0 0 19 K6 Did you employ any labour as the results of DAC activities? By group Original GDA 1 GDA 2 Yes 9 4 6 Percent yes 16.07 5.63 12.24 K6.1 If yes, how many (average): Total adults Adults Total youth Youth Male Female Male Female Full-time 1.42 0.84 0.26 1.79 0.36 0.11 Part-time 0.37 0.26 0.11 4.74 0.05 0 81 L1 Did you experience any serious pest or disease problem for vegetable production that you could not manage as part of normal farming operations? Frequency Percent No 0 0.00% Yes 182 100.00% L1.1 If yes, what was the problem? Frequency No Yes Locusts 21 161 Other insect pests 2 180 Leaf diseases 12 170 Soil diseases 25 157 Birds 171 11 Other 177 5 When was the last time? Locusts Month Frequency Year Frequency 1 18 2015 84 2 15 2014 76 3 5 2013 1 4 10 5 12 6 13 7 6 8 30 9 28 10 5 11 10 12 9 When was the last time? Other insects Month Frequency Year Frequency 1 23 2015 100 2 13 2014 80 3 7 4 7 5 19 6 15 7 7 8 28 9 37 10 5 11 10 12 9 82 When was the last time? Leaf diseases Month Frequency Year Frequency 1 6 2015 106 2 9 2014 64 3 7 4 7 5 15 6 13 7 12 8 32 9 55 10 5 11 6 12 3 When was the last time? Soil diseases Month Frequency Year Frequency 1 12 2015 94 2 9 2014 63 3 11 4 6 5 17 6 7 7 10 8 24 9 44 10 4 11 8 12 5 When was the last time? Bird Month Frequency Year Frequency 2 1 2015 9 8 2 2014 2 9 7 12 1 When was the last time? Other Month Frequency Year Frequency 6 1 2015 3 8 1 2014 2 9 1 11 1 12 1 83 L1.4 In the last year about how many times would a situation like this have occurred for you? Locusts Frequency Percent 0 0 0.0% 1 25 15.7% 2 22 13.8% 3 4 2.5% 4 9 5.7% 6 3 1.9% 7 0 0.0% 12 32 20.1% 24 0 0.0% 36 2 1.3% 365 62 39.0% Total 159 Other insects Frequency Percent 0 1 0.6% 1 34 19.1% 2 20 11.2% 3 5 2.8% 4 13 7.3% 6 3 1.7% 7 1 0.6% 12 32 18.0% 24 1 0.6% 36 2 1.1% 365 66 37.1% Total 178 Leaf diseases Frequency Percent 0 1 0.6% 1 29 17.3% 2 22 13.1% 3 6 3.6% 4 4 2.4% 6 2 1.2% 7 1 0.6% 12 31 18.5% 24 2 1.2% 36 2 1.2% 365 68 40.5% Total 168 Soil diseases Frequency Percent 0 1 0.6% 1 24 15.5% 2 19 12.3% 84 3 3 1.9% 4 8 5.2% 6 4 2.6% 7 1 0.6% 12 30 19.4% 24 1 0.6% 36 2 1.3% 365 62 40.0% Total 155 Birds Frequency Percent 1 5 45.45 2 1 9.09 12 2 18.18 365 3 27.27 Other Frequency Percent 1 1 20 2 2 40 12 1 20 265 1 20 85 ANNEX III: EVALUATION SCOPE OF WORK Parts of Avansa M&E SoW that relate to DAC Evaluation DAC Final Evaluation: To extract lessons learned and to generate recommendations to be incorporated into AVANSA. C.4.8.2 Evaluation Questions DAC final performance evaluation questions: 1) Do target beneficiaries report an increase in income as a result of project activities? If so, by how much? How was income used? Has income resulted in increased benefit all members of the household? 2) Were there any unintended outcomes (positive and/or negative) that resulted from project activities (i.e., child labour, increased school attendance, increased household burden for women)? 3) Are linkages between farmers and buyers likely to be sustainable? 4) Did technical assistance to farmers reach all intended beneficiaries (i.e., women and youth)? 5) What variations were there in implementation approaches or strategies and what was most effective/efficient in achieving results? C.3.2 Developing Agricultural Communities Project (DAC) Development Alternatives, Inc. (DAI) was awarded a three-year contract under the USAID Rural and Agricultural Incomes with a Sustainable Environment PLUS (RAISE-PLUS) Indefinite Quantity Contract for the DAC project. This project focuses on the completion of a development model that improves the economic and social livelihood of members of rural households in poor communities in a way that distributes the benefits fairly among people of both genders. This outcome is being achieved through two approaches: (i) the introduction of improved agricultural production technologies to poor rural households and (ii) the provision of focused technical and management training of community members in agribusiness and production technologies. It is expected that these interventions will strengthen the supply chain that links rural households agricultural producers to markets in Dili. The DAC project was originally planned for three years, August 2010-August 2013, and was extended to accommodate resources made available by Conoco-Phillips and to accommodate a trilateral activity between the US Government, Government of Timor-Leste, and Government of China. Main activities of the project include: technologies and management modeling and adoption, farmers’ association formation, training in book keeping, and farmer field school. The project originally focused in three communities in the district of Aileu. During the first 86 extension another six communities were added in the district of Aileu and during the second extension 10 more communities from the districts of Ermera, Bobonaro and Liquica were added. Major accomplishments to date are: the establishment of Special Horticulture Areas (SHA) in all locations intended under the original contract and the contract extensions; evidence of improvement in farmer’s income; evidence of adoption of new technologies and management practices as originally planned; the complete management and operational handover of two green houses to the communities. A midterm evaluation was conducted in June 2012 with the intention to determine the project relevance, effectiveness, efficiency, potential impact and sustainability of results achieved. The evaluation identified gaps, challenges and provides solutions and recommendations. Many of these recommendations are being addressed during the remaining project period. Deliverables for Evaluation Performance Evaluation Design/ Methodology for DAC and AVANSA Evaluations Written evaluation design/ methodology to be submitted in an electronic format to the COR for each of three evaluations planned under the contract. The evaluation design/methodology should include proposed sub-questions that can lead to answering key evaluation questions, methods, main features of data collection instruments, and data analysis plans, and dissemination plan. The design/methodology will be shared and discussed with country-level stakeholders as well as with the implementing partners for comments before being finalized. Final Performance Evaluation Report for DAC Project A Final Report will be provided to the Mission within 15 days following receipt of comments from the Mission. The report shall include an executive summary and shall not exceed 60 pages (excluding appendices). The executive summary shall be 3-5 pages in length and summarize the purpose, background of the DAC project, main evaluation questions, methods, findings, conclusions, and recommendations and lessons learned. The report will meet the criteria outlined in USAID’s Evaluation Policy (http://www.usaid.gov/evaluation) and will follow the Agency’s evaluation report template. The Final report should be addressed to the M&E mechanism COR in hard and soft copies. An encrypted flash drive including all instruments and data formats suitable for analysis should also be submitted. The report must include signed disclosures of conflict of interest from each member who worked on any part of the evaluation report process. When applicable, the report must include statements regarding any significant unresolved differences of opinion on the part of the funders, implementers and or members of the evaluation team. 87 ANNEX IV: DISCLOSURE OF ANY CONFLICTS OF INTEREST 88 89 90 ANNEX V: SOURCES OF INFORMATION / CONTACTS Evaluation of Developing Agricultural Communities Project (DAC): People Consulted Name Position Organization Contact Details Clarence Lim H.M. Director (owner) of Kmanek Trading Kmanek Trading Supermarket Palm Business and Trade Centre Surikmas, Fatumeta – Bairo Pite, Hudi Laran HP: +670 7728 4388 E: ClarenceLim@kmanekgroup.com Adi Setiadi W-Four Staff (use to work with DAC) W-Four Supermarket HP: +670 7323 7786 E: adisetiyadi29@gmail.com Manuel Pereira Horticulture Manager Dilimart Supermarket HP: +670 77872282 Peter Dougan Director (owner) FarmPro 77609013 Jenny Ikelberg Value Chain Development Expert ILO - Josephina Farm Octavio de Almeida Inspector General of Policy, Planning and Monitoring Ministry of Agriculture and Fisheries 77241639 Amaro Ximenes Director of Agriculture, Horticulture and Extension Ministry of Agriculture and Fisheries Galeni Galios Director of Aileu Municipality Agriculture Department MAF Aileu Municipality Aileu HP: +670 7731 2329 Bruno Benevente Ex - DAC Operation Manager DAI HP: +670 77336554 Cesaltino Lopes Ex - DAC Team Leader (DAI) DAI HP: +670 7734 9578 Filomena Almeida Salsinha Ex - DAC Field Staff (DAI) DAI HP: +670 7743 9836 Antonio Ex - DAC Field Staff (DAI) DAI HP: +670 7737 4693 Mercelino and Cipriano Ex - DAC Area Coordinator Ex – DAC Admin Assistant and Area Coordinator DAI HP: +670 7729 6416 HP: +670 7759 9496 Silveiro Amaral B. Felix President AHDISTAL, Aileu HP: 7829 5528 Anina Bareto Project Officer Timor Aid HP: +670 7736 4038 91 Natalino Galoso and Sabino Mendes Trainer HIAM Health Agostinho Sena de Jesus Staff of Empreza Diak Empreza Diak Jose Lobato Country Manager ConocoPhillips 77327606 David de Araujo External Relations Officer Conoco Phillips Lourenco Soares Chief Data Management General Directorate of Statistics, Ministry of Finance +670 7857 2261 Geraldo Aceituno Pulta National Accounts Adviser General Directorate of Statistics, Ministry of Finance Erwan Thomas Director of the input supplier Jupiter Motor – Input Supplier HP: +670 790 3731 E: Jupiter.dili@yahoo.co.id Silvanus Siri Owner of MANFAT shop MANFAT – Inputs supplier HP: +670 7726 2355 12 Focus Group Discussions Isac Maurema Group Leader Liurai Aileu HP: +670 7694 5818 Marcelino Mozinho (individual interview) Group Leader Sarin Sarin, Aileu Refused to provide contact number Dulce Matias Group Leader Foin Kaman (women’s group) Aileu HP: +670 7734 4912 Florindo Da Costa Group Leader Hakiak Moris Aileu HP: +670 7554 5122 Vicente Cruma Group Leader Tasonih Aileu HP: N/A Zeca da Silva and Gaspar Mesquita Group Leader Fatubosa 1&2 Aileu HP: +670 7704 6878 HP: +670 7579 8253 Rosminina Da Conceicao Group Leader Fo Liman ba Malu Aileu HP: +6707734 9169 Alfredo Da Costa Group Leader Remixio (inactive) Remexio HP: +670 7629 9831 Antonio Mesquita Group Leader Saralala Aileu HP: +670 7819 1571 Batista Ulan Group Leader Gleno 1 Gleno 1&2 Gleno HP: +670 7512 2554 92 Manuel Lesu Mali Soares Group Leader Loes (H4) Loes HP: +670 7541 8303 or 7743 2921 Abel Pires Group Leader Atabae (Madameta) Atabae HP: +670 7551 0939 Victor Soares Group Leader Miggir (inactive) Miggir - Refused to provide contact number 93 ANNEX VI: MATRIX OF KEY FINDINGS AND RECOMMENDATIONS Evaluation Questions Preliminary/Interim Findings Preliminary Analysis and Conclusions Preliminary Recommendations 1) Do target beneficiaries report an increase in income as a result of project activities? If so, by how much? How was income used? Has income resulted in increased benefit to all members of the household?  Beneficiaries report an increase in income on average 40 – 75 USD per week ($57/week/person).  Farmers reported that this income was insufficient to meet all their needs. FGD Results Fatubosa: $ 10-15/week/person Hakiak Moris: $ 25-100/week/person Madameta: $ 80/week/person Liurai: $ 50/week/person Remexio: $ 50/week/person Sarlala: $ 25/week/person Gleno: $ 25-75/week/person Fo Liman baMalu (women): $ 30- 40/week/ person Loes: $ 150/week/person Tasonih: $ 10-180/week/person Sarin:  Therefore, the average: $ 40 – 75 per week/person, or ($57/week/person).  By FGD list the averages that were reported by farmers (incl. range).  From DAC Excel sheets graph the quarterly income data by group and average.  By calculation graph or tabulate additional income per beneficiary.  For enterprises to be sustainable, the size of each beneficiary farmer’s enterprise needs to be sufficient to allow for reinvestment.  Financial models need to be developed to demonstrate financial viability of enterprises.  Projects which support commercial enterprises should target farmers with entrepreneurial attitude and of viable scale. 94 Evaluation Questions Preliminary/Interim Findings Preliminary Analysis and Conclusions Preliminary Recommendations  The amount of income varied significantly between beneficiaries depending on their level of commitment.  Additional information will come from the HH survey of 180 beneficiaries.  Projects should be clear whether they have a poverty alleviation focus or a commercial focus  All of the HH members benefit from this additional income. In some cases children benefit from their own plot of land and income.  Example of FGD e.g. Seloi, Liurai, Gleno, for children with own raised vegetable growing beds.  If children will benefit, this should not be at the expense of school attendance.  Some of the original beneficiaries dropped out of the groups as they found the commitment too much.  Check if this finding is correct. May be applicable for one or two groups. Confirm with group members who were previously active.  More care is needed to select committed beneficiaries.  May be better to have fewer more commercially orientated farmers rather than to work with a large groups of farmers.  Income was used for daily living expenses, school fees, traditional ceremony, house improvements, and to a lesser extent reinvestment on farm.  DAC did improved farmers’ living standards; however, reinvestment on farm was a lower priority.  Scale of enterprise for each farmer needs to be sufficient to allow a surplus for reinvestment.  Reinvestment would be more likely if more commercially minded farmers were selected.  New projects should encourage farmers own reinvestments on￾farm.  Farmers need coaching on the requirements of commercial farming and reinvestment. 95 Evaluation Questions Preliminary/Interim Findings Preliminary Analysis and Conclusions Preliminary Recommendations 2) Were there any unintended outcomes (positive and/or negative) that resulted from project activities (i.e., child labour, increased school attendance, increased household burden for women)?  Fixed price structure gives farmers price certainly for one agreement period (6 months). Farmers were positive about the fixed price structure.  Fixed price may be necessary to encourage framers but results in lag for price adjustments to market conditions.  Fixed price gives farmers confidence for the immediate growing period.  Farmers get discouraged when the price is subsequently reduced.  Awareness needs to be raised on reason for price setting and the fact that prices can change due to market conditions.  Market surplus now reported by farmers for some products (e.g. tomatoes, lettuce, bok choy, Chinese cabbage).  Currently sufficient vegetable production groups. However, for some vegetable lines produce can still be in insufficient supply (e.g. cauliflower, broccoli)  Consolidate existing groups rather than continue to expand to new groups for similar produce lines.  Project cost per famers is not good value for money USD 8.5 Million / 550 = $15,000 per direct beneficiary or $4,000 per year.  20 year time horizon needed to build farmer and institutional capability.  Avansa equivalent cost per direct beneficiary is $3,000.  New project design should include an analysis of cost effectiveness of predicted benefits 96 Evaluation Questions Preliminary/Interim Findings Preliminary Analysis and Conclusions Preliminary Recommendations  Rapid expansion of number of groups by supermarkets over and above DAC groups has increased the cost of transport without increasing sales volumes. This was because farmer groups proved unreliable to supply the desired volumes of produce.  Dilimart now collects from 13 groups, whereas under DAC only established 6 groups.  Need access to Dili markets and so growing on same access route close to market would minimizes transport cost.  Three zones are required for production high altitude, mid zone, and low altitude. These zone can cater for all crop types that can be grown in TL.  It is more cost effective to concentrate production few areas close to the Dili market rather than expand into remote locations (such as Bobonaro).  Avansa Agrikultura should continue to support and consolidate existing vegetable groups rather than form new groups for similar vegetable production enterprises.  Some farmers do not plant the seed, or plant not according to the timeframe, and more control / planning of actual planting area is required (W4/Dilimart).  Control of seed distribution is an ineffective way of controlling production volumes.  Market demand can be matched better by working more closely with groups to control planted area.  More coordinated approach to control of planting area to meet market demand required. This requires more field supervision of groups with respect to planting areas and dates. 97 Evaluation Questions Preliminary/Interim Findings Preliminary Analysis and Conclusions Preliminary Recommendations  MAF Aileu copied the DAC approach and formed their own groups to implement DAC activities using the same approach. Focus was on cauliflower and broccoli due to national priority in MAF strategic plan.  This is evidence of the demonstration effect of DAC. MAF formed 3 groups with 76 HH in all. Plan for MAF to establish more groups.  Danger in uncoordinated and unplanned expansion of groups and overproduction. Groups not under AHDISTAL.  Sales are to existing supermarkets which buy from DAC farmers.  Group expansion should only be encouraged if there is market demand for fresh produce, facilities for transportation and links to markets.  We did encounter children working in vegetable gardens. However, they received their own income.  There were few observed negative consequences of child labour. However, there were some instances of children stopping school because they wanted to earn money on the farm.  Awareness training of importance of children completing education.  In few cases (2 in Lois) children (13 - 14 years) had dropped out of junior high school to work on the farm.  In some cases children did say they spent too much spare time working on the farm.  There are some instances of children being burdened by on farm work.  New project should raise awareness not to overburden children with farm activities and to encourage children to continue learning. 98 Evaluation Questions Preliminary/Interim Findings Preliminary Analysis and Conclusions Preliminary Recommendations 3) Are linkages between farmers and buyers likely to be sustainable?  DAC closed in Feb 2015. Currently supermarkets continue to buy from farmers and to support some field staff. In the short-term supermarket buyers continue to support farmers.  Without supermarket support the groups will not continue.  Seed is imported and unavailable locally.  Farmers feel confident that existing marketing arrangements will continue.  There is risk as extension staff no longer receive technical training.  If supermarkets should stop supporting farmers, some farmers will continue but use local seeds and sell to local markets.  Avansa Agrikultura should continue to support existing groups before expanding to new groups for similar crops.  Avansa Ag will need to continue to support technical training for field staff.  MAF gave AHDISTAL old MAF office and site but a land dispute prevented the use of this site.  Kmanek now pays $300 per month for the AHDISTAL offices in Aileu.  This demonstrates existing support for the AHDISTAL, but in itself is insufficient to ensure sustainability of AHDISTAL.  Association lacks resources to be self-sufficient.  Association requires mechanism for funding own office facilities.  Farmer contribution needs to be adequate for self-sufficient management.  Vegetable production requires cold chain development that is not well established in Dili or in Aileu.  There is seasonal over-production during the dry season, when it is easier to grow vegetables.  In rainy season vegetables need to be grown under plasticulture or in greenhouses and humidity makes pest and disease control more difficult.  Fresh produce is one of the most challenging value chains.  Establish additional cold chain facilities in Dili and in Aileu.  Project should consider less demanding high-value value chains that have longer shelf life such as fruits and nuts and less perishable vegetables, as well as livestock and fisheries.  Processing facilities are required to cater for overproduction. 99 Evaluation Questions Preliminary/Interim Findings Preliminary Analysis and Conclusions Preliminary Recommendations  Supermarkets report they don’t have the resources to train their own extension staff. Currently relying on existing knowledge from ex DAC staff and on new staff already trained.  Supermarkets do not have the resources to provide technical training to their own field staff  AHDISTAL needs to have the capability to train farmers and extension staff. However, critical mass to make this economic may not exist.  Supermarkets are no longer able to support farmers in the same way DAC did with free inputs.  There is risk of asset deterioration due to lack of maintenance by farmers.  Economic analysis needs to be completed to provide evidence of economic viability for individual farmers, groups, AHDISTAL, and buyers.  Farmers are reliant on selling higher value product to one supermarket buyer; surplus or low grade product is sold in the local market.  DAC is a risky business model because the market link depends on the relationship with one supermarket buyer.  Alternative support mechanism needs to be put in place to support risky business model. AHDISTAL was intended to be this mechanism.  Dilimart, W4 and Kmanek say there is a social aspect to buying from farmers, so the relationship is not entirely a commercial one.  Farmers are supposed to do grading to eliminate produce that doesn’t match quality requirements but buyers do not always enforce this policy.  The business model may not be viable economically.  For competition in the commercial market DAC is not a sustainable model.  If the project is to be commercially sustainable commercial practices must be adopted. This will require a new business model. 100 Evaluation Questions Preliminary/Interim Findings Preliminary Analysis and Conclusions Preliminary Recommendations  AHDISTAL is not sustainable:  Only 2/3 farmers pay the $1/month association membership fee.  Staff volunteers are not paid, so a lot depends on the existing chairman.  No professional staff and no income to pay professional staff.  Formed very late (2015) after working with regional federation concept.  Only have farmer trainers and request Government support for staff and training center infrastructure in Seloi.  Only have 2 only motorbikes from DSP time; just managing to maintain these.  No transport other than motorbikes.  Rely on Kmanek to rent office space. Containers used for office space donated by DAC.  AHDISTAL is not sustainable and lacks capacity to manage farmers’ issues in the long term.  The AHDISTAL was formed too late and the capacity of the organization should have been supported from the start of the project.  Recommendation for establishing AHDISTAL was made at the mid￾term evaluation.  Need to hire professional manager to manage the association business.  Need to build the capacity of the farmer organization to ensure sustainability.  Need to ensure that there is sufficient income source for the organization to manage effectively on behalf of farmers.  Avansa Agrikultura should consider supporting AHDISTAL and employing professional staff.  A business plan is needed that demonstrates any project model has a viable and profitable end point. 101 Evaluation Questions Preliminary/Interim Findings Preliminary Analysis and Conclusions Preliminary Recommendations  Those groups which started late under DAC did not complete their irrigation infrastructure (tanks and pipes)  There is a need to complete what DAC started but did not finish.  When project is winding down it is better not to support new groups if the project time￾horizon is insufficient.  Avansa Agrikultura should continue to support existing farmer groups with TA and infrastructure for the groups that DAC started late.  Evidence for some groups of lack of maintenance for infrastructure. E.g. Liquitura greenhouse damaged but not repaired; Sarlala water tanks; Atabai unused tunnels.  Dry season production does not require tunnels for maintenance might be deferred.  Groups and individual farmers should establish a maintenance fund as at some stage maintenance will be required (say about 5% of income)  Some groups which were started ceased operation. Talitu group never really started.  Remexio production too low to make it worthwhile to collect produce. From 12 farmers only 4 continued but now sell to local markets and to Dili fresh produce markets.  Miggir group after training completed there was no follow up action from DAC. However, anecdotal information suggests dem. plot was unsuccessful and road challenging/remote.  Talitu group never started because field assessment negative. Lack of water, road renovation, etc.  More informed selection of groups required and improved communication with groups. 102 Evaluation Questions Preliminary/Interim Findings Preliminary Analysis and Conclusions Preliminary Recommendations 4) Did technical assistance to farmers reach all intended beneficiaries (i.e., women and youth)?  For the original groups, farmers report that all the intended beneficiaries received assistance. However, for GDA expansion groups, farmers received training but some of the materials, including water supply, market linkages, registration with MoJ and association formation was not completed.  Initially the project envisaged the formation of a farmer cooperative but this was later changed to an association i.e. AHDISTAL. This association now only covers Aileu, not the other 4 districts (including Dili).  Expansion groups were started too late for group consolidation.  Concept for AHDISTAL changed. At one point District Federation of five groups was envisaged but this changed to one Association in Aileu late in the project Aug 2014; there was insufficient capacity development of the association.  Sarin and Liurai greenhouse groups were integrated into AHDISTAL; however, they do not pay association fees and don’t appear to accept integration.  Important to have robust criteria for selection of beneficiaries.  Project should not start new groups without sufficient time for consolidation.  Concept for farmer association should be clear at the start of the project.  Adequate time is required for capacity development of the association.  At group formation, farmers should be given a clear concept and benefits of association development.  Commercially motivated farmers should be included as a beneficiary selection criteria.  Women represent 32% of all group members.  DAC effectively included women as beneficiaries.  Projects should include leadership capacity development for women.  Some groups have elected women leaders or officials e.g. Gleno 2 (Ermera), Foin Kman, Fo Liman (in Seloi)  Women are well integrated into groups and can be elected to leadership positions. However, most women’s positions are mainly as elected treasurer or other non￾leadership role due to TL cultural norms.  Gender equity should be promoted for farmer representation. 103 Evaluation Questions Preliminary/Interim Findings Preliminary Analysis and Conclusions Preliminary Recommendations  Youth, including those still at school, were included in some but not all groups. e.g. Tasonih, Foin Kman, Gleno 1  Youth are include more in family activities than as formal group members.  In some groups there is no youth participation  Youth do benefit as family participants in DAC activities.  Youth should be encouraged to have a more positive attitude to agriculture.  Proactive programs should be included for youth.  Women told us that there were no problems for women being involved in DAC activities.  In some communities women do not normally do heavy work such as cultivation, but if there will be financial benefits they are willing to do such work.  Projects should raise awareness of things that women should not do for health reasons.  Women participated equally in DAC training, including training in Indonesia. E.g. Foin Kman, Gleno 1,  Technical training was equally available to men and women  Training schedule should be adapted to women’s time schedules.  Gender equity training should be included in project activities. 5) What variations were there in implementation approaches or strategies and what was most effective/efficient in achieving results?  Most training was very effective; one day a week over 3 months.  Farmers spoke highly of the practical and theoretical training.  For training to be effective it needs to be of sufficient duration and include practical skills training.  Training needs to have a balance of practical and theoretical content.  The farmer training conducted by DAC is a good model and should be adopted by Avansa Agrikultura. 104 Evaluation Questions Preliminary/Interim Findings Preliminary Analysis and Conclusions Preliminary Recommendations  Some trainings were inappropriate or not practical. E.g. Potato cultivation in lowland areas; financial training by Timor Aid proved inaccessible to farmers. Raised bed system was inappropriate for soil and water conditions in Miggir.  Some training was inappropriate to local growing conditions and for some farmers’ educational capability, especially some financial training  Training should be adapted to farmers’ educational capability.  Crop related training should be checked against local growing conditions and constraints  Agronomic technical skills are required for project activities to be adapted to local conditions.  Wind damage to tunnels and greenhouses are common especially in Aileu. So plasticulture needs to be tested before wide adoption everywhere.  Some technologies are not appropriate to all project locations e.g. standard raised beds, quality of plasticulture, crop type matching to environment.  Technologies should be regularly monitored to ensure they are locally appropriate.  The training in Indonesia was highly effective (one month in Bandung).  Farmers presented finding from International training to MAF staff, other local Government officials, and to other farmers.  Visits to Indonesia allowed farmers to observe successful application of technologies in other locations.  In some cases international visits are inaccessible to farmers due to own contribution for passport and other costs.  Project should contribute travel costs including passport and document processing.  Farmers appreciated the cross visits to see technology in other places (in particular the non Aileu farmers).  Cross visits are an important aspect of farmer training.  Cross visits should be planned as part of farmer trainings. 105 Evaluation Questions Preliminary/Interim Findings Preliminary Analysis and Conclusions Preliminary Recommendations  Tetun and Indonesian manuals were presented by farmers and appreciated as reference material. Group leaders keep the manuals.  Indonesian manual were only provided to farmers who attended training in Bandung.  Observation suggested that farmers were not using the manuals regularly. Only the group leader has a copy of the manuals.  If farmers are illiterate the manuals may not be appropriate to farmers’ level of education.  Training materials need to be more appropriate for illiterate farmers. More pictures and diagrams rather than textual instructions.  Farmers stated that linking farmers to the supermarket buyers was the most effective intervention provided by DAC.  DAC adopted a good approach when linking farmers to supermarkets; however, there are risks because farmers are reliant on a single link in the value chain.  It is uncertain whether supermarket buyers have a viable commercial relationship with farmers.  Demonstrates the importance of linking farmers to the market.  Need to have multiple relationships with the market. Josephine Farm established contract farming relationships with farmers but this is still a risky model as there is only one micro entrepreneur arranging contact farming.  Some groups which stared late in DAC did not complete their infrastructure especially for water reticulation.  GDA 2 groups only started selling products in May 2014 and so only sold produce for 10 months under the project  Do not start new groups late in the project cycle.  Farmers were provided with free inputs and this has created a dependency of expecting everything free.  All inputs were provided free to farmers. This does not encourage a commercial mentality amongst farmers.  A farmer contribution should be required for all inputs supplied by the project.  Donors all need to have a consensus policy of not providing totally free inputs. 106 Evaluation Questions Preliminary/Interim Findings Preliminary Analysis and Conclusions Preliminary Recommendations  Josephina Farm: Alternative business supported by ILO. This model uses an entrepreneur for organizing contract farming with farmers and to negotiate sales with supermarkets. However, stress related illness for owner; 150 farmers down to 50 farmers; 20 year time horizon required for viable business model development.  Any marketing arrangement which relies on one link in the value chain is associated with risks of non￾sustainability.  DSP supported Zero Star as a trader but eventually this business failed when DSP support ended; it then stopped buying from farmers.  Multiple trading arrangements should be established to ensure project continuity if one business link should fail. For example: Multiple buyers selling to multiple outlets in a coordinated manner.