ITIKI Performance Evaluation Bridging Indigenous Knowledge and Science Drought Prediction Tool in Kenya SECURING WATER FOR FOOD AUGUST 2019 Prepared By: Amanda Satterwhite, Master’s student, The Bush School at Texas A&M University The Kaizen Company | Contract #: AID-OAA-C-15-00011 www.securingwaterforfood.org | securingwaterforfood@gmail.com TABLE OF CONTENTS ABSTRACT 2 INTRODUCTION 3 BACKGROUND 5 Gender 5 Age and family size 6 Farm size 7 Sources of income 8 METHODOLOGY 9 RESULTS 13 EXPERIENCE WITH INNOVATOR 14 Familiarity with ITIKI 14 Perceptions of ITIKI 16 BENEFITS OF INNOVATION 18 Farming practices 18 Water usage and access 20 Crop yield and survival 22 Income 24 Input usage 27 Gender differences 28 Regional differences 29 DISCUSSION 33 Innovation usage 34 Crop yield and survival 35 Income 36 Gender differences 36 Affordability 37 Other benefits 38 Difficulties using ITIKI 41 User recommendations for improvement 42 CONCLUSION 43 ANNEX I: SURVEY INSTRUMENT 45 2 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 This evaluation assesses the effects of the ITIKI drought prediction tool on farming practices and outcomes in Embu County, Kenya. ITIKI combines indigenous weather indicators with meteorological data to generate localized predictions on the timing of rain onset and approximate volume of rainfall for each growing season (i.e. “less than normal” rainfall, “normal” rainfall) and sends this information to farmers via text message. The evaluator surveyed 62 registered ITIKI users, who were selected through a cluster randomized sampling design, regarding their experience using this innovation. User perceptions of the innovation were mostly positive. Despite drought in recent seasons, users felt ITIKI intelligence had improved their crop yields compared to what yields would have been without the information. The reported effects occurred through a change in farming practices in which, based on the weather predictions, users planted early and changed to more drought-resistant crops and varieties. However, approximately 20 percent of respondents were unfamiliar with the innovation despite being registered users, which highlights the need to address issues with ITIKI’s recruitment and follow-up. Quantitatively measurable effects are sparse because ITIKI has operated for a limited time in Kenya and highly variable rainfall makes it impossible to attribute changes in yields or income across seasons to the innovation. Nonetheless, positive user perceptions of ITIKI and reported changes in farming practices as a result of ITIKI intelligence suggest the innovation is helping farmers cope with drought. ABSTRACT SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 3 INTRODUCTION 4 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 In Kenya, the vast majority of smallholder farmers depend on rainfall to water their crops.1 Yet, variable rainfall patterns mean drought is a common reality, leaving these farmers with harvests too small to feed their families. The ITIKI drought prediction tool seeks to help farmers adapt to less-than-normal rainfall levels by providing information on the timing and amount of rains. With this data, farmers can take advantage of the entire season of rainfall by planting before its onset. Furthermore, they can use information on predicted levels of rainfall to decide whether to plant more drought-resistant crops or varieties. ITIKI is unique because it uses both meteorological information from weather sensing stations and indigenous indicators to predict rainfall. These indicators are signs used for generations to plan for the growing season. They include the behavior of birds and insects, the position of stars, and the flowering of trees and plants. For example, people in the Embu region of Kenya noticed dragonflies begin flying near the ground when the rains are one to two weeks from onset. By combining these information sources, ITIKI seeks to make predictions that utilize rich cultural knowledge and are scientifically rigorous. Formulated through a computer algorithm, the predictions are sent to farmers via text messages in their local languages. While ITIKI currently operates in Kenya, Mozambique, and South Africa, this evaluation investigated its effects in Embu County, Kenya (its current area of operation in Kenya). The evaluator conducted 62 interviews with registered ITIKI users in 14 villages. Respondents were selected through a cluster randomized design and surveyed regarding their farm activities, use of ITIKI, and perception of its benefits, among other topics. The evaluator also conducted key informant interviews with ITIKI founder Muthoni Masinde and local project manager Peter Ngoci and held a focus group discussion with ITIKI ambassadors. Findings from these interviews helped triangulate and explain quantitative survey results. These results illuminate areas where ITIKI has helped users adapt their farming practices to less-than-normal rainfall levels. The findings also highlight challenges to ITIKI uptake and impact, which are discussed here to help the organization identify areas for improvement. The report proceeds as follows. The Background section discusses demographic characteristics of interviewed farmers. Section II explains sampling methodology and data collection activities. Section III presents results, including farmers’ familiarity with and usage of ITIKI; effects in terms of farming practices, crop yields, income, and water usage; and, differences in results by gender and region. Section IV discusses these results and elaborates on challenges and recommendations for improvement. Section V provides a conclusion. 1 The Food and Agriculture Organization of the United Nations reports that on farms below the median size in Kenya, only six percent of land is irrigated. (Food and Agriculture Organization of the United Nations. Family Farming Knowledge Platform. Accessed from http://www.fao.org/family-farming/data-sources/dataportrait/indicator-details/en/?ind=83468.) SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 5 The field evaluator surveyed 62 farmers in 14 villages in Embu County, Kenya. The survey used Fulcrum mobile technology and probed respondents’ personal and farm characteristics and experience with ITIKI. The full survey instrument is in Annex I. This section presents demographic characteristics of the sample. (The sampling methodology is covered in detail in the next section.) Gender Of the surveyed farmers, 73 percent (45 of 62) were women (Figure 1). This mirrors the registered ITIKI user population: among the sample’s 14 villages , women made up an average of 72 percent (and a median of 75 percent) of ITIKI users. While the sample gender breakdown is representative of the population, this was not the case for each village. In some villages, it was more difficult to schedule interviews with male respondents because they appeared more likely than women to be away from the home. FIGURE 1: GENDER DISTRIBUTION OF SURVEY RESPONDENTS N=62 73% 27% GENDER DISTRIBUTION OF SURVEY RESPONDENTS Female Male BACKGROUND 6 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 Age and family size Survey respondents ranged in age from 24 to 80, with a mean age of 49 years (Figure 2). Female respondents were, on average, significantly younger than males, with a mean age of 47 years compared to males’ mean age of 55 years. Older respondents were more likely to be unfamiliar with ITIKI, despite being listed as registered users. The mean age of the 12 non-users was 57 years, compared to 47 years for users. This age difference is statistically significant at the 95 percent confidence level, a finding which is discussed in Section IV. Average household size was 5.9 members, with no statistically significant difference between male and female respondents or between ITIKI users and non-users. Household size ranged from one person (respondent only) to 15 people. FIGURE 2: AGE DISTRIBUTION OF SURVEY RESPONDENTS AGE DISTRIBUTION OF SURVEY RESPONDENTS N=62 0 24-39 40-50 56-70 71-80 NUMBER OF RESPONDENTS 19 6 15 22 20 25 15 10 5 AGE GROUP SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 7 Farm size Farm size among respondents ranged from 0.25 acres to 13 acres, with a mean acreage of 3.5 and a median of 2.5 (Figure 3). Plot size varied by gender, with an average plot size 3.2 acres for women and 4.3 acres for men. However, the largest plot among respondents was 13 acres and was owned by a female respondent, even though only one acre was actively farmed. One respondent was unable to estimate her farm size. FIGURE 3: FARM SIZE OF SURVEY RESPONDENTS N=61 FARM SIZE (ACRES) 20 25 30 35 15 10 5 0 <=1 1-3 4-9 10-13 NUMBER OF RESPONDENTS 10 6 16 29 ACRES Furthermore, 45 of 62 respondents (73 percent) owned all their land, while four respondents owned some of their land. Of the 17 respondents who owned none or some of their land, only two paid rent. The remainder were granted the right to use land owned by a parent or other relative or simply used government land without holding a land title. The percentage of respondents who owned the land they farmed was similar among men and women. 8 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 Sources of income All respondents were engaged in farming. On average, respondents grew four different crops, with maize, cowpeas, green grams, sorghum, and millet as the most common crops. Many respondents were subsistence farmers: 21 respondents (33 percent) reported they did not sell any crops in the last year, while 15 sold only one crop and kept the rest for household consumption. While many farmers engaged in income-generating activities off the farm, 58 of 62 (94 percent) reported farming as their primary occupation while 21 respondents (33 percent) indicated farming as their sole occupation (Figure 4). Nine respondents (14 percent) also raised and sold livestock. It is probable that some respondents who reported farming as their sole occupation may also raise livestock but consider it part of their farming business and did not report the activity. Additionally, 21 respondents (33 percent) reported a small enterprise, such as operating a motor-taxi service, running a small shop or grocery, or sewing clothing. Lastly, 12 respondents (19 percent) engaged in wage labor, often in a stone quarry or on a khat farm (khat is a major cash crop in the region). FIGURE 4: INCOME SOURCES N=62 33% 33% 14% 19% FARMER OCCUPATIONS Farming ONLY Small Enterprise Livestock Wage Labor SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 9 METHODOLOGY 10 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 The sample was selected through a cluster randomized design, based on a list of over 2,500 registered ITIKI users across 245 villages.2 Villages’ mean number of users was 11, while the median was 3. About one-third of villages had only one user. To improve fieldwork efficiency, the evaluator restricted the village (cluster) selection to villages with users at or above the mean (11 users). This yielded 48 villages. ITIKI began operations in the villages surrounding Kiritiri town, in Mbeere South sub-county of Embu County (Figure 5). In 2018, users were recruited in the Mwea area and in Mbeere South. The local ITIKI manager expressed concern that users in Mwea would be less familiar with ITIKI because they were recruited by contacts local to Mwea rather than the ITIKI manager or ITIKI ambassadors. This may have lead to less complete sensitization to ITIKI than experienced in Kiritiri area villages.3 Furthermore, no ITIKI ambassador was posted in the Mwea villages so Mwea users may not have benefitted from the same follow-up as Kiritiri users. The issues surrounding recruitment and ambassador presence are discussed later in this report. However, the potential difference in Kiritiri and Mwea user experiences did influence the evaluator’s selection of villages. To assess impact among individuals who had ample experience with ITIKI while avoiding the bias of only selecting more seasoned users, the evaluator selected 12 villages in the Kiritiri area and four in the Mwea area. These were randomly picked from the restricted list of 48 villages. FIGURE 5: KIRITIRI TOWN, EMBU’S CENTER OF OPERATIONS IN KENYA 2 The list included 2,745 users, but it is noted that several duplicate names were found. The evaluator did not identify every instance of duplication, as some names are written differently in different entries (e.g., alternate spelling, middle name used vs. omitted), and thus identifying all instances requires a line-by-line review of the list. Similarly, some village names had variant spellings and appeared as different villages in the list. While the evaluator attempted to correct this with assistance from ITIKI staff, it is likely that the count of 245 villages is somewhat overestimated. 3 During the time of recruitment in Mwea, communities were experiencing tension and violence over land rights. Because of this, community members were suspicious of outsiders. The local ITIKI manager thus chose to recruit a local contact to introduce ITIKI to these communities, rather than visiting himself or sending an ambassador from Kiritiri area. SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 11 For each of the 16 selected villages, the evaluator stratified users by gender. Six were then randomly selected from each village, with the number of males and females determined by the ratio of users in the village.4 For each village, three to five females were selected. In many cases, there was no phone number for the user , and these users were replaced with new randomly selected users. All six selected individuals were contacted for interviews, with the goal of confirming at least four interviews. In cases where contacting six people did not yield four scheduled interviews, two to four randomly selected users were contacted. Because some confirmed interviewees did not arrive for their interviews during the first few days of fieldwork, the evaluator began to schedule additional interviews in each village in an effort to conduct at least four. For this reason, the evaluator conducted up to six interviews in some villages.5 Unfortunately, though the overall sample’s gender breakdown (73 percent female) closely matched the average percentage of female users among the selected villages (72 percent), the intended gender ratios did not hold for each village (Figure 6). In Makima, Gatirari, Gatoori North, Ngarira, and Irabari, the difference in percentages of women in the population (all ITIKI users) and in the sample was more than 20 percentage points. Despite contacting the correct ratio of women and men, a small sample in each village made it impossible to control the gender ratio of individuals who actually arrived for the interview. FIGURE 6: WOMEN AS A PERCENTAGE OF TOTAL USERS AND SAMPLED USERS, BY VILLAGE The sampled users were 62, the “total users” were approximately 2,500 0% 60% 40% 20% 70% 50% 30% 10% 90% 100% 80% WOMEN AS A PERCENTAGE OF TOTAL USERS AND SAMPLED USERS, BY VILLAGE PERCENTAGE Women as % of total users Women as % of village sample Kiambere Ngiori Makima Rogogwe Kathera Gatirari Kandongu Gatoori North Gatumbiri Kariari Ngarira Namuri Irabai Mariguru VILLAGE 4 The percentage of females in each village was calculated, then multiplied by six and rounded to the nearest whole number to calculate the number of females and males in the sample for each village. For example, in Ngiori, 81% of users are female, equivalent to 4.8 out of 6. Thus, 5 females – and therefore one male – were selected for the sample in Ngiori. 5 It should be noted that respondents in one of the selected villages, Kiambere, did not live in Kiambere but rather in surrounding villages. During recruitment, their locations had been recorded as Kiambere. During the interviews, the respondents’ villages of residence were recorded. These were Giatugu, Gituri, Kaithungu, Kanthenge, and Kinyaga. In this report, however, Kiambere is considered as a single village cluster in alignment with the ITIKI user list. 12 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 When visiting the first two villages in Mwea (Makima and Namuri), the evaluator found six out of nine respondents were unfamiliar with ITIKI and did not remember receiving a message. Rather than waste resources with visits to villages where few individuals were familiar with ITIKI, three scheduled respondents from the remaining two villages in Mwea were contacted. None had heard of ITIKI, and the remaining two villages in Mwea (Kamweli and Ndune) were not visited. In all, 14 villages were visited and 91respondents were interviewed (Figure 7). FIGURE 7: SAMPLED VILLAGES Interviews were conducted using the Fulcrum mobile application. At the beginning of each interview, the evaluator explained the evaluation’s purpose and the types of questions that would be asked. She advised respondents to notify the evaluator if they did not wish to answer a question, and it would be skipped. Each respondent was asked to verbally confirm their desire to participate, and confirmation was captured in audio recordings. While the evaluator conducted a few interviews in English or Swahili, most respondents felt most comfortable communicating in their local language. For the villages in Kiritiri area, this was Kimbeere. For the villages in the Mwea area, this was Kikamba. Thus, for most interviews, communication occurred through an interpreter. Lastly, the evaluator conducted a focus group discussion with three ITIKI ambassadors. They were asked to discuss their role, the reaction of users to ITIKI when it is introduced, and any challenges in recruitment. The evaluator also carried out a key informant interview with local ITIKI project managerPeter Ngoci, to gather information about the user recruitment process, the collection of indigenous indicators, and other project operations. SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 13 RESULTS 14 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 Familiarity with ITIKI ITIKI was first introduced in Kenya in late 2016 and was gradually rolled out throughout Embu County. While 12 respondents (19%) were not familiar with ITIKI, the majority who do use it have done so for approximately 1.5 to 2 years (Figure 8). This is consistent with ITIKI’s heavy recruiting in the second half of 2017. Furthermore, because recruitment of ITIKI users occurs through village-level meetings, there was a moderately strong correlation of length of use among respondents in a given village (intraclass correlation coefficient of 0.52). FIGURE 8: DISTRIBUTION OF LENGTH OF ITIKI USE N=50 DISTRIBUTION OF LENGTH OF ITIKI USE 20% 30% 40% 50% 25% 35% 45% 15% 10% 5% Does not use About 1 year 1.5–2 years 2.5–3 years PERCENTAGE 19% 19% 16% 45% 0 Most respondents had difficulty answering the “length of use” survey question. The evaluator assisted respondents by asking about specific growing seasons and whether they had used ITIKI in the given season. Still, recall was a challenge, and many answers were rough estimates. Furthermore, ITIKI did not send any messages during the October to December 2018 season because of logistical problems with mobile telephone service providers. These problems persisted with some (though not all) mobile telephone companies into the March to May 2019 season, so not all users EXPERIENCE WITH INNOVATOR SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 15 received messages during that season. The villages with the greatest percentage of respondents unfamiliar with ITIKI were those in the Mwea area. According to ITIKI staff, recruitment occurred in this area in mid-2018 before the logistical problems began. The fact that many in this area never received messages after signing up helps explain their unfamiliarity with ITIKI. Familiarity with ITIKI is correlated with some demographic characteristics. Men were less likely to be ITIKI users despite being on the user list: only 65 percent of male respondents were familiar with ITIKI, compared to 87 percent of female respondents. Age was also predictive: two-thirds of non-users were above the median age of 46, and the mean age of non-users was statistically significantly higher than that of users. Interestingly, ITIKI users had smaller farms than non-users on average . Among users, the mean farm size was 3.2 acres, compared to 4.9 acres for non-users. This difference is statistically significant at the 95 percent level. At the same time, there is a moderately strong, positive correlation between age and farm acreage (correlation coefficient of 0.45), so it is possible that the difference in use is explained more by age than by farm size. ITIKI Ambassadors 16 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 The evaluator found that 78 percent of respondents familiar with ITIKI (39 of 50) had heard about it from an ITIKI ambassador or from local project manager Peter Ngoci. ITIKI has contracted seven ambassadors, who are residents of villages where ITIKI operates and are charged with enrolling village residents to receive ITIKI messages. Ambassadors do this through a short presentation at an existing village meeting , such as a church service, a village savings and loan association meeting, or another common interest group’s meeting, e.g. an association of livestock farmers. After the short presentation to“sensitize” people to ITIKI, the ambassador schedules a follow up meeting with interested individuals to enroll in ITIKI. Ambassadors are assigned to recruit individuals in their home village and in two nearby villages. They are charged with following up with ITIKI users before and throughout the growing season to remind them of weather predictions and encourage them to adjust their farming plans accordingly. With this “hands on” approach, it is not surprising that the villages where the majority of respondents reported being unfamiliar with ITIKI were those with no ambassador. These include Ngarira (in Kiritiri area) and Makima and Namuri (both in Mwea area). These villages had been introduced to ITIKI, but did not receive follow up. The focus group discussion with ITIKI ambassadors revealed they often reminded ITIKI users of the presence of “indigenous indicators” that help predict timing and amount of rainfall. Thus, even during the season in which no messages were sent, ITIKI users in villages with ambassadors received weather information. However, in the villages without an assigned ambassador, the break in messages in 2018 (and into 2019 for some users, depending on their mobile telephone service provider) meant no information was received. This sheds light on the frequency of respondents reporting unfamiliarity with ITIKI in these villages. Of the remaining 22 percent of ITIKI users that did not report hearing about the innovation from ITIKI staff or ambassadors, nine of 11 respondents noted they had heard about it from a neighbor. This may have been an ITIKI ambassador, but the respondent did not recognize the neighbor as such. An additional respondent was told about ITIKI from “someone he didn’t know from outside his village,” which may have been an ambassador or Peter Ngoci. The remaining ITIKI user did not remember where she heard about the technology. Perceptions of ITIKI Among respondents familiar with ITIKI, general perceptions of the technology were positive. None of the 50 users reported any reluctance to recommend ITIKI to their friends or neighbors, with 40 percent responding they would recommend it, and 60 percent reporting that they would strongly recommend it (Figure 9). One respondent who reported that she would recommend ITIKI seemed reluctant to do so, possibly because the respondent had not reported any income or yield benefits from ITIKI. When pressed, the respondent maintained that she would recommend the technology. While it is possible that some respondents may have reported they would recommend the technology out of fear of appearing ungrateful for the project (thus introducing bias in survey results), many respondents demonstrated enthusiasm in their recommendations. SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 17 FIGURE 9: USER WILLINGNESS TO RECOMMEND ITIKI N=50 60% 40% USER WILLINGNESS TO RECOMMEND ITIKI Would Strongly Recommend Would Recommend Furthermore, 46 of 50 users reported they shared the knowledge gained through ITIKI updates with their families or communities. For example, one noted, “Yes, just today I was sharing my knowledge with my neighbors!” Another related that community members ask how her crops perform better than theirs, and she tells them it is because of knowledge gained through ITIKI. Additionally, all 50 ITIKI users reported they would continue using it because the weather information was helpful for their farming practice. Once ITIKI was explained to the non-users, 11 of the 12 reported they would be interested in using it in the future. One non-user said, “If I knew what the rains would be like, I could make better decisions on what to plant. I wouldn’t waste so much time planting things that will not grow.” Another qualified her answer, explaining that she would be willing to use ITIKI information if she first saw that it had benefited existing users. The single non-user who said she would not use ITIKI in the future did so because she did not own a phone. 18 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 Farming practices By providing farmers with predictions on when the rains will arrive and how heavy they will be, ITIKI enables farmers to change their farming practices to help cope with drought. Among the most important changes is early planting. Traditionally, farmers wait until the rains arrive to plant, but when they are informed on when the rains will arrive, they can have their seeds in the field before this occurs. In the region served by ITIKI, the rainy period is often too short; one respondent said her village only had one day of rain during the most recent season. By planting before the rains come, farmers improve crop survival and yields by taking advantage of every opportunity to water their crops. Early planting was the most common benefit cited by respondents. Thirty-seven of 50 users (74 percent) mentioned it in response to an open-ended question about ITIKI benefits. Furthermore, 49 of 50 users indicated that ITIKI helped them decide when to plant in response to a question about how technology has changed their farming practices (Figure 10). FIGURE 10: CHANGES IN FARMING PRACTICES N=50 98% CHANGES IN FARMING PRACTICES % of ITIKI Users Who Changed Time of Planting 76% % of ITIKI Users Who Selected Different Crops/Varieities Respondents also commonly reported that ITIKI information about how long the rains would last influenced their selection of crops and varieties. Thirty-eight of 50 users (76 percent) stated that knowledge of impending drought led them to make one or more of the following changes: BENEFITS OF INNOVATION SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 19 (1) Add crops that perform better in drought, such as sorghum and cowpeas, or increase their acreage. Thirty-three of 50 users (66 percent) reported adding a new drought-resistant crop or increasing acreage because of ITIKI information (Figure 11). The most common addition was sorghum, with nearly one-half of users having introduced it or increased its acreage after ITIKI. Green grams also appeared to have been a popular choice, with 21 of 50 users introducing it or planting more. Lastly, 16 users began or increased cultivation of cowpeas and millet, respectively. FIGURE 11: USER ADDITION OF DROUGHT-RESISTANT CROPS N=50 USER ADDITION OF DROUGHT-RESISTANT CROPS 40% 60% 80% 20% Any crop Cowpeas Green grams Sorghum Millet PERCENTAGE 66% 32% 48% 42% 32% 0 While these were the most common crops added because of ITIKI, others were mentioned by users. One respondent began cultivation of sweet yellow (a fruit similar to passionfruit) and papaya. Another reported starting cultivation of tomatoes, and a third added acreage to his pigeon pea cultivation because of ITIKI information. (2) Eliminate crops that require more water, such as maize and beans, or reduce acreage of these crops. Twenty-three ITIKI users (46 percent) stopped or reduced their bean cultivation, while 10 users (20 percent) stopped or reduced maize cultivation. It is worth noting that even among the 12 non-users, two respondents had stopped growing beans and one had stopped growing maize. This raises the possibility that these changes are due not to ITIKI, but simply to the region’s increasing frequency of drought. Despite the small sample size of non-users and thus the need for caution in interpreting results, the rate at which ITIKI users abandoned bean cultivation is higher than the rate among non￾users, with the difference statistically significant at the 95 percent level. While the rate at which users abandoned maize also is higher than the rate at which non-users abandoned maize, the difference is not statistically significant. Since maize is the most important staple crop in the region, farmers may be much more reticent to stop or reduce their maize cultivation. 20 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 (3) Purchasing and planting a drought-resistant seed variety of one (or more) of their existing crops. Although respondents were not directly asked whether ITIKI information led them to switch to drought-resistant seeds, 13 of 50 respondents (26 percent) mentioned this as an ITIKI benefit. The evaluator analyzed changes in seed purchase before and after ITIKI. Thirteen respondents (including three who mentioned changing seed varieties due to ITIKI) reported purchasing seeds for one or more crops in the season after they began using ITIKI, although they did not purchase seeds in the season before ITIKI and instead reused seeds from their own harvest. This may be indicative of a change to drought-resistant varieties. However, it is unlikely that this change was driven by ITIKI, as almost as many respondents (10) reported they had bought seeds in the season before ITIKI, but not in the season after. Thus, it is impossible to draw a conclusion about the switch to drought-resistant varieties based on farm input data. Water usage and access In Embu County, the vast majority of farmers rely only on rain for their agricultural activities. Among the 50 ITIKI users interviewed, 43 (86 percent) have no other source of water for crop irrigation (Figure 12). The percentage is similar among non-users, with 10 out of 12 reporting rain as their only agricultural water source. SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 21 The most common alternate source of water was river water. Five respondents (three from Makima village, one of whom is an ITIKI user, and 2 from the villages surrounding Kiambere, both of whom are ITIKI users) reported using river water for irrigation. Three had a pump to deliver the water from the river to their farms, while one carried it by donkey and watered his crops by hand, and another carried it by hand. One used river water for khat and tomato crops but not staple cereal crops. Respondents in the remaining 12 villages did not have access to river water for irrigation. FIGURE 12: SOURCE OF WATER FOR IRRIGATION N=50 86% 2% 2% 6% 4% CHANGES IN FARMING PRACTICES Rain Only Own Pond River Groundwater Tap Water Two ITIKI users had their own ponds (plastic-lined trenches near their farms), while another hand￾watered her crops with water from a well. A final ITIKI user watered her kitchen garden with tap water, but relied on rain water for her staple crops. Since most of the respondents relied on rainwater, they were unable to quantify the amount of water used before and after ITIKI. All 43 ITIKI users who relied on rainfall alone reported no change in their water use before and after ITIKI, though many made comments about rainfall being sparser during the last season. Four additional users of river water, groundwater, or tap water also reported no change in water use (Figure 13). One ITIKI user reported that now using less water from her pond because she plants early and now relies more on rainfall and less on supplemental water. Another user reported pumping less river water to his tomatoes for the same reason. Neither was able to quantify the water savings. A final ITIKI user reported using more river water after ITIKI, but not because of ITIKI. Because the drought has been more severe since he began using ITIKI, he now uses about 800 liters of river water daily, compared to 300 liters daily in the season before ITIKI. Finally, no respondents reported any change in water access using ITIKI. 22 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 FIGURE 13: CHANGE IN WATER USE N=50 94% 4% 2% CHANGE IN WATER USE AFTER ITIKI No Change More Less Crop yield and survival Since the majority of respondents rely on rain for their yields, highly variable rainfall means large variation in yields across seasons. In the most recent season (March to May 2019), rainfall was extremely sparse, with 29 of 50 respondents (58 percent) reporting they had harvested nothing. An additional seven respondents had only harvested one of the crops they planted, although most people planted three or four different crops in the season. While prior seasons had some rainfall, respondents did not consider rains to have been “good” for the past two years. To collect data to compare yields before and after ITIKI, the evaluator asked about previous seasons, going back to March to May 2016 depending on how long the respondent had been using ITIKI. However, respondents frequently mentioned different levels of rainfall in each season and often remembered “good rains” when asked about growing seasons prior to ITIKI. In some cases, respondents who had used ITIKI for more than two years had difficulty recalling yields from seasons before ITIKI. Many respondents reported changing their crop acreage season-to-season or adding or eliminating crops. These variations make it impossible to conduct a quantitative comparison of yields before and after ITIKI. However, when asked whether ITIKI had improved their yields, respondents were very positive (Figure 14). Thirty-seven of 50 respondents (74 percent) felt their yields had either increased or increased substantially. Another nine respondents (22 percent) felt yields had remained the same or increased only slightly, although four made statements such as, “Because of the drought, my yields didn’t improve. But they would have increased if the rains had been better.” Two respondents reported their yields had declined, with one clarifying it was because of the lack of rain. SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 23 FIGURE 14: CHANGE IN CROP YIELDS N=50 56% 18% 16% 6% 4% CHANGE IN CROP YIELDS Increased Increased Substantially Declined Remained the Same Increase a Bit The evaluator sensed some respondents did not feel their yields had increased, but reported an increase so as not to appear negative about ITIKI. The evaluator probed these respondents and asked if yields had truly increased or were lower because of the lack of rain. Most respondents kept their original answer, though some qualified an earlier statement that yields had increased and explained that they had only increased “a little bit.” Some appeared to have answered with a comparison of their yields to what their yields would have been without ITIKI, with a few making statements such as, “My yields are lower than before, but without ITIKI I would not have harvested anything.” Based on this comparison, these respondents considered their yields to have “increased.” Similarly, of the 33 ITIKI users who reported having grown new crops or added acreage to existing crops because of ITIKI, 28 reported these crops had done well, while an additional four reported one or more of the crops grown because of ITIKI had done well, while other(s) had not (Figure 15). One respondent stated that neither crop he grew because of ITIKI (cowpeas and sorghum) had done well, but the reason was drought. 24 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 FIGURE 15: PERCEPTIONS OF CROP SURVIVAL N=33 28 4 1 PERCEPTIONS OF CROP SURVIVAL Yes Mixed No Perceptions of crop survival also were positive. Forty-four of 50 users reported crop survival had improved with ITIKI. Nine commented the change was due to planting the right crops (namely, drought-resistant crops rather than maize and beans), while three users felt the improved survival was a product of early planting. Two respondents explicitly qualified that improvements in this area were only seen for the drought resistant crops (with several respondents mentioning millet, cowpeas, sorghum, and green grams), while survival rates for maize and beans did not improve. The same result appeared in analyzing respondents’ answers about the benefits of ITIKI: only two of 50 felt that ITIKI had helped them produce more of their “most important crop,” which is maize in this region. Income It was difficult for respondents to accurately estimate their incomes. For those who sell crops, income varies from season to season and is inconsistent across years. Because those respondents with small enterprises such as motor-taxi services or sewing businesses have farmers as their clients, their income also varies with the agricultural seasons. To assist respondents in estimating their incomes, the evaluator asked about monthly or seasonal incomes and estimated annual incomes. Even with this aid, 13 of 50 users were not able to provide an estimate and another eight were unwilling to share. Some hinted at having a pension or a spouse with a pension, although they were not comfortable disclosing the amount. Among the 29 users who provided an estimated income, 28 fell into the category of “extreme poor” (under 240,000 KSh per year, or $2,400 USD), and one reported an annual income of 360,000 KSh, qualifying as “poor” (240,001 KSh to 480,000 KSh per year) (Figure 16). ITIKI is thus well-targeted to the extreme poor. (Mean income for ITIKI users in the before-ITIKI period was 85,700 KSh annually, while the mean income for non-users was 98,300 KSh. However, this difference in mean income between users and non-users is not statistically significant.) SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 25 FIGURE 16: ANNUAL INCOME N=50 56% 26% 2% 16% ANNUAL INCOME AMONG ITIKI USERS Extreme Poor Poor Does Not Know Not Willing to Share Income gains after ITIKI were limited. Only seven users reported a change in income, with an average increase of about 30,000 KSh. However, including the users who reported no change in income, the average increase was only 8,500 KSh (with a median of 0 KSh). Furthermore, one of the seven users who reported increased income clarified that the increase happened after ITIKI was introduced but was not because of ITIKI. While ITIKI’s improvement of pecuniary income was limited, many ITIKI users reported the innovation had improved their income because it had helped them grow more food. Because of this, they bought less food for their families and had more money to spend on other things. Thirty-two of 50 users (64 percent) felt ITIKI had improved their income “somewhat,” while another nine users (18 percent) felt it had improved their income “significantly” or “very significantly” (Figure 17). Of the 41 users who felt ITIKI had improved their income at least somewhat, 20 (49 percent) noted that it had improved income through increasing food for the household. 26 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 FIGURE 17: PERCEIVED INCOME IMPROVEMENTS N=50 64% 8% 10% 18% AMOUNT ITIKI IMPROVED INCOME Somewhat Significantly Very Significantly None Respondents spent additional income in a variety of ways, but the most common was school fees. Thirty-one of the 41 ITIKI users who reported an income improvement after using ITIKI put it toward schooling for their children or grandchildren (Figure 18). In Kenya, even public schools charge fees, which are quite high considering the average household income. Other popular uses of additional income were on home improvements, such as buying a water tank, or farm investments, such as buying livestock. Buying additional food not grown on the farm also was common, with 11 of 41 respondents reporting this use to purchase items such as salt, sugar, or tea. SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 27 FIGURE 18: RESPONDENTS’ REPORTED USE OF ADDITIONAL INCOME N=41 USER ADDITION OF DROUGHT-RESISTANT CROPS 15 25 35 10 20 30 5 School Fees House Improvements Farm Investment Business Investment (non-farm) Food for Family Clothing for Family Health Costs Savings NUMBER OF RESPONDENTS 31 9 2 2 4 2 7 11 0 Within the ITIKI user sample, no respondent reported a change in income that moved them out of the “extreme poor” or “low income”categories. In fact, with an average annual income of 94,200 KSh in the period after ITIKI (and a median of 80,000 KSh), most users remain far below the extreme poverty line. Input usage Respondents were asked about their use of farm inputs before and after ITIKI, specifically their spending on and usage of fertilizer, pesticide, herbicide, seeds, charcoal (as a natural pesticide), water, and family and hired labor. No systematic differences were found in use of any of these inputs when compared with the period before and after ITIKI. This is not surprising, given that ITIKI’s model does not include provision of or training in farm inputs. Three respondents mentioned learning about farm input usage as a result of ITIKI, but this knowledge sharing appears to have occurred on an ad-hoc basis during meetings where ITIKI was introduced. It also is possible that these respondents confused ITIKI’s work with that of another NGO. These issues are explored in detail in the “Discussion” section. 28 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 Gender differences Though respondents were chosen from registered ITIKI user list , some were unfamiliar with ITIKI messages. As noted in the previous section, this proved far more likely for men: only 65 percent of interviewed men reported having ever received a message from ITIKI, while 87 percent of female respondents were familiar with the ITIKI messages (Figure 19). This difference in ITIKI usage and gender breakdown among respondents reflects an overall trend in the region of low male participation in development projects, as reported by the local project manager. Furthermore, since ITIKI often is introduced at meetings of existing village groups where women are more likely to be present, women are more prone to hear about the innovation and have an opportunity to enroll. FIGURE 19: ITIKI UPTAKE, BY GENDER N=62 0 40 45 30 20 10 35 25 15 5 ITIKI UPTAKE, BY GENDER Female Male NUMBER OF RESPONDENTS 6 39 6 11 Users Non-users While women were more likely than men to use ITIKI, they were less likely to make changes in their crop choices. While 10 of 11 male users of ITIKI (91 percent) reported changing what they planted because of ITIKI, only 72 percent of female users did so (Figure 20, Panel 1). Though the small sample size for men requires caution in interpreting results, this difference between men and women is significant at the 90 percent level. Similarly, men were more likely than women to stop growing crops that do poorly in drought, with eight of 11 men (73 percent) reporting they had stopped or reduced maize or bean cultivation, while only 46 percent of women did so (Figure 20, Panel 2).6 This difference also is statistically significant at the 90 percent level. Lastly, men reported adding new crops based on ITIKI information at a slightly higher rate than women (Figure 20, Panel 3), but this difference is not statistically significant. 6 The question about previously grown crops did not probe whether respondents had stopped growing a crop because of ITIKI. Rather, it asked whether there were any crops they used to grow that they no longer cultivated (or of which they had reduced acreage). As noted in the “Farming practices” sub-section, however, the rate of abandoning bean cultivation (as a crop that is not drought-resistant) was higher for ITIKI users than for non-users. SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 29 FIGURE 20: CHANGES IN DECISIONS REGARDING CROPS, BY GENDER WOMEN: N=39 MEN: N=11 72% 28% CHANGES IN DECISIONS REGARDING CROPS, BY GENDER Yes No WOMEN 91% 9% Yes No MEN I have changed what I plant because of ITIKI 72% 28% Yes No WOMEN 73% 27% Yes No MEN I have stopped growing maize or beans 64% 36% Yes No WOMEN 73% 27% Yes No MEN I have added one or more new crops because of ITIKI Regional differences Respondents from villages with an ITIKI ambassador were far more likely to use ITIKI than those from villages without an ambassador. As explained previously, ITIKI contracted seven ambassadors in its areas of operation.7 These ambassadors are charged with recruiting users and following up with them before and during the growing seasons to discuss their farming practices relative to weather indicators and ITIKI updates. Each ambassador conducts recruitment and follow-up in his or her home village and in two additional villages. After interviews, the field evaluator asked the local project 7 Official contracting of ambassadors occurred in 2019. Before this time, ambassadors provided recruitment and follow-up services on a volunteer basis. 30 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 manager to identify the sampled villages with assigned ambassadors and found that seven of the selected villages had an assigned ambassador and seven did not. Only three of 32 respondents in the villages with an assigned ambassador were unfamiliar with ITIKI, while nine of 30 respondents (30 percent) in the villages with no ambassador were unfamiliar (Figure 21). This suggests an ambassador’s presence in a user’s village (or in a nearby village) matters for uptake of ITIKI among registered users. A focus group discussion with three ambassadors revealed they often discuss weather indicators with members of their villages. One ITIKI user noted the ambassador in her village gave her ITIKI updates through word of mouth. In contrast, when asked why registered ITIKI users in some villages reported having never heard of ITIKI, one ambassador believed it was due to lack of follow-up. She said she had been sent to conduct recruitment through one-time meetings in some villages. However, she did not return to discuss weather updates with registered users because the villages were far away. This indicates reinforcement of ITIKI updates through in￾person contact does impact usage. SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 31 FIGURE 21: ITIKI USAGE, DISAGGREGATED BY AMBASSADOR PRESENCE IN VILLAGE N=62 0 30 20 10 35 25 15 5 ITIKI USAGE, DISAGGREGATED BY AMBASSADOR PRESENCE IN VILLAGE Ambassador No Ambassador NUMBER OF RESPONDENTS 3 29 9 21 Users Non-users Additionally, uptake was notably different between villages in the Kiritiri and Mwea areas. Restricting the sample to only those villages without an ambassador showed respondents in Kiritiri villages were significantly more likely to be familiar with ITIKI than those in Mwea villages. Among the Kiritiri villages without an ambassador, 86 percent of respondents were familiar with ITIKI, compared with only 33 percent of respondents in the Mwea villages (though the sample of Mwea respondents was only nine individuals). This difference likely is rooted in recruitment methodology. While Kiritiri users were almost exclusively recruited by the local project manager or an ambassador, Mwea users were recruited by non-ITIKI contacts. This occurred because local residents in Mwea were experiencing conflict over land rights during recruitment. Because of this, the project manager was concerned that an outsider would be received with suspicion. Therefore, he liaised with a local contact to recruit users instead of visiting the villages himself or sending an official ITIKI ambassador. The quality of the recruitment efforts may have suffered, and new users may not have fully grasped the purpose or benefit of ITIKI. Another reason for the variation in recruitment was ITIKI’s partnership with a Nairobi-based organization that offers technical and financial support to farmers in Embu. This organization shared contact information for farmers who “graduated” from its program so these farmers might benefit from ITIKI updates. However, while these individuals were added to the ITIKI user list, the process by which ITIKI was explained to them by the partner organization – or whether it was explained at all – is unclear. It is possible these individuals were added to the user list without having been sensitized to the source of ITIKI updates. Because of this, SMS updates may be unexpected and perceived as lacking credibility, leading farmers to ignore the information. In sum, the different recruitment processes in Mwea translated into large differences in ITIKI uptake. 32 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 33 DISCUSSION 34 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 Innovation usage Familiarity with and usage of ITIKI were heavily correlated with demographic characteristics. First, the majority of ITIKI users are women. Because ITIKI recruitment usually occurs through meetings of existing village development groups (such as savings and loan groups or cooperatives of livestock farmers), men are less likely to be exposed to the innovation. As reported by the local project manager, men are generally much less involved than women in development projects or village associations working toward development goals. This is the likely reason that women made up, on average, 72 percent of ITIKI users among the sampled villages . While men are less likely to be users of ITIKI, those who registered as users also are less likely than women to be familiar with the technology. Despite being on the user list, only 65 percent of male respondents were familiar with ITIKI, compared to 87 percent of female respondents. Since women were more likely to have been sensitized to ITIKI through a village meeting, they are more likely to recognize the source of ITIKI messages, perceive the information as credible, and consider the information in their farming decision-making. Men, who are less likely to have undergone this sensitization, may have received the ITIKI messages but not recognized their source, thus reporting unfamiliarity with ITIKI during the interviews. Regardless, it is unclear how the men who reported unfamiliarity with ITIKI – particularly in villages with ITIKI ambassadors and where women were very familiar with the technology – became registered as users. Younger farmers also were more likely to be familiar with and users of ITIKI. Two-thirds of respondents who were non-users of ITIKI (despite being on the user list) were above the median age of 46, and the mean age of non-users was significantly higher. There are multiple reasons older users may be unfamiliar with ITIKI. Many reported not being able to read, which may prevent them from recognizing an ITIKI message has arrived. However, some users reporting illiteracy had their children or grandchildren read the messages to them, allowing them to use the information without issue. Some may not own a phone, such as one elderly woman who reported that she probably gave the recruiter her son’s number when asked how she was on the user list. It is possible that older people register for ITIKI when presented with the opportunity because others are doing so, even if they do not own a phone. In this case, they do not receive any messages. Most notably, usage of ITIKI was highly dependent on geography. Respondents in villages without an assigned ITIKI ambassador were far more likely to report unfamiliarity with ITIKI, highlighting the importance of ambassador follow-up. Registered ITIKI users are much more likely to recognize the arrival of a message, perceive it as credible, and employ the information in their farming decisions if the information has been reinforced by an ambassador as a member of the local community. Furthermore, respondents in Kiritiri-area villages reported familiarity with ITIKI at much higher rates than those in Mwea-area villages. Recruitment in Mwea did not occur through ITIKI staff or ambassadors, but rather through third parties that were either local contacts recruited by the project manager to sensitize individuals or representatives of the Nairobi-based partner organization. Therefore, the extent to which users understood ITIKI upon registration is dubious. It is possible that these users did not recognize ITIKI messages when they arrived and ignored them. SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 35 Crop yield and survival While quantitatively comparing crop yields and survival rates before and after ITIKI was impossible for reasons outlined in the previous section, qualitative analysis reveals most respondents felt use of ITIKI information had benefited them. Despite severe drought in recent seasons, the majority of users reported their yields and crop survival rates, even when poor, were better than they would have been without ITIKI. The primary reason for this change was early planting, with 98 percent of ITIKI users reporting the information led them to plant before the rains arrived, allowing crops to benefit from limited rainfall. These results suggest notification of when rains will arrive is the most valuable piece of information for ITIKI users. A secondary – yet still notable –reported driver of crop benefits was ITIKI information regarding the length of the rains, which led 76 percent of ITIKI users to change their crop selection. Based on predictions of less-than-normal rainfall, farmers chose to start planting or increase their acreage of drought-resistant crops, reduce their acreage of crops requiring more water, and switch to drought￾resistant varieties of their existing crops. Of the 33 ITIKI users who started growing new crops or added acreage to existing crops based on ITIKI information, 85 percent reported these crops had done well. Thus, ITIKI information on the length of rains also appears to be useful information for farmers, motivating decisions that increase crop yields and survival. 36 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 Income ITIKI had little to no impact on income, with less than 15 percent of respondents reporting an increase in income following ITIKI as compared with the period before ITIKI. Even these respondents appeared unsure of their income estimates, and they might have reported higher income after ITIKI because they felt it was the answer the evaluator desired despite encouragement to report income honestly. It is therefore impossible to conclude that ITIKI information improved users’ income. A primary reason is that the majority of respondents are subsistence farmers and do not sell crops or have very limited sales. Thirty-four percent of ITIKI users reported no sales of any crop in the last year, while an additional 22 percent reported selling only one crop in the last year, although most users cultivate four to five different crops, on average. Thus, even with an increase in crop yields, these farmers would not have earned additional income. However, over 80 percent of ITIKI users interviewed felt ITIKI had improved their income. When probed, many stated it had improved their income by increasing food for the household, which allowed them to spend household funds on other items such as school fees, improving the house, or investing in the farm. Thus, while ITIKI didn’t improve pecuniary income, it appears to have improved household welfare for most users. Even so, these gains were limited in size: 64 percent reported their income had only increased “somewhat” while 18 percent reported no increase. One reason may be that all users suffered a severe drought in the most recent season and yields were especially low. Even if yields were better in previous seasons while using ITIKI, the hardship of the most recent season features most saliently in respondents’ minds, causing them to feel that their income has not improved. As most farmers do not keep income records, it is difficult for them to produce an objective comparison of income before and after ITIKI. Thus, their reporting likely depends much more heavily on their perception. Given the drought, most respondents perceived their income as lower than usual at the time of the interview. All respondents except one fell into the “extreme poor” category, indicating the innovation is well targeted to the lowest-income households. However, given the lack of pecuniary income gains, there is no evidence that use of ITIKI moved any users into a higher income category. Gender differences As discussed earlier, uptake of ITIKI was higher among women. However, changes in farming practices among active users differed among women and men, with women being less likely than men to make changes in their crop choices. In particular, men were more likely to stop growing or reduce acreage of crops that perform poorly in drought, such as maize and beans. One reason may be that women bear primary responsibility for feeding their families and are more reticent to take the risk of abandoning familiar crops. SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 37 In contrast, the rates at which women and men added new, drought-resistant crops or increased acreage of these crops based on ITIKI information are not statistically different. This demonstrates that women do have openness to adapting their crop selection to weather conditions. However, since adding new crops is likely perceived as a lower risk choice than stopping cultivation of familiar crops, it may still be that women in the sample are more risk-averse than men. This implication should be considered in ITIKI’s strategy for motivating changes in farmers’ practices based on weather information. Affordability ITIKI’s long-term business plan involves charging users a small fee for updates. To assess the plan’s viability , respondents were probed about their willingness to pay (WTP) for ITIKI updates. Thirty of 50 users (60 percent) reported unwillingness to pay anything for ITIKI updates (Figure 22). Among the 20 users who reported a positive WTP, amounts varied: while two users were willing to pay up to 100 KSh per update, most were not willing to pay more than 40 KSh (Figure 23).8 Including the 60 percent of users unwilling to pay anything, the average WTP was 13 KSh per update. FIGURE 22: USER WILLINGNESS TO PAY FOR UPDATES N=50 40% 60% AMOUNT ITIKI IMPROVED INCOME Willing to Pay Not Willing 8 Some respondents reported WTP per season, rather than per update. For these respondents, the evaluator halved the reported amount under the assumption of two updates per season. Thus, if a respondent reported WTP 200 KSh per season, he was assigned a WTP value of 100 KSh per update. While the ITIKI business plan includes four updates per season, this target has not been reached in recent season (for reasons described elsewhere in this report). Therefore, the evaluator chose to calculate WTP based on two updates per season. 38 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 FIGURE 23: USER WILLINGNESS TO PAY FOR UPDATES (AMOUNT) N=20 USER WILLINGNESS TO PAY FOR UPDATES (AMOUNT) 15 25 10 20 5 100 KSh 50 KSh 40 KSh 20 KSh 10 KSh 5 KSh 2 KSh 1 KSh NUMBER OF USERS 2 9 10 11 16 18 19 20 0 When asked about WTP, many respondents were confused by the prospect of paying for updates. The evaluator clarified that there is no current payment, and the question was whether they would be willing to pay in the future. Although several respondents answered yes, they recanted their answer when asked to provide an estimate of how much they would pay and instead stated they were unwilling to pay anything. The frequency of this occurrence suggests that answers suffer from social desirability bias: respondents may report WTP to please the evaluator, when in reality they are not willing to pay. Therefore, it is important to recognize that WTP estimates may very well be inflated. Furthermore, the high rates at which ITIKI users share updates with other community members (46 of 50 users reported having shared the information) suggest that securing payments from individual users may be a challenge. If users can receive the information from a neighbor, they may be reticent to pay for it themselves. Furthermore, the ambassadors’ role includes in-person follow up on ITIKI updates. The higher usage rates in villages with ambassadors suggest this is an important part of the ITIKI model, but it may also discourage payment by providing an alternative, free route for farmers to receive ITIKI information. Other benefits A few respondents mentioned gaining knowledge from ITIKI that led to changes in their farming practices beyond those addressed elsewhere in this report. Some users remarked that ITIKI helps with the timing of activities beyond the planting stage during the growing season : three users mentioned that ITIKI helped them know when to spray crops with pesticide, one remarked that ITIKI helped her know when to weed, and another noted that ITIKI information helped her know when to harvest. ITIKI sends, on average, three messages throughout the season to keep farmers up to date on weather predictions. For example, a message toward the end of the season may notify farmers that, though the rains have halted, they will return in a week. With this information, farmers can leave their crops in the field in anticipation of more rain. On the other hand, if they were to harvest and lay their SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 39 crops out to dry before the rains had ended, the rain would damage the crops. It is interesting that so few respondents mentioned that ITIKI helped them with timing of activities beyond planting. This may be because ITIKI had issues in recent seasons with mobile telephone service providers, which limited the number of messages sent. The delivery of information to farmers beyond the planting period – both through text messages and through follow-up by ambassadors – is a topic that ITIKI should consider carefully to ensure messages remain impactful throughout the season. A few farmers also mentioned changes in the way they farm. Two said they formerly intercropped but began partitioning crops after ITIKI. Another stated she began using terraces after becoming an ITIKI user. Two users began using a plow rather than tilling by hand because of ITIKI, with one stating that ITIKI taught her to use a plow to reduce water runoff and another indicating she bought a plow because it is difficult to till the land before the rains and a plow helped with early planting.9 In regard to farm inputs, two users said ITIKI taught them how to use manure as fertilizer, while one mentioned that ITIKI taught her to use ash as a natural pesticide. The evaluator probed the local project manager about these results, as provision of technical farming knowledge is not a central tenet of the ITIKI model. He explained that, during sessions in 9 Many users reported using a plow, but only two attributed this to ITIKI. Most users farmed using a plow both before and after ITIKI. 40 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 which ITIKI is introduced to potential users, topics such as use of natural pesticides and fertilizers are sometimes discussed. This is part of a larger discussion on indigenous knowledge related to farming and is intended to help farmers understand indigenous weather indicators and the source of ITIKI predictions. However, topics other than weather indicators seem to be discussed on an ad-hoc basis. Thus, it is logical that so few respondents reported gaining knowledge on farming practices and farming inputs as a benefit of ITIKI. Relatedly, some respondents mentioned being provided technical training by other NGOs or government agricultural extension services. It is worth noting that some respondents may have been unsure of the source of knowledge and may attribute changes to ITIKI that were driven by another NGO’s training activities. Lastly, one respondent felt the information provided by ITIKI was valuable in itself, independently of any farming improvements it motivated. She stated she was happy to have gained the indigenous knowledge of her tribe (the Mbeere) and understand how her people traditionally predicted weather patterns. The sense of pride over retention of this indigenous knowledge is an intangible, yet not insignificant, benefit of ITIKI. SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 41 Difficulties using ITIKI The majority of interviewed ITIKI users (86 percent) reported no difficulties or problems using the innovation. Of the nine respondents reporting they are illiterate, seven felt this was not an impediment to their ITIKI usage. Four indicated their children or grandchildren read them the messages while three received the information orally from an ITIKI ambassador. Four users reported problems receiving messages, although two were among the illiterate respondents, with one acknowledging she may have received messages and not realized it because she cannot read. One cited unreliable cell service and another said that she didn’t always have her phone with her so her children may accidentally delete messages before she reads them. Two users mentioned a period during the last year when they received no messages. This is likely due to ITIKI’s contractual and logistical problems with mobile telephone service providers during the October to December 2018 season and, to a lesser degree, in the March to May 2019 season. These issues are discussed earlier in this report. Lastly, one ITIKI user related it was hard for him to implement ITIKI’s recommendations about early planting because of his farm’s rocky soil. Because of this, he must wait for the rains to soften the soil so he can plant. While this difficulty was cited by only one respondent, it is important to note that farmers without access to a plow may not have the option of early planting, and thus are unable to make use of one of the most important pieces of ITIKI information (the timing of rain onset). The evaluator also probed non-users (individuals who are on the ITIKI user list but unfamiliar with the innovation) for any information about why they had not received messages. Most non-users were able to verify their phone number was correct, although one was incorrect. Another noted that she could not remember signing up for ITIKI and did not have a phone, so she likely had given her son’s number. One non-user mentioned that his poor vision made it possible that he may have received messages without noticing. Another stated he was unable to read and would have been unaware of messages. Lastly, two non-users acknowledged receiving a message in June or July 2019, but it was the first message they remembered receiving. This likely has to do with the logistical problems with mobile service providers which were being resolved at the time of the evaluation. None of the respondents reported any negative impacts stemming from ITIKI. However, two respondents (from two different villages) noted that some people in their communities had doubts about the accuracy of the weather predictions. Both users stated the predictions are sometimes inaccurate. Furthermore, both elaborated that some community members were skeptical of the use of science to predict weather, believing that God alone controls the weather. This belief fed skepticism of the innovation. 42 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 User recommendations for improvement The evaluator asked ITIKI users for ideas on how to improve the innovation. Sixteen users (32 percent) expressed preference for more frequent ITIKI messages throughout the season. When asked how often they would like messages, answers varied and included, “several times during the growing season,” “at least once a month,” and “four times per year.” One respondent clarified she would like three messages during the growing season: one during planting, one during cultivating, and one during harvesting. Founder Muthoni Masinde related to the evaluator that, to date, ITIKI has sent an average of three messages per season, with plans to send four per season. Based on respondents’ answers, however, it appears that users have received less than three messages each season. This may be because of mobile service difficulties or because users do not recognize they have received a message. ITIKI should make users aware that information is sent throughout the season and not just during planting time. This might be achieved by promoting more follow-up by ambassadors. Three users desired more in-person follow-up from ITIKI staff and ambassadors. Two requested seminars on the weather and how to respond to it in farming practices. Another suggested regular farm visits from ITIKI staff to assess whether the changes made in response to ITIKI information had been beneficial. These results indicate some users prefer receiving information through word of mouth, rather than by text message. Furthermore, these results align with findings that respondents in villages with ambassadors are more likely to have made use of ITIKI weather predictions, likely because the information has been reinforced by those ambassadors. These findings suggest that as ITIKI expands, its staff must remain cognizant of how foundational the person-to-person connection is to farmer uptake of the innovation. One ITIKI user expressed a desire for an interactive system to respond to ITIKI messages with questions or feedback and receive a response. Another user felt it would be helpful if ITIKI provided more information, suggesting that instead of simply predicting less-than-normal rainfall, ITIKI send recommendations about the best seed varieties to plant and where to procure them. The local project manager explained that ITIKI’s model is to provide weather information, while it is the role of the government agricultural extension office to provide advice on farming practices. However, weather predictions are shared with that office so its staff may better assist farmers. ITIKI users’ desire for a more interactive experience and more detailed advice suggests that ITIKI should explore deepening its partnership with local agricultural extension officers. Two respondents recommended that ITIKI advertise more. Eleven users asked ITIKI to assist in providing seeds, fertilizer, water, or credit for farm inputs. While this is not part of ITIKI’s model, it highlights the reality that farmers in its region of operation face severe financial constraints in their farming operations. Of these 11, three asked for assistance in irrigation, while one asked for support with activities to provide income during drought, such as livestock. These responses shed light on the hardship experienced by ITIKI users due to less-than-normal rainfall in recent years. SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 43 CONCLUSION 44 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 The ITIKI drought prediction tool aims to help farmers adjust to less-than-normal rainfall by providing information via text message regarding the timing and amount of rains. Through a survey of 62 registered ITIKI users (50 of whom were active users of the innovation), the evaluator assessed ITIKI’s impact in regard to its objective. In particular, the evaluation probed users’ perceptions of the innovation and its effect on farming practices, crop yields, and income. ITIKI users had a very positive perception of the innovation, with all 50 active users reporting they would recommend it to others. All except one user described a change in their farming practices driven by ITIKI, with early planting as the most common adjustment. The majority of users also reported having changed their crop choices in favor of more drought-resistant crops or crop varieties. Even with severe drought in the region in recent seasons, many felt ITIKI information had improved their crop yields compared to what yields would have been without the intelligence. There were no significant impacts on income; however, this is largely because most respondents are subsistence farmers who sell very little of their harvest. Regardless, many users felt ITIKI had improved their income because it had increased food for their households, thus freeing funds to spend on non-food items. Approximately 20 percent of respondents were unfamiliar with the innovation, despite being on the user list. This was more common among men, older users, users in Mwea-area villages, and users in villages without assigned ambassadors. The geographical differences in familiarity with the innovation suggest that the person-to-person recruitment strategy and ambassador follow-up is crucial for user uptake. Lastly, it is notable that the majority of ITIKI users reported unwillingness to pay for the innovation. This, combined with the finding that users share information with one another, may pose challenges for ITIKI’s future plans, which include payment by users. SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 45 ANNEX I 46 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 FARMER INFORMATION NAME AGE DATE TIME GROUP INTERVIEW? o Yes o No GROUP INTERVIEW NOTES HOW MANY FAMILY MEMBERS LIVE WITH YOU? GENDER o Male o Female WHAT IS YOUR PRIMARY OCCUPATION? o Farming o Wage Labor o Seasonal Migrant Labor o Small Enterprise o Other: DO YOU HAVE ANOTHER OCCUPATION? o Farming o Wage Labor o Seasonal Migrant Labor o Small Enterprise o Other: SIZE OF FARM (ACRES) NAME OF VILLAGE HOW MUCH LAND DO YOU OWN? HOW LARGE IS YOUR FARM/PLOT? o Large o Medium o Small o Very Small SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 47 HOW MUCH IS LAND RENT? OTHER LAND NOTES HOW LONG HAVE YOU BEEN USING ITIKI? DID YOU PARTICIPATE IN AGRICULTURAL ACTIVITIES THIS YEAR? o Yes o No HOW MANY MONTHS IS THE PRIMARY GROWING SEASON? HOW MANY TIMES DO YOU HARVEST PER YEAR? FARM INFORMATION WHAT CROPS DO YOU GROW AS A RESULT OF THE INNOVATION? LIST FROM MOST IMPORTANT TO LEAST IMPORTANT: 1. 2. 3. DID THE MOST IMPORTANT CROP BENEFIT FROM ITIKI? o Yes o No DID THE SECOND MOST IMPORTANT CROP BENEFIT FROM ITIKI? o Yes o No DID THE THIRD MOST IMPORTANT CROP BENEFIT FROM ITIKI? o Yes o No WHAT IS THE WATER SOURCE FOR YOUR IRRIGATION OF CROPS? o Own pond o River o Groundwater o Innovation Source o Other: 48 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 WHAT IS YOUR METHOD OF IRRIGATION? o Drip feed o Flooding o Hand watering o Rainfed o Other: HOW MUCH HAS YOUR WATER USAGE CHANGED SINCE USING ITIKI, IF AT ALL? USING ITIKI HAS YOUR ACCESS TO WATER: o Had no change o Improved o Fundamentally improved (Improved a lot) o Other: PREVIOUSLY GROWN CROPS: DID YOUR FARM PRODUCE DIFFERENT CROPS IN THE PAST THAT ARE NO LONGER GROWN HERE? IF SO, WHICH ONES? MASS OF PRODUCE: WHAT YIELDS DID YOU HAVE FOR EACH CROP YOU MENTIONED? MASS OF PRODUCE 2: WHAT YIELDS DID YOU HAVE FOR YOUR CROPS BEFORE USING ITIKI? USING ITIKI HAVE YOU, FOR EACH CROP: o Used more water o Had no change in water use o Used less water o Other: USING ITIKI HAVE YOUR CROP YIELDS (ASK FOR EACH CROP): o Declined o Remained the same o Increased o Substantially increased SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 49 IS THERE A DIFFERENCE IN THE SURVIVAL RATES OF YOUR CROPS DUE TO ITIKI? o Yes o No HOW MUCH OF YOUR PRODUCE DID YOU CONSUME IN YOUR HOUSEHOLD? (PERCENTAGE – NOTE IF DIFFERENT FOR EACH CROP) HOW MUCH OF EACH OF THE FOLLOWING INPUTS DID YOU USE BEFORE ITIKI? FERTILIZER (KG) PESTICIDE (KG) HERBICIDE (L) CHARCOAL (KG) WATER (TOTAL) LABOR (DAYS) OTHER HOW MUCH DID YOU SPEND ON EACH OF THE FOLLOWING INPUTS BEFORE ITIKI? FERTILIZER (KG) PESTICIDE (KG) HERBICIDE (L) CHARCOAL (KG) WATER (TOTAL) LABOR (DAYS) OTHER HOW MUCH OF EACH OF THE FOLLOWING INPUTS DO YOU USE AFTER ITIKI? FERTILIZER (KG) PESTICIDE (KG) HERBICIDE (L) CHARCOAL (KG) WATER (TOTAL) LABOR (DAYS) OTHER HOW MUCH DID YOU SPEND ON THE FOLLOWING INPUTS AFTER ITIKI? FERTILIZER (KG) PESTICIDE (KG) HERBICIDE (L) CHARCOAL (KG) WATER (TOTAL) LABOR (DAYS) OTHER 50 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 HOW MUCH DID YOU SPEND ON EQUIPMENT BEFORE AND AFTER ITIKI? HOW MUCH DID YOU SPEND ON TRANSPORT AND STORAGE BEFORE AND AFTER ITIKI? DO YOU HAVE PROBLEMS FINDING A MARKET TO SELL YOUR CROPS IN? o Yes o No PLEASE EXPLAIN. DO YOU HAVE PROBLEMS GETTING YOUR CROPS TO THE MARKET? o Yes o No PLEASE EXPLAIN. OTHER FARM NOTES (OPTIONAL). INCOME AND EXPENDITURES WHAT IS YOUR ANNUAL HOUSEHOLD INCOME? HOW MUCH INCOME DID YOU MAKE BEFORE ITIKI? AFTER ITIKI? HAS ITIKI IMPROVED YOUR FAMILY INCOME? WHAT PERCENTAGE OF YOUR INCOME DO YOU GET FROM NON-FARM SOURCES? HOW MUCH PRODUCE DID YOU SELL FOR EACH OF YOUR CROPS IN THE LAST SEASON AND THE LAST YEAR? WHAT IS THE PRICE PER KILO YOU RECEIVED FOR EACH OF YOUR CROPS FOR THE LAST SEASON? SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 51 USING ITIKI HAS YOUR ACCESS TO CREDIT: o Not improved o Improved o Improved and have been able to repay over a short period HOW DO YOU CURRENTLY FINANCE AGRICULTURAL ACTIVITIES? o Own savings o Credit and savings scheme o Other credit HOW MUCH DO YOU PAY FOR ITIKI? HOW MUCH ARE YOU WILLING TO PAY FOR ITIKI? o Nothing o ITIKI is free o The same as what I pay now o 50% less o 50% more o Other: HOW HAVE YOU SPENT YOUR NEW INCOME? o N/A (if no new income) o Send children to school or keep children in school o Social functions (like weddings) o Investment in farming o Improving house o Other: OTHER INCOME NOTES (OPTIONAL) 52 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 PERCEPTIONS OF ITIKI WILL YOU USE ITIKI IN THE FUTURE (5 TO 10 YEARS)? o Yes o No WHY? HOW, IF AT ALL, HAVE YOU CHANGED YOUR FARMING PRACTICES DUE TO ITIKI? o No change o Introduced new crops o Changed irrigation system o Reduced water usage o It helps me decide when to plant o It helps me decide which crops to plant HAVE YOU FACED ANY DIFFICULTIES OR PROBLEMS USING ITIKI? o Yes o No HOW CAN ITIKI BE IMPROVED? HOW DID YOU HEAR ABOUT ITIKI? o Wealthy farmer o Neighbor o Innovation personnel o Extension worker o Other: WHAT FACTORS INFLUENCED YOU TO TRY ITIKI? o Demonstration from neighbor’s farm o Innovation is free from extension services o No alternative water source o Other: DO YOU SHARE YOUR KNOWLEDGE SKILLS FROM ITIKI WITH OTHERS? o Yes o No IF SO, HOW? WHAT DO YOU FEEL ARE THE BENEFITS OF ITIKI? SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 53 HAVE YOU HEARD ABOUT CLIMATIC VARIATION? HAVE CHANGES IN RAINFALL OR TEMPERATURE AFFECTED YOUR FARMING PRACTICES OR CROP YIELDS COMPARED TO YOUR HISTORICAL RAINY/DRY SEASON PERIODS? o Yes o No PLEASE SPECIFY HOW. HOW HAS ITIKI HELPED YOU? PLEASE RANK THE TOP 3 AND EXPLAIN POSITIVES/NEGATIVES. Makes water reusable Helps women farmers as well as men They made a special effort to include women farmers Helps in producing more of our most important crop Increases my yield through timely forecasts Helps by lowering cost of inputs Improves health and strength of livestock Helps reduce labor Reduces crop wastage Helps me decide when to plant Helps me decide which crops to plant Other: WOULD YOU RECOMMEND ITIKI? o No o Yes o Yes, would strongly recommend ARE THERE NEGATIVE IMPACTS FROM ITIKI IN THE COMMUNITY? o Yes o No PLEASE EXPLAIN IF YES. 54 SECURING WATER FOR FOOD: ITIKI PERFORMANCE EVALUATION | AUGUST 2019 IF THERE HAVE BEEN ANY NEGATIVE IMPACTS, HAVE EFFORTS BEEN MADE TO RESOLVE THEM? o Yes o No EXPLAIN. OTHER INCOME/POVERTY NOTES GENDER OBSERVATIONS QUESTIONS/REQUESTS OTHER NOTES Securing Water for Food has sourced and invested in a portfolio of innovative solutions that aim to help farmers use water more efficiently and effectively, improve water storage for lean times, and remove salt from water to make more food. Our cohort of innovators are helping people in 35 low-resource countries with tools they need to produce more food with less water. To learn more about Securing Water for Food, visit www.securingwaterforfood.org.