SPRING Impact Evaluation iSocial Baseline Report November 2018 Sharmin Akter and her child Department for International Development SPRING Monitoring and Evaluation PO 7117 Partners • SRGB Authors • Eileen Lambourne • Gordon Freer • Panayiota Kastritis • Florian Poli Contributions • Alexandra Deniel November 2018 This document has been approved for submission by Coffey’s Project Director, based on a review of satisfactory adherence to our policies on: • Quality management • HSSE and risk management • Financial management and Value for Money (VfM) • Personnel recruitment and management • Performance Management and Monitoring and Evaluation (M&E) Heidi Ober, Programme Director Signature: SPRING Impact Evaluation Baseline Report: iSocial November 2018 IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 Abbreviations and Acronyms BMI Body Mass Index BPE Business Performance Evaluation DFAT Australian Department of Foreign Affairs and Trade DFID UK Department for International Development FGD Focus Group Discussion GSP Girl Safety Protocols IE Impact Evaluation IGA Income Generating Activity IP Implementing Partner KII Key Informant Interview KK Kishori Kallyani M&E Monitoring and Evaluation NGO Non-Government Organisation PPI Poverty Probability Index PQ-LES-Q Paediatric Quality of Life Enjoyment and Satisfaction Questionnaire PSM Propensity Score Matching RCT Randomised Controlled Trial SMS Short Message Service SRGB SRG Bangladesh SRH Sexual and Reproductive Health ToC Theory of Change USAID United States Agency for International Development WASH Water Sanitation and Health IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 Contents Executive Summary i Section 1 Context 1 1.1 Purpose and structure of this document 1 1.2 Overview of the Impact Evaluation 1 1.3 Introduction to iSocial 1 1.4 Evaluation design 3 1.5 Evaluation questions 3 1.6 Evaluation data collection tools 4 1.7 Sampling strategy 5 1.8 Methodological limitations and fieldwork issues 5 1.9 Data analysis and matching 6 Section 2 Findings 7 2.1 Profile of intervention (customers) and control (non-customers) 7 2.2 Well-being 8 2.3 Learning 15 2.4 Kishori Kallyani Service 23 2.5 Kishori Kallyani Intervention 29 2.6 Adolescent girl Kishori Kallyanis in the value chain 32 Section 3 Next Steps 36 Annexes 38 Annex A: iSocial Theory of Change 38 Annex B iSocial Baseline Survey 39 Annex C: Tree of Life 67 Annex D: iSocial Focus Group Discussion Guide – School Girls 68 Annex E: iSocial Kishori Kallyani Data Sheet 72 Annex F: iSocial Kishori Kallyani Key Information Interview Guide 75 IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 i Executive Summary iSocial is a social enterprise offering a last-mile distribution micro-franchise to women in rural communities. The women micro-franchisees (‘Kallyanis’) provide rural communities with access to information, products and services (both directly and on behalf of a third party) at their doorstep. Through SPRING involvement, iSocial has launched a service targeting adolescent girls through a specialist Kishori Kallyani (KK) service. The KK is envisaged as an ally and informal counsel to the adolescent girl, working alongside the girl’s guardian to help bridge the gap in information on SRH, mental health and well-being, as well as training and career opportunities. This baseline data collection encompasses face to face interviews with 745 girls that had transacted with the KK (KK customers) and 180 girls from outside the KK catchment area (non–customers); six focus group discussions (FGDs) with adolescent girls in schools within the KK catchment area; and 10 Key Informant Interviews (KIIs) with KKs. iSocial first launched their prototype in August 2017 and was due to launch an online digital platform in December 2017. To allow sufficient time for KKs to initiate the intervention and to facilitate identification of beneficiaries and KK customers, the data collection was conducted after iSocial’s planned launch of their Digital Platform in April 2018. Key findings include: Girls’ Well-being • The vast majority of KK customers and non-customers alike (99%) are attending school and most are in Secondary School (84% of KK customers v 72% of non-customers). Most girls have not undertaken paid employment in the last 12 months (84% v 76%, respectively). The likelihood to work increases with age, only 10% of KK-customers under 13 years of age having undertaken paid work, compared with 28% of girls 17 years and older. • A fifth of KK customers and non-customers alike are married, with the average age of marriage of 15 years old. Of those married, 52% of KK customers and 33% non-customers have at least one child. KK customers and non-customers alike had their first child when they were 16 years old on, average. • Quality of life and mental well-being scores are consistent across KK customers and non-customers. Girls rate their own health, well-being and feelings as good or almost good. Girls perceive aspects of getting things done, mood or feelings and health as areas where they are less content. • Mothers act as key support and confidant; and KK customers and non-customers alike have little awareness of external support options. • KK customers access a wider network of support than non-customers, in particular accessing members of their family and friends. • KK Customers make little reference to KKs as a source of counsel and support at this early stage. Girls current awareness, knowledge and learning • KK customers more frequently cited the KK as a source of information in physical health and biometrics1 than SRH which suggests the KK has had more impact in this area. • At this early stage, it seems both customers and non-customers have similar levels of knowledge with regards to SRH. • Little reference to the KK as a source of information on career and training is consistent with feedback from the KK and suggests this area of intervention has yet to be delivered. Transacting with the KK • Most customers know of the KK through her visiting their home, and only around a quarter know her through social or community networks. Nearly half of KK customers (48%) stated the KK visited her house every one or two weeks, which is higher than the iSocial business model anticipated, though this may be a necessary part of establishing her business. A third of KK customers (35%) stated the KK visited monthly KKs are nearly as likely to visit the mother as the adolescent girl. 1 Including: weight, BMI, blood type, anaemia IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 ii • By their own account, KK customers mainly buy Orsaline and sanitary pads compared to stationary, undergarments, Internet services and top-up cards. The most frequently cited reason for not buying from the KK is that she is not available when needed. • While some KK customers were aware of the third party referral services (ranging from 43% aware of Aponjon; 21% of Hello Doctor; 10% Ajkerdeal and 5% of Green Delta Nibedita), very few had heard about it from the KK. • Nearly all customers (94%) stated they trusted the KK and the information she shared, and eight in ten felt their behaviour had changed as a result of information shared. KK Account of the Intervention (from 10 KIIs) • KKs are reaching far fewer households with more repeat visits than the iSocial business model assumes and this may have bearing on SPRING logframe assumptions of beneficiary reach.2 • As reflected in customer survey data, while KKs’ target the adolescent girl, they frequently transact with the mother on behalf of the girl. KKs also transact with others in the adolescent girl household, including other women and even boys who require the products the KK sells. • As reflected in the customer survey data, information sharing has concentrated on the areas of SRH, sanitation & nutrition and to a far lesser extent, mental health and career & training. • In addition to household visits, KKs have engaged in yard meetings (an organised gathering in someone’s house) and school activation sessions to raise awareness in selected topics, such as SRH, sanitation & nutrition. The yard meetings and school’s activation sessions also provide the KK with an opportunity to raise her profile and sell her products and services. • However, far fewer school activation sessions than yard meetings have been delivered. KK feedback suggests that obtaining necessary permission to gain access to schools is proving more difficult than anticipated. iSocial is responsible for obtaining permission on behalf of the KKs and this lack of progress can be expected to impact on the KKs business growth. Adolescent girls and other KKs in the value chain3 • KKs enjoy working as Kishori sales agents and talk of improved self-confidence, enjoyment of information sharing and of improving customer well-being. Adolescent girl KKs have also learned more about their own physical, menstrual and mental health. All KII participants had worked before becoming a KK so this was not their first income earning experience. • Despite enjoying the job, all KKs, including adolescent girl KKs, do not feel they are earning enough for the effort they make to deliver their business. Many KKs, and all adolescent girl KKs, question the long-term viability of their business. • KKs invest a lot of time to build customer relationships; yet as they are selling items of low value, or unit price, their commission-earnings are low. It may be that as the KK establishes her business, she will require less time to sell her products and services, making the micro-franchise more attractive. • Digitisation of the service appears slow in taking off and currently imposes an additional layer of reporting activity on the KK instead of helping her to run her business. • At the point of baseline data collection, some aspects of the prototype did not seem to have reached the KK – for example the hotline, call centre, paid activation session. These additional prototype services are important part of delivering the envisaged full suite of interventions. 2 This is consistent with iSocial KPI 2017 report. The reports reflect that KK reach falls far short of their anticipated reach. While the total number of KK customers is not specifically detailed. the 2017 KPI report suggests each of the 11active KKs were reaching on average 25 adolescent girl customers in the month of May 17 and 56 in June 17. This is lower than iSocial’s business model assumption that each KK will visit 15-20 households per day and approximately 300 in a month from a catchment of 1,200 households. The 2018 KPI report suggests within the context of a notable increase in the number of KKs, this has fallen to as low as 30 – 40 transactions per KK per month. At the time of writing this baseline report we are in the process of clarifying some of these trends with the business. 3 iSocial does not purposively target adolescent girls to become KKs and by March 2018 had only three active KKS that were adolescent girls. KIIs were undertaken with all three adolescent girl KKs and a further seven KIIs were undertaken with a representative spread of the other active KKs. To improve robustness of findings and provide anonymity to the adolescent girls, their feedback is presented in the context of feedback from KKs overall IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 1 1 Context 1.1 Purpose and structure of this document This document presents the initial findings from the baseline research for iSocial. Section 1 provides an overview of the SPRING impact evaluation, introduces the iSocial business, and describes baseline data analysis methods. Section 2 presents the findings in line with the SPRING programme Theory of Change, mapping the findings across the four impact pillars of: learning, earning, saving and well-being. Section 3 outlines the next stages in the evaluation. The approach to the iSocial evaluation, including the design, data collection methodology, data sources, sampling strategy, and data tools, is detailed in the Fieldwork Report submitted to DFID on 4 August 2018 4 . The data collection tools are also appended to this report (see Annex B – Annex F). 1.2 Overview of the Impact Evaluation SPRING is a five-year accelerator programme that supports business ventures to develop products and services to help girls earn, learn, save, keep safe and experience well-being. SPRING envisages that their successful engagement with businesses will lead to a broader shift in markets that enable girls and their communities to contribute to ending the cycle of poverty. As one of the three components of the overall SPRING evaluation, the Impact Evaluation (IE) provides evidence of the overall effects of SPRING in terms of improvements in economic and social outcomes for girls as a result of using products and services delivered by SPRING businesses. While other components focus on how well the programme works and what works (or does not work) well, the IE contributes evidence to help understand the effect of SPRING-funded business activities on the socio-economic circumstances of the adolescent girls. Over the lifetime of SPRING, the Evaluation Team will conduct a total of eight IEs, two per cohort. IEs generally adopt a quasi-experimental design approach. This has been selected over an RCT approach as it accommodates the limitations imposed where interventions are administered by businesses whose priority is to establish a profitable prototype in an uncertain world. The quasi-experimental design approach offers the greatest adaptability and cost-efficiency by taking into account the adaptive and iterative nature of the SPRING prototypes, as well as the diversity of SPRING businesses considered for the evaluation. The adopted quasi-experimental approach is also a theory-based, mixed methods evaluation. The approach builds on the causal chain, or theory of change, of the individual businesses. This includes a consideration of changes in the business theory of change over the lifetime of the programme to reflect how both the intervention has evolved. The selection of the right businesses to include in the IE is crucial to ensuring that the impacts of the programme can be measured. Selection is done in consultation with the Implementing Partner (IP) to include businesses that are likely to reach their scale targets within the programme’s timeframes, and which have a sufficiently deep impact on girls to ensure that we are able to trace and detect impact. The selection criteria also seek to maximise potential for learning for a broad range of future interventions. To achieve this, the businesses selected for the IEs come from a range of countries, market sectors and business models (including those that impact girls through different channels, e.g. as direct users, indirect users or through participation in the value chain). Following an evaluability assessment of Cohort 2 businesses, and in consultation with the IP, iSocial and Paritran were selected for the IEs for Cohort 2. This report presents the findings from the baseline research for iSocial. 1.3 Introduction to iSocial iSocial is a social enterprise offering a last-mile distribution micro-franchise to women in rural communities. The women micro-franchisees (‘Kallyanis’) provide rural communities with access to information, products and services (both directly and on behalf of a third party) at their doorstep. Each Kallyani is equipped with a smartphone-tablet, selected health test equipment5 , and a Wi-Fi hotspot via USB stick. 4 iSocial Baseline Impact Evaluation: Fieldwork Report July 2018; Coffey International Development Ltd. 5 This includes a set of scales for weighing girls and calculating their body mass index (BMI), as well as blood, urine and anaemia tests. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 2 When iSocial joined SPRING, the business offered a network of generalist Kallyanis (then called Infoladies) who were trained to help customers with a variety of services, from administering blood sugar checks to offering health and legal advice and selling consumer products. The Infoladies covered their last mile distribution network using bicycles purchased with the aid of a personal loan from the Bangladeshi government. Through SPRING, iSocial received support in: conducting girl research to inform their prototype design; mentoring in service, branding and digital design; investor readiness support, including a Smart Impact Capital course and £73,000 of PricewaterhouseCoopers (PwC) pro bono investor readiness support; £62,400 in prototype development funding; a legal review of their trademark application; and in-country and international business mentoring. Following Bootcamp 2, iSocial focused on relaunching its service offer, shifting from the generalist Kallyani to a specialist Kallyani model across four areas. The new service included: Health Kallyanis, Information Kallyanis, Agricultural Kallyanis and Kishori Kallyanis (adolescent girls). The Kishori Kallyani model is the prototype that was developed and launched through the SPRING programme, and is attributed to SPRING. iSocial launched the Kishori Kallyani (KK) prototype in August 2017, approximately 6 months before data collection. The KKs provide a range of services, including: • Sale of products tailored to adolescent girls (i.e. a tailored basket of goods developed through SPRING, including items such as sanitary pads and soaps); • Delivery of products bought on e-commerce sites (last mile distribution); • Informal counsel to girls while visiting them in their home or at school; • Activation / awareness-raising sessions at schools or at the KK’s home (or where necessary, the home of a relative/someone from iSocial network). Activation sessions involve a short talk to inform girls about relevant issues such as sexual and reproductive health (SRH) and mental health, and may include providing details on professional support services available to adolescent girls. The sessions also provide an opportunity for KKs to sell their products; and • Informing and connecting adolescent girls with different (career) training opportunities, either through schools or door-to-door visits. This is a service funded by the government or a training provider, which is free to the adolescent girl, but where KKs are paid. The activation and connecting services are provided free of charge. Depending on the KK’s catchment area, the same girl may interact with the KK at home and at school (i.e. they are not mutually exclusive). While only some girls reached by the KK may choose to buy products from her, all girls reached by the KK receive free information services. The KK earns income through commission on products sold and delivered and connection services delivered. She also earns from activation sessions that are funded by an institution or third party6 . Adolescent girls do not pay for the activation or connecting services. iSocial has mapped a theory of change (ToC) for their KK services which centres on the premise that adolescent girls lack access to information and counsel in the areas of SRH, mental health and well-being; as well as in training and career opportunities. The KK is envisaged as an ally and informal counsel to the adolescent girl, working alongside the guardian to help bridge the gap in information and build more informed and understanding household relationships. The KK ToC is detailed in Annex A. KK beneficiaries include: • Girls (or their households) who have purchased products from the KK (including purchase from the iSocial KK basket of goods, third party sales of products such as Green Delta insurance; or delivery such as on behalf of Ajkerdeal (the Bangladesh equivalent of Amazon); • Girls (or their households) who have purchased services from the KK (including health tests, third party sales of health services, such as Aponjon (SMS health service for expectant and new mothers) or Hello Doctor; 6 At baseline, no income earning activation sessions were in place. Activation was done instead in the interest of growing the KKs’ business. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 3 • Girls who the KK has visited at home (door-to-door visit) and with whom information has been shared, where the KK connects the girl to referral services (e.g. Marie Stopes SRH services) and/ or training and career opportunities (where the KK earns commission)7 ; • Girls who participate in a yard or school activation session, either held at schools or at someone’s home; and • Potentially, girls who the KK has visited in their home and shared information with but who have not transacted with the KK in any way8 . Adolescent girls have varying degrees of interaction with the KK, depending on whether they: purchase products or services directly from the KK; buy third party products and services and have goods delivered by her; receive information both at home and at school; or choose to take up the opportunity for additional information at the KK’s home. Each KK has a catchment area of 1,200 households and 24 schools, and is expected to visit 300 households (roughly 15-20 per day) and two schools each month. No target for yard meetings was specified. As an entrepreneur, the KK sells products and services to the adolescent girl, alongside the provision of information on nutrition, SRH, mental health and training and career opportunities. The KKs carry all products and equipment with them in a shoulder bag provided by iSocial. The impact on the girl is envisaged to include improved knowledge and confidence through the support of the KK, and through this, better health and life outcomes. In addition to targeting girls as end users, iSocial estimates that 10-20% of all KKs will be adolescent girls9 and will benefit through participation in the value chain. This translates to around 5-10 adolescent girls who work as KKs. 1.4 Evaluation design The iSocial impact evaluation tests if benefits delivered via the KK model and the digitisation of the iSocial business can be attributed to SPRING. It also hypotheses that the impact to all adolescent girls reached by the KK are attributable to SPRING. Finally, the impact evaluation tests if iSocial products and services have an attributable impact on adolescent girls by assessing changes to beneficiary knowledge of SRH, mental health, awareness/take up of counselling and further (career) training opportunities, and access to relevant (e.g. feminine hygiene) products before and after receiving KK services. The iSocial Fieldwork Report submitted to DFID on 4 August 2018 includes a detailed account of the evaluation design, methodology, fieldwork and data tools associated with this baseline data collection. iSocial has provided support and input throughout the baseline research: they have assisted us in the design of data collection tools, facilitated enumerator training and supported us during fieldwork. Below we outline key features. 1.5 Evaluation questions The iSocial Impact Evaluation is guided by the overall impact evaluation questions from the SPRING M&E Evaluation Framework. These questions will be applicable in measuring the impact of iSocial’s prototype on adolescent girls who are customers of the KKs. The evaluation questions are indicated in Table 1 below (on the left column), with the applicable sub-questions, and judgement criteria in the right column. Evaluation Question Sub-Questions E2. To what extent have girls improved their safety and well￾being as a result of accessing products, • Learning: Have girls improved their SRH and mental health knowledge? Has their awareness of nutrition and WASH improved? How has their knowledge of career and training opportunities changed? Have they created a career path as a result of the information? If so, what is this career path? 7 iSocial also has intentions of establishing a customer call centre, where customers call and order products to be delivered by the Kallyani and their SPRING ToC makes reference to the provision of an anonymous hotline service to provide girls with additional support and counsel. At baseline these had not yet been established. 8 The pilot study (April 2018) did not identify any such cases. 9 In the Infolady model (the predecessor business model of the KK), approximately 20% of Infoladies were adolescent girls. However, with changes to the KK model including the investment of $250 upfront, it is not clear what proportion of KKs will be adolescent girls. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 4 services or business opportunities provided by SPRING businesses? • Well-being: Have girls participated in activation / awareness sessions about their SRH and mental health? Are they aware of the free mental health services accessible through KKs? Are the KKs a source of informal counsel? Have other health services been displaced as a result of the KKs? Are girl customers accessing more health services through the KKs, or have they just switched service provider? Has improved nutrition and WASH awareness led to a change in behaviour for the girl or household? Have girls delayed early marriage? Have parents’ / guardian’s attitude to early marriage changed as a result of improved understanding of health risks? E1. To what extent have girls improved their earning potential (including employability skills) and savings as a result of accessing products, services or business opportunities provided by SPRING businesses? • Empowerment: Have girls’ aspirations changed as a result of access to information on health and career opportunities? Has girls’ agency and ability to achieve their aspirations improved? Have girls been able to use information on career and training opportunities to take up additional training, create a career path or find a job? Are girls earning or saving more as a result of access to KK services? E3. What have we learned about girls as end consumers or beneficiaries in the value chain? • Girl as consumers: Have girls taken up the KK products and services? Have their behaviours and choices changed? Do they access and/ or buy the same products and services elsewhere? Why? • Unintended consequences: Has involvement with the KK had any unintended positive or negative consequences (e.g. sharing of knowledge from daughter to mother, exclusion of adolescent boys leading to jealousy)? • Contribution questions: The surveys will be designed to probe at external factors (e.g. government and NGO initiatives) that may have contributed to the set outcomes. The impact evaluation will also seek to measure the impact on adolescent girls who take up work as KKs: • Learning: Have the KKs improved their micro-franchise and entrepreneurship skills? • Earning: Have the KKs improved their earnings? What do they spend their earnings on? Do they make decisions on how to spend their income? Are they saving? • Well-being: Do KKs enjoy better health and well-being than before they started their micro-franchise? • Girls in the value chain: How have adolescent girls’ work preferences and behaviours changed? Are there differences between adolescent girl KKs and older KKs in delivering information-sharing activities and establishing and growing their business? Has involvement led to any positive or negative unintended consequences? 1.6 Evaluation data collection tools To measure and track iSocial’s outcome and impact indicators, we have designed a set of complementary data collection tools. The core tool is a face-to-face survey that will be administered in-home to adolescent girl customers of the KK (treatment group). The survey will also be administered to girls who are eligible but not potential customers of the Kallyani (comparison group). The adolescent girls’ guardian will be present where necessary. At baseline and endline, the surveys will be supplemented with a set of qualitative tools to facilitate: IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 5 • Six Focus Group Discussions (FGDs) with girls that have benefited from KK activation/information sessions; and • Ten Key Informant Interviews (KIIs) with KKs, which include an additional Data Sheet, to be completed by the KK ahead of the interview. Similar to the survey, the qualitative tools gather data to measure and track changes in girls’ knowledge and behaviour with regards to SRH, mental health and career and training opportunities. The tools place particular emphasis on where girls get their information from and the role of the KK as informal counsel. 1.7 Sampling Strategy Separate sampling strategies were used for each of the baseline evaluation tools: 1.7.1 Quantitative Surveys: Treatment and Control The sampling strategy for the questionnaire was designed to capture information from a representative sample of KK customers in the four districts that iSocial is active in. Sampling also captures information from non-customers in three districts that iSocial was not active in at the time of baseline data collection, which formed the comparison group. The Evaluation Team worked with iSocial to identify suitable districts that KKs would not become active in during the lifetime of the evaluation. The treatment group is made up of 745 girls who are current KK customers. iSocial provided a list of KKs working in the four districts of Jessore, Bogra, Jhenaidah and Kushtia10 and the number of their registered customers, amounting to about 2,000 adolescent girls in total. To limit fieldwork costs, KKs who had less than 20 customers registered were removed from the selection process. To limit a clustering effect, the number of customer interviews per KK was capped at 40. Using this method systematically, from an initial list of 46 KKs, 19 KKs were identified as eligible and included in the sample. This corresponded to 745 adolescent girl customers. The treatment group combines the districts of Kushtia and Jhenaidah as the KK services had only recently been introduced to these districts and it was understood from iSocial that customer numbers were lower than in the more established districts of Bogra and Jessore. How customers were selected in practice and their distribution is elaborated further in sub-section 2.3.1 of the fieldwork report. The control group is formed of 180 adolescent girls who are not KK customers. Sixty girls were interviewed in each of the four districts within upazilas (administrative regions in Bangladesh) that are not part of iSocial’s current or planned areas of intervention. To limit a clustering effect, a maximum of 20 interviews per village were conducted (corresponding to a minimum of three villages per district). Households were selected through random walks from the field teams, using questions to screen out households who did not have any girl aged 10-19 or who did not wish to take part in the survey. 1.7.2 Focus Group Discussions with beneficiaries The FGD sampling strategy was designed to obtain feedback from a range of girls in schools within the KKs’ catchment area and who had or would be receiving school activation (or information-sharing) sessions from the KKs over the evaluation period. Six schools were sampled, stratified by district and girls age: two FGDs were conducted in each iSocial district; and two with each age band of 10-13 years of age; 14-16 years of age; 17-19 years of age 1.7.3 Key Informant Interviews with Kishori Kallyanis iSocial reported to have 52 trained KKs working across Bogra, Jessore, Jhenaidah and Kushtia. iSocial estimated that 10-20% of all KKs would be adolescent girls11, equating to approximately 5-10 KKs. On this basis, we sought to complete 10 KIIs and interview all adolescent girl KKs. If we did not manage to interview 10 adolescent girl KKs, the remaining KIIs would be made up of older KKs using random stratification by district and age. 11 In the Infolady model (the predecessor business model of DNET/iSocial sales agents, that was replaced by the KK), approximately 20% of Infoladies were adolescent girls. However, with changes to the KK model including the investment of $250 from the KK upfront, it is not clear what proportion of KKs will be adolescent girls. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 6 1.8 Methodological limitations and fieldwork issues Methodological limitations: The key limitations of the evaluation methodology were design, internal and external validity and sampling and non-sampling errors. Fewer KKs: iSocial anticipated they would have over 50 KKs from which to sample, but slower progress than anticipated activating the KKs meant only 19 were deemed eligible to participate in the quantitative exercise. Lack of progress with digitisation: Only a handful of the KKs were using the online platform and those that were did so in addition to their pen and paper records of sales. This meant that accurate customer records and real-time data was not available. Lack of an iSocial customer database: Building on the point above, the main challenge during data collection was the absence of a customer list for each KK or aggregated database of all iSocial customers. The lack of a customer database posed challenges to the sampling selection as it was not possible to randomise interviewees, either from an exhaustive customer list or from the level of the KK. To overcome this challenge, the field teams had to rely on the KKs to identify and contact their customers. The reliance on the KKs to provide customer information could introduce bias, in that the KKs likely provided girls that they visit more frequently or who buy products more often. This has, however, potentially reinforced the power of the treatment group in that selected girls are more likely to be actually benefitting from the KK intervention. Respondent availability: Securing adequate time for respondents to complete the questionnaire proved challenging, as girls were often either in school or engaged in after school tutoring. In most cases, field investigators first had to make an appointment with girls which put a strain on scheduling fieldwork. Distance of interviewees: Respondents were often spread throughout an entire village. Traversing the village to secure all interviews proved to be very time-consuming. 1.9 Data Analysis and Matching The data was cleaned, processed and analysed using the statistical software Stata. Datasets were received in .sav format and converted in .dta format in Stata. Data was checked and screened for data errors and outliers. No outliers were found; hence the full sample was kept in the final dataset. However, for variables with low samples, arithmetic mean can be sensitive to extreme values, such as the age girls wish to get married, which is why the median was used making it more robust. To calculate statistical differences between the customer and non-customer samples, statistical tests were run through Stata. These included the Student’s t-test for binary and continuous variables and the Wilcoxon-Mann￾Whitney test for categorical variables. As detailed in section 2.1, average age, prevalence of disability, girl's highest level of education, current enrolment status (and school phase), as well as the household size and the proportion of girls married are statistically similar across the two groups12 . However, the number of years girls had attended school, their religion and average PPI scores were significantly different at the 5% level. The latter suggests that non-customers may be better-off than customers, rather than the other way around. Overall, the variables of difference (religion and years of school) are deemed less important than the variables that are similar (social, economic and demographic variables) across the two groups. Propensity score matching (PSM) using part or all of these variables did not lead to the exclusion of any girl from the sample and we conclude that the two groups are sufficiently similar to validate our longitudinal study in that control and intervention are deemed to have similar trajectories of change. In the final evaluation, several difference-in-difference models will be run which include these variables as covariates to check the robustness of results. 12 The sampling margin of error for the group of adolescent girl customers is equal to +/-3%. For the group of non-customers, it is equal to +/- 7%. This means that, for example, if 50% of customer respondents have received training information from their KK, there is a 95% probability that between 47% and 53% of all KK customer girls received training information. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 7 2 Findings This section details findings from the survey, key informant interviews (KIIs) and focus group discussions (FGDs). Findings are organised in the following sub-sections: • Section 2.1 outlines the basic demographics of KK customers and non-customers. • Section 2.2 describes the intervention more broadly and provides the KKs’ own account of the interventions they are delivering in their communities. This gives context for the data collected from beneficiaries. • Section 2.3 on well-being explores girls’ perceptions of their personal safety and well-being, agency, quality of life, mental well-being and sources of counsel that girls seek for support during adolescence. • Section 2.4 on learning establishes girls’ knowledge of aspects of their physical health, including biometrics, SRH and menstrual hygiene; career and training opportunities; and sources of information girls currently access to find the information they need. • Section 2.5 on the KK micro-franchise details awareness and take-up of the KK’s products and services, drawing on the experiences of the customer and the KK. • Section 2.6 on adolescent girls that are KKs give an account of their experiences of learning and earning through their business, with particular emphasis on the adolescent girl. The survey and FGDs gather feedback from slightly different groups of end user beneficiaries: direct KK customers and girls who participate in school activation sessions, respectively13. Findings are distinguished between these two groups. The tables presented in this section employ heat scales, with darker shades indicating a higher frequency of response. Where a question allowed for multiple response, its colour map is pink. Single response questions are green. 2.1 Profile of intervention (customers) and control (non-customers) The basic demographics of KK customers and non-customers are outlined below. Variables on household size, marital status and number of children are detailed in Well-being sub-section 2.2.2. Location of respondents: The distribution of districts is different between the customer and non-customer samples, reflecting the sampling strategy. While the non-customer sample is even distributed amongst the districts of Jessore, Jhenaidah/Kushtia and Bogra (60 interviews each), the customer sample contains 48% of interviews in Jhenaidah/Kushtia, 32% in Bogra and 20% in Jessore. Sampling weights were calculated to counterbalance this difference and mimic an even distribution in the customer sample. However, using these weights did not reduce the disparities across the two groups with respect to other structural variables such as religion or school enrolment. Rather, they even increased the disparities in terms of PPI scores. Sampling weights were therefore not used in the rest of the analysis. Age: Girls’ average age is the same between customers and non-customers: 15 years old, and all girls in the sample are between the ages of 10 and 19. In the rest of the report, the following age-bands are used to cross￾tabulate analytical variables against girls’ age: 13 or below (31% of the overall sample), 14-16 (37%) and 17-19 (32%). Poverty Probability Index (PPI) 14: PPI scores were calculated using the Simple Poverty Scorecard Poverty￾Assessment Tool for Bangladesh. The PPI score ranges from 0 to 100, its value being a proxy to poverty 13 Girls who participate in a school’s activation session may or may not be visited by a KK at home as the girls may live outside the KK catchment area but attend a school that is inside the catchment area. The frequency of contact through school activation sessions is expected to be much lower and it is unclear whether this will lead to sales for the KK. As such, girls who participate in school activation sessions are different to survey respondents, who are customers of the KK. 14 All SPRING impact evaluations use PPI as a proxy for poverty. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 8 likelihoods15 . The mean PPI score is slightly higher in non-customer areas (66) than in customer areas (63). These scores equate to a 0.4% and 0.9% probability, respectively, that the households are below the national lower poverty line; and a 1% and 0.1% probability they are below the USAID Extreme Poverty line. Alternatively, there is a 44.5% and 54.6% probability that the households are living on less than $2 a day 2005 PPP. As with age, PPI￾score bands were created and used for cross-tabulations. These bands split the sample into three approximately equal parts: 59 or below (38% of the overall sample), 59 to 69 (30%) and 70 and higher (32%). Religion: A large majority of the sample is comprised of Muslim households. The proportion of Muslim respondents is 98% in the customer sample and 91% in the non-customer sample. The remaining respondents declared themselves as Hindus. Disability: The Washington Group Short Set of Disability Questions was included at the end of the survey. These questions do not show statistical difference across the two groups. Amongst KK customers, 12% of girls declared having at least some difficulty seeing, even if wearing glasses; 5% of girls declared having difficulties hearing, even if using hearing aids; 21% declared having difficulties remembering things or concentrating on activities; and 5% said they have at least some difficulty with self-care, such as washing or dressing themselves. 2.2 Well-being iSocial’s Theory of Change (ToC) states that through working with girls holistically and providing them with information and improved opportunity, while simultaneously working with guardians to educate them on issues such as early marriage and the importance of education, the KK will improve girls’ personal safety, agency and well-being. To test this, information was gathered on girls’ participation in education, family status, work history and self-stated quality of life. Findings are presented below in sections 3.3.1 to 3.3.5. iSocial’s ToC further asserts that through acting as counsel to the adolescent girls and providing them with referral to organisations equipped to help girls through difficult times, the KK will also serve to improve girls’ mental health. As such, data on girls’ current mental well-being and source(s) of counsel was also collected. Findings are presented in sections 3.3.6 and 3.3.7. Summary of findings on Well-being • The vast majority of girls are attending school and most of these girls are in Secondary School. Most girls have not undertaken paid employment in the last 12 months. • Girls have, on average, four or five other household members who may serve as source of information on personal health and well-being. • A fifth of girls are married, with the average age of marriage of 15 years old. Those not married express the desire to marry at 21 or older. At endline we will track girls’ fortunes to explore the determinants of age of marriage. • Quality of life and mental well-being scores are consistent across customers and non-customers. Girls rate their own health, well-being and feelings as good or almost good and this does not seem to depend on external factors or contextual variables. Girls perceive aspects of getting things done, mood or feelings and health as areas where they are less content. • Mothers act as key support and confidant; and girls have little awareness of external support options (e.g. Marie Stopes) • KK customers access wider network of support than non-customers, in particular, all members of their family and friends. • Girls make little reference to KKs as a source of information. 2.2.1 School enrolment and attainment Figure 1 shows girls’ enrolment status in the areas of KK operation (KK customers). A large majority of girls are currently enrolled in secondary or high school (84%), with 15% being enrolled in primary or tertiary education. Only 15 More information on PPI score calculations for Bangladesh as well as poverty likelihood equivalences can be found here: https://www.povertyindex.org/country/bangladesh. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 9 three out of the 745 KK customers are currently not enrolled in school and/ or educational training. All of these respondents are 17 – 19 years old. Proportions are largely similar in comparison (non-customer) areas, though more non-customer girls are still enrolled in primary school (18% compared to 8% of KK s). One percent of non￾customers are not currently enrolled in school and/or education training. 16 Figure 1: KK customer enrolment Base: all respondents (n=745) Amongst those enrolled in education, 52% of KK customers and 61% of non-customers had completed some or all of primary or junior school; 46% of KK customers and 37% of non-customers had completed some or all of secondary or higher secondary school. The average number of years spent by girls in school is 8 years amongst non-customers, and 8.5 years amongst KK customers. With the average age of girls in the sample being 15 years old, this implies that girls started schooling around the age of six or seven, which corresponds to the beginning of primary school, and have not dropped out or missed any year since. This is consistent with the extremely low proportion of non-enrolled girls reported previously (< 1%). 2.2.2 Household, marriage and children Girls are living with 4.7 other household members on average, consistent across KK customer and non-customer groups. 92% of KK customers live with their mother, 85% with their father and 80% with brothers or sisters (Table 1). Most non-customers also lived with their nuclear family. This provides insight into what sources of information and support are readily available to the girl. Table 1: Other household members Who else lives in your household? KK customers Non-customers Mother 92% 85% Father 85% 79% Mother or Father’s partner 2% 2% Grandmother and/or Grandfather 15% 13% Aunt and/or Uncle 1% 2% Husband or your partner/boyfriend 13% 16% Brothers and/or sisters 80% 74% My own children 8% 7% Other children 4% 2% Other adults 7% 15% Base 745 180 16 The UNICEF MICS 2012 – 2013, pp 121 reports 12.2% of 10 - 15 year olds had not attended school in the last year. 7% 84% 8% Primary school Secondary or High school College or University IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 10 In accordance with these figures, 17% of customers and 21% of non-customers were married, rising to 41% and 55% of 17 – 19 years olds, respectively. Nine percent of customers (52% of those married) and seven percent of non-customers (33% of those married) have at least one child. Among them, 92% of customers have only one child, and the remaining 8% have two children.17 Girls who are married got married at age 15, on average. Interestingly, girls who are not currently married mention much older ages for when they would like to get married: 21 amongst customers and 22 amongst non-customers. Girls who have a child had their first child when they were 16 years old, on average. Girls who did not have a child stated they would like to have one at the age of 20, across both customer and non-customer groups. Age of marriage and age of first child will be monitored during the evaluation period. 2.2.3 Paid work 84% of KK customers and 76% of non-customers reported not having done any paid work over the past twelve months (Table 2). Where girls had undertaken paid work, it was mostly for house cleaning or collecting wood or water, or to run their own business (8% v 20%, and 6% v 4% for customers and non-customers, respectively). Unsurprisingly, the likelihood of being involved in paid work varied with age: only 10% of KK customers below 13 years old were involved in paid work, as opposed to 28% of KK customers 17 years or older. Similarly, 16% of unmarried KK customers did paid work, compared to 39% of KK customers who were married with children. This was consistent amongst non-customers. The relationship between paid work and poverty is however not clear in the sample, as PPI scores are not correlated with the amount of time spent by girls on paid work. 18 Table 2: Girls’ paid work Done any paid work in the last twelve months? KK customers Non-customers I didn’t do any work for which I was paid 84% 76% Sell things for somebody else 0% 1% Planting/harvesting/farm labour 2% 2% Paid for House Cleaning or collecting wood or water 8% 20% Child Care 2% 3% Run my own business 6% 4% Base 745 179 2.2.4 Tree of Life As a warm up exercise and insight into girls’ agency and well-being, we used a projective technique where we asked girls to choose a figure from a metaphorical “Tree of Life” which figure they thought best represented them and why. The “Tree of Life” drawing along with the number of girls who selected each different picture are shown in Annex B. Table 3 shows aggregated codes of the reasons girls gave in explaining the figure they had selected. There was no statistical difference19 between KK customers and non-customers. The majority of girls mentioned that they enjoyed being with their friends, family or other people; either playing with them, helping them do their work or simply hanging out with them. Nevertheless, one out of every five girls said that they preferred being alone and quiet, or staying at home. 8% of girls saw themselves as doing good work and achieving their goals. Only 2% of girls gave negative comments about feeling sad or anxious about their life. 17 The DHS 2014 pp 40: reports 44.2% of 15–19 year olds are married; with the average age of marriage of 15 years of age; 49% of those married have had one child and 5% have had 2 children. UNICEF MICS 2012/13 reports 34.3% of 15 – 19 year old adolescent girls are married. 18 The DHS 2014 pp 26: reports 19% of 10 – 19 year old adolescent girls are working/in employment at the time of survey. 19 Statistical difference is calculated at the 5% level. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 11 Table 3: Girls’ comments on the "Tree of Life" Reason for choosing picture KK customers Non-customers Play with, being around, happy with, help friends 31% 21% Play with, being around, happy with, help family 25% 41% Play with, being around, happy with, help others 6% 3% Like to be alone, quiet, stay at home 19% 21% Do good work, achieve my goals, ambitious 8% 8% Feel happy, cheerful, relaxed 4% 1% Sad, stressed 2% 2% Other 4% 2% Don’t know, unable to explain 0% 0% Base 745 180 2.2.5 Quality of Life The Tree of Life was complemented by the Paediatric Quality of Life Enjoyment and Satisfaction Questionnaire (PQ-LES-Q), a list of 15 items that measure girls’ views about their general health, well-being and feelings about their life. Girls were asked to assess “how things have been over the past week” using a series of themes which they could rate from “Very Poor” to “Very Good”. The detailed frequencies associated to each item are shown in Figure 2. None of the items demonstrate a statistically significant difference between KK customers and non￾customers. Girls rated lowest their capacity to getting things done (2.9 and 2.7, respectively), their mood or feelings (3.4 in each) and their health (3.6 and 3.7, respectively). On a scale of 1 to 5 both customers and non-customers alike rate their life overall as “Good” on average (4.1 and 4.0, respectively), along with the way they get along with friends, family, their ability to get or buy things, and the place where they live. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 12 Figure 2: Paediatric Quality of Life, Enjoyment and Satisfaction Base: All respondents (KK customers: n = 745; non-customers: n = 180) A composite index made up of the 15 aggregated items was created, associating a score of 1 to “Very Poor” and 5 to “Very Good”. This average score across the 15 items is equal to 3.8 in both groups (between “Fair – 3” and “Good – 4”). Girls therefore rate their own health, well-being and feelings as good or almost good, on average. This average score is consistent across contexts and does not seem to depend on the district where girls live, their age, school level or PPI score, which suggests that these feelings are largely related to individual factors. Only girls who are married with children show a slightly lower score (3.6) than girls who are not married or married without children (3.8). 2.2.6 Mental well-being To measure girls’ mental health and well-being, the Warwick-Edinburgh Mental Well-being Scale (WEMWBS) was adapted. The number of items was reduced from 14 to six, and some of the statements were “reversed” to include negative statements alongside positive ones. The idea behind these changes was to make girls think more carefully about their thoughts and feelings so as to avoid the sense of them being led to the “right” answer. Girls had to assess the frequency of each statement from “Never” to “Always”. The detailed frequencies associated to each item are shown in Figure 320 . 20 More than two-thirds of girls said that in the past twelve months, they “never” felt that other people like them, which is at odds with the rather positive responses from other variables. This question has thus been excluded from the analysis at baseline. The responses to this question will be closely monitored at endline to ensure that it is understood properly by all respondents. 3.6 3.4 3.9 3.9 4.1 4.2 3.8 2.9 4.1 4.0 4.1 3.8 3.8 3.7 4.1 1 2 3 4 5 Your health Your mood or feelings School or learning Helping out at home Getting along with friends Getting along with your family Play or free time Getting things done Your love or affection (with family and friends) Getting or buying things The place where you live Paying attention Your energy levels Feelings about yourself Overall how has your life been KK customers Non-customers Very poor Fair Very good IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 13 Figure 3: During the past 12 months, how often have you... Base: All respondents (KK customers: n = 745; non-customers: n = 180) Using the same methodology as for the general well-being scale, an average score across all five items (excluding ‘felt other people like you’) was created by associating a score of 1 to “Never” up to 5 for “Always” for positive statements, and a score of 5 to “Never” down to 1 for “Always” for positive statements. The aggregated score is therefore coded positively, meaning that the higher the score, the better girls’ mental health and well-being. The average score is equal to 3.7 for both KK customers and non-customers, which demonstrates rather good mental well-being and feelings from girls. Like the general well-being score, the mental well-being score does not seem to depend on external factors and is largely constant when tabulated against contextual variables. 2.2.7 Mental well-being: confidants and support Girls were asked who they would speak to in a range of different scenarios. A showcard listing various sources of information21 was used to prompt their response. In each scenario, girls were asked to provide three possible people they would seek support from or speak to in each scenario. All sources of support mentioned have been combined and detailed responses are listed in Table 4. From their responses, it is apparent that few girls identified the KK as a source of counsel or support. Table 4: Source of support (multiple response – all mentions) Arguments with friends Being bullied by others in the village Unwanted attention from boys What to do when feeling worried, sad, lonely or afraid Customer Non￾Customer Customer Non￾Customer Customer Non￾Customer Customer Non￾Customer Mother 67% 78% 81% 83% 34% 79% 71% 67% Father 17% 7% 13% 7% 11% 4% 11% 4% Grandparents 28% 20% 33% 29% 29% 15% 30% 17% Other family 41% 31% 25% 14% 29% 25% 37% 38% Friend 27% 14% 13% 4% 29% 21% 28% 23% Teachers 2% 0% 1% 1% 1% 1% 1% 0% 21 Each time respondents were asked about the source of information, they were prompted with the use of a showcard that listed the following: Mother, Father, Grandmother/Grandfather, other family member, Friend, Teachers, Doctor/Nurse, the KK, Radio, Leaflet, or other. 1.9 1.7 2.9 2.6 1.3 1.8 1 2 3 4 5 Been so worried about something that you could not sleep at night? Felt lonely? Felt good about your abilities compared to others? Felt that you can be yourself around other people? Felt useless? Felt other people like you? KK customers Non-customers Never Sometimes Always IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 14 Doctor / nurse 3% 0% 4% 0% 4% 0% 8% 1% The KK 0% 0% 1% 0% 0% 0% 0% 0% Don’t know 1% 2% 4% 6% 3% 4% 3% 10% Refused 1% 1% 1% 1% 1% 1% 0% 1% Base 745 180 745 180 745 180 745 180 While the majority of girls surveyed indicated that they knew who to speak to when they feel worried, sad, lonely or afraid, data gathered from FGDs in schools suggest otherwise. When speaking to girls about how they manage moments when they feel down, it appeared that many girls were less familiar and comfortable with the subject. Some girls were unwilling to participate in the discussion, stating they “didn’t realise mental well-being was an issue” (Group 6, 17-18 Y). Amongst the girls that participated in discussions on mental health and where they sourced support, girls frequently spoke of turning to their mother for advice and support, referencing close and open relationships with their mothers and their mother being the natural person to whom they would turn when in need. One participant shared: “My mother is most helpful and supportive. I am absolutely open with my mother. I do not have any barriers with her…I know the world through my mother” (Group 4, 10-13 Y). As in the survey, support from older sisters and other female members of the immediate family (such as aunt, grandmother and sister-in-law) were also frequently mentioned. One girl recounted turning to various members of her family, but while she had been offered advice, she did not feel that she was getting support from them: “I did not get support from anyone. My mother, sister and relatives told me to cheer up during menstruation” (Group 4, 10-13 Y). When asked about the advice they had been given, many girls spoke of being told that mood changes and feeling down were a natural part of growing up. They should thus accept they would sometimes feel this way and try to cheer up, keep smiling and take their mind off their worries by having fun with those around them: “According to my understanding, when my mood changes, I should try to keep smiling all the time and do fun with my near ones” (Group 1, 10-13 Y). Girls mentioned that when feeling down, they “shouldn’t keep sad in [their] faces [and] not feel lonely” (Group 1, 10-13 Y). They were advised to “listen to music, watch movies, move outside with friends [and] read story books” (Group 6, 17-18 Y). 2.2.8 Mental well-being: organisational support To assess other forms of formal support, respondents were asked about whether they know of any organisations that could provide help and advice to them or their friends if they were feeling worried, sad, lonely or afraid. Only 2% (n=13) of KK customers responded positively to this question, as all other respondents had either affirmatively not heard of such an organisation (90%) or were unsure (8%). Of the thirteen KK customers who had heard of a well-being support organisation, six did not know the organisation’s name. Three could recall an actual name, Manobkallyan, an Islamic NGO focusing on extending social welfare to those in need, while the remaining respondents could only provide vague descriptions, with two respondents describing it as ‘counsellor’ and two as a ‘vocational institution’. This could be indicative of a lack of access to formal well-being support services outside the household. 2.2.9 Overall sources of information for mental well-being As reported in sub-sections 3.3.7 and 3.3.8, combining all questions where girls were asked who they speak to when in need of emotional support22, it becomes evident that while KK customers do not yet seek counsel from the KK, they do access a much wider support network than non-customers. 22 Cumulating the following survey questions: B2.2: Who to speak to about: b. Arguments, c. Bullied, d. Unwanted attention, e. Worried; B2.3 Awareness of organisations. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 15 Figure 4: Sources of information for mental well-being (aggregate) Base: All respondents (KK customers: n = 745; non-customers: n = 180) 2.3 Learning iSocial’s Theory of Change (ToC) asserts that through providing girls with information, services and products about their physical health, sexual and reproductive health (SRH), menstrual hygiene, and career and training opportunities, the KK will improve the girls’ knowledge and behaviour, ability to make the right health and career choices, and ultimately follow her aspirations. In this sense, the impact on learning is to improve the girls’ health and agency as well as employment potential through information and access to improved choices. iSocial has developed training content for KKs to use in their information sharing sessions. This content, within the context of specific products and services offered by the KK, informed the design of data collection tools to measure knowledge acquisition as a result of the KK intervention. The sub-sections below establish the extent to which customers have benefited from the KKs’ health test services and products, career and training connections, and awareness raising campaigns at this early stage in the intervention. Summary of findings on current knowledge and the opportunity to learn Physical Health KK Customers show greater awareness and uptake of services related to their body weight, its classification, their BMI, bloody type and of anaemia testing than non-customers. While a doctor/nurse are frequently mentioned sources of information and service provider in all these areas, so too is the KK. Allowing for multiple responses, the KK is the second most frequently mentioned source of information amongst customers, with approximately a third of all mentions. Younger girls' customers, in particular, were more likely to mention the KK than other sources of information. Sexual Reproductive Health (SRH) • Knowledge of SRH was consistent across both KK customer and non-customer group, with girls answering 3 out of 5 true or false questions correctly. Allowing for multiple responses the KK shares third place as the most frequently mentioned source of information, by a fifth of customers. Non-customers were more likely to cite their mother. • Fewer girls stated they knew what to do if they experienced pain during menstruation, suggesting there may be a gap in knowledge in how to manage pain. Career and Training • While the vast majority of girls are aware of educational scholarships, only around a third are aware of educational competitions; and only one in ten girls over the age of 16 years knows of other training providers. Around half of KK customers felt they knew a fair amount about their future career opportunities 63% 13% 30% 33% 24% 1% 5% 0% 0% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Mother Father Grandmother and Grandfather Other family member Friends Teachers Doctor or Nurse The Kishori Kallyani Radio Leaflet KK customers Non-customers Base: all respondents IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 16 compared with only a quarter of non-customers. That said, there is very little evidence of the KK as a source of information in career and training. • This is consistent with KKs own account of her activities and suggests this area of the intervention has yet to be delivered. Overall conclusions on learning • Reference to the KK as a source of information for physical and sexual reproductive health issues is positive testimony to the activities of the KK. Customers more frequently cited the KK as a source of information in physical health than SRH which suggests the KK has had more impact in this area. • At this early stage, it seems both customers and non-customers have similar levels of knowledge with regards to SRH. • Little reference to the KK as a source of information on career & training is consistent with feedback from the KK and suggests this area of intervention has yet to be delivered. 2.3.1 Awareness and knowledge of physical health and biometrics Girls were asked a series of questions on their knowledge of their weight, body mass index (BMI), blood type and anaemia, as well as their source of knowledge for each aspect of their physical health. Weight and Body Mass Index (BMI) 68% of KK customers and 60% of non-customers reported knowing their weight. Of those who knew their weight, 71% of KK customers and 59% of non-customers stated knowing if they were considered underweight, normal or overweight. Of those who knew their weight classification, the source of information23 varied greatly between customers and non-customers. The majority of non-customers (67%) stated knowing their weight classification from a doctor or nurse, while the primary source of information for KK customers was the KK herself (37%) followed by a doctor or nurse (22%). In terms of age variation, in general, older girls (aged 16-19) tended to cite learning about their weight from a doctor or nurse, while younger girls (less than 14 years old) learned about their weight categorisation from their teachers. For customers who identified the KK as the source of information, the frequency of citing the KK increased with age. When asked about their body mass index (BMI), 23% of customers reported knowing their BMI, as opposed to only 9% of non-customers. Of those who knew their BMI, more non-customers said they obtained this information from their teacher (38%) than any other source of information. For KK customers, the KK was again referenced as the primary source of information (38%), followed by teacher (30%) and doctor or nurse (14%). For KK customers, the vast majority of younger girls (less than 14 years-old) learnt their BMI from the KK, followed by a teacher. For the middle age group (14-16), most referenced their teacher or the KK. In the oldest age group (17-19), about half cited the KK with roughly a fourth referencing a teacher or doctor or nurse, respectively. 23 Sources of information: Mother, Father, Grandmother/Grandfather, other family member, Friend, Teachers, Doctor/Nurse, the KK, Radio, Leaflet, or other. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 17 Table 5: Girls biometric and physical health knowledge (% responding yes) KK Customer Non-customer Do you know your weight 68% 60% Do you know your weight classification 71% 59% Do you know BMI 23% 9% Do you know blood type 58% 50% Have you taken an anaemia test 13% 7% Base 745 180 Blood Type and Anaemia When asked about knowledge of their blood type, only 58% of KK customers and half of non-customers stated knowing their blood group. The vast majority of those who answered positively were girls aged 14 and up. Of non￾customers who knew their blood type, all except one individual stated knowing this information from a doctor or nurse (69%), or their teachers (30%). KK customers also frequently referenced doctors or nurses (47%) and teachers (28%). The KK was also referenced by 19% of customers. As with knowledge on weight, older girls were more likely to have gained information on their blood group from a doctor or nurse across both groups. The age spread of customers who cited the KK was distributed rather evenly. Respondents were also asked about whether they have ever taken an anaemia test. Only 13% of KK customers and 7% of non-customers had taken an anaemia test. Of the few non-customers who had taken an anaemia test, the majority were aged 17-19. Amongst KK customers who had, the likelihood of having taken an anaemia test also increased with age, though the age spread was less stark. Overall sources of information for physical health Combining all sources of information girls accessed to learn about their physical health and biometrics24, it is evident that amongst KK customers, the doctor / nurse and KK are of almost equal importance. Compared with non-customers, it appears that the KK may have taken the place of the doctor / nurse. Bearing in mind that customers show significantly higher levels of knowledge of all biometrics asked, this would suggest that the KKs have improved physical health knowledge amongst their customers. 24 Weight, BMI, Blood Type, and Anaemia Test. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 18 Figure 5: Sources of information on physical health and biometrics Base: All respondents to source of information questions on B1.1c weight (n=359/64), B1.1e BMI (n=172/16), B1.2b blood (n=435/87) and B1.2d anaemia tests (n=98/12) 2.3.2 Awareness and knowledge of menstrual and sexual and reproductive health To test the theory that KKs improve adolescent girls’ knowledge of menstruation and hygiene, girls’ self-perceived knowledge of menstruation and menstrual health was measured. Respondents were asked how much they thought they knew about: why girls and women menstruate, eating healthily during menstruation, what a normal menstrual cycle is, and what to do if they were in pain during menstruation. There were no significant differences between the two groups, indicating both customers and non￾customers felt they were equally knowledgeable. Amongst customers, 82% felt they knew a lot or a fair amount about why they menstruated, 74% what a normal menstrual cycle is, 72% how to eat healthily during menstruation and 55% what to do if they were in pain (Figure 6). This suggests there may be a gap in knowledge in how to manage menstrual pain. Confidence increased with age across both groups, with girls in older cohorts being more likely to declare more knowledge of different aspects of menstruation. Figure 6: KK customers’ self-perceived knowledge Base: all respondents (n=745) 8% 0% 0% 4% 3% 20% 35% 30% 0% 0% 0% 10% 20% 30% 40% 50% 60% 70% Mother Father Grandmother and Grandfather Other family member Friends Teachers Doctor or Nurse The Kishori Kallyani Radio Leaflet KK customers Non-customers 23 29 23 19 59 45 49 35 11 13 13 11 7 13 15 34 0 10 20 30 40 50 60 70 80 90 100 The reason why girls and women menstruate Eating healthily during menstruation What a normal menstrual cycle is What to do if you are in pain during menstruation A lot A fair amount Not very much Nothing at all IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 19 Girls were also asked a series of true or false questions to test their knowledge of menstruation and menstrual hygiene. The aspects tested were taken from iSocial‘s training content. Respondents’ answers are detailed in Table 6 below, with the correct answer indicated in italics. Overall, girls correctly answered most questions, the exception being that anaemia was the result of too little calcium in the body. Overall levels of knowledge were consistent across KK customer and non-customer groups. Table 6: True or False statements on knowledge of menstruation TRUE FALSE Don’t know KK customers Non￾customers KK customers Non￾customers KK customers Non￾customers Menstruation takes iron from your body 81 % 76% 4% 1% 15% 23% Watermelon is a good source of iron 64% 70% 12% 4% 24% 26% A normal menstrual cycle is 21 days 40% 37% 40% 44% 19% 19% Using a sanitary pad during menstruation is as healthy as a piece of cloth 23% 11% 70% 74% 7% 14% Anaemia is the result of too little calcium in your body 66% 66% 11% 4% 23% 29% Base 745 180 745 180 745 180 Sexual and reproductive health services: Marie Stopes Centre The Marie Stopes Centre is an independent provider of SRH services. The KKs’ impact on adolescent girls will be evidenced through an increase in awareness and use of the Centre. Only 13% of customers and 9% of non-customers reported having heard of the Centre. Across both groups, older girls (13 and over) were more likely to be aware of the Centre. Of the few non-customers who were familiar with the Centre, about half had heard of Marie Stopes from a family member, primarily their mother. For KK customers who were familiar with the Centre, the primary source of information was the KK (29%). This was followed by another family member (24%), then mother (17%). As opposed to sources of knowledge for basic biometrics (weight, BMI, blood type, anaemia), teachers and doctors / nurses were not cited very widely, which could be indicative of trust or cultural factors around sexual and reproductive health. However, when later asked who they would speak to about changes to their body, most girls cited their mother (82% customers and 91% non-customers), with a few mentions of other members of the family and only 3% of customers mentioning a doctor / nurse. There were no mentions of the KK. Sources of information on menstrual and sexual and reproductive health Respondents were asked about where they receive information about their menstrual health and SRH. The majority across both groups cited their mother (85% of KK customers, 88% of non-customers), followed by another family member (39% and 37%, respectively) and a friend (39% and 24%, respectively). Thirty percent of KK customers also made reference to the KK as a source of information on their menstrual and SRH. The age spread was fairly even amongst girls who mentioned the KK as an information source. Younger girls across both groups were more likely to reference a grandparent while older girls were more likely to cite a friend or doctor / nurse. A notable difference between groups is the frequency that KK customers cited their father as a source of information on their menstrual and SRH (20% KK customers v 1% non-customers). Younger girls in particular were more likely to receive information about their menstrual and SRH from their father. Similarly, when asked who they would speak to about changed in their body/menstruation, the majority of girls cited their mother (82% of KK customers and 95% of non-customers. A minority of KK customers cited their grandparents (5%) or other family members (6%). IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 20 Overall sources of information on sexual and reproductive health Combining all sources of information girls accessed to find out about SRH and menstrual hygiene25, it is evident that both KK customers and non-customers alike rely on their mother as their primary source of information, followed by another family member (Figure 7). Thereafter, customers are as likely to cite the KK as they are their grandparents, and as like as non-customers cite friends. Figure 7: Overall source of information on SRH Base: all respondents to sources of information on SRH: B1.5 Marie Stopes (n=97/17), B1.6 Overall SRH (n=744/180), B2.2a Changes in body/menstruation (n=745/180) Girls’ reliance on their mother as a primary source of information is consistent with feedback from FGDs undertaken at schools that form part of the KKs’ information-sharing activities. While the KK herself was not mentioned at this early stage, the importance of the mother and older females in the extended family was reinforced. One participant said of her mother: “The most helpful person is my mother. My mother is always close to my me. If I face a problem then my mother understands, she has faced these types of problems in adolescence. She understands what is good or bad” (Group 2, 14-16 Y). However, girls were also selective in who they chose to speak to about different issues, with some highlighting that on occasions where they felt too shy to speak to their mothers, they would seek advice from other females in the family: “Sometimes we feel shy to get the help from [our] mothers. Then we discuss with [our] sister-in-law” (Group 2, 14-16 Y). The FGDs further suggested that as girls got older, they were taught about SRH in school and could source their textbooks as an additional source of information. Young girls, however, were not yet learning about SRH in school and so were more dependent on their family for information. This knowledge base was deemed by some as inadequate, with some younger girls feeling a lack of support. According to one respondent: “Girls mainly depend on their mother or elder sister regarding their physical changes issues. Family members have limited knowledge; therefore, we do not get adequate support” (Group 1, 10-13 Y). 2.3.3 Careers and Connectivity Girls that were still in education or training were asked if and how they knew about scholarships and education competitions. Girls aged 16 older were asked if and how they knew of training providers and their future careers. Scholarships and Education-related Competitions When asked about knowledge of any scholarships available for their education, the vast majority of girls across both groups responded positively. 85% of customers and 81% of non-customers that were still in education or training said that they were aware of an available scholarship, with girls across all age cohorts showing similar 25 Cumulating the following survey questions: B1.5 Aware of Marie Stopes, B1.6 Overall source of info for SRH; B2.2a who to speak to about changes to the body/menstruation. 64% 14% 20% 34% 17% 6% 10% 20% 1% 2% 0% 10% 20% 30% 40% 50% 60% 70% 80% Mother Father Grandmother and Grandfather Other family member Friends Teachers Doctor or Nurse The Kishori Kallyani Radio Leaflet KK customers Non-customers IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 21 levels of awareness. The most widely cited scholarship was a stipend, mentioned by 92% of customers and 90% of non-customers. Talentful was the only other scholarship mentioned by the remaining respondents. Teachers were the primary source of information on scholarships for both customers and non-customers (40% and 96%, respectively) across all age groups, followed by mothers (28% and 17%, respectively), who tended to be preferred by younger girls. Overall, KK customers referenced more varied sources of information. In addition to teachers and mothers, customers also cited their friend (27%), fathers (23%) and another family member (13%). The KK was mentioned as a source of information on scholarships by only five KK customers (1% of the sample). Family members were mentioned by mostly younger girls, while teachers displayed more prominence with age. Across all respondents, about half on average said that they would consider applying for the scholarship they had mentioned. Girls were also asked about knowledge of any education-related competitions available to them. Only one-third of KK customers stated knowing of any such competitions, compared to almost half of non-customers. Knowledge of competitions was spread fairly evenly across age cohorts. The five most frequently cited competitions included mathematics, debate, science, sports and general quizzes. While only 44% of non-customers that were aware of the competitions, said they would consider applying for the competition(s), 55% of KK customers said they would. KK respondents were spread fairly evenly across age cohorts. When asked about how they know of these competitions, the vast majority in both groups mentioned teachers (92% of customers and 93% of non-customers). Family members were also mentioned by roughly half of respondents, and friends by about one-fourth of girls. The KK was only mentioned by 4 KK customers, or 2% of girls. Table 7: Source of information for career and training opportunities Scholarship Competition Training provider Future career Customer Non￾Customer Customer Non￾Customer Customer Non￾Customer Customer Non￾Customer Mother 28% 17% 15% 15% 0% 17% 58% 58% Father 23% 10% 10% 16% 7% 8% 35% 32% Grandparents 1% 0% 2% 0% 0% 0% 3% 4% Other family 13% 2% 20% 18% 17% 33% 28% 13% Friend 27% 12% 29% 20% 33% 50% 28% 21% Teachers 40% 96% 92% 93% 93% 67% 57% 56% Doctor / nurse 0% 1% 0% 0% 3% 0% 2% 0% The KK 1% 1% 2% 0% 10% 0% 11% 0% Don’t know 0% 0% 0% 0% 0% 8% 0% 15% Base 630 146 242 80 30 12 326 78 Career and Training Opportunities All girls 16 years of age and over were asked if they knew of places where they could obtain further training26 . Only 9% of customers and 14% of non-customers knew of where they could obtain further training27. Age had little bearing on awareness. Teachers and friends were the most important source of information for both KK customers and non-customers (93% v 67%; and 33% v 50%, respectively). Girls who knew of a training provider were also asked about whether they had applied to a training course. Eight KK customers and one non-customer stated having applied to a training course. Source of information and perceived knowledge of Future Career / Training Opportunities All girls aged 16 and above were asked if they had obtained any information from a list of possible sources. Across both customers and non-customers, over half of respondents cited their mother (58% in each) and teachers (56% v 57%). Girls also mentioned their fathers (35% v 32%) and friends (28% v 21%). 11% of KK customers mentioned having obtained future career / training opportunities from the KK. 26 iSocial provide these services to girls 16 and older, so all girls will be eligible to receive these services at endline data collection. 27 It should be noted that the number of girls who responded positively to knowing a training provider was low (30 customers and 12 non￾customers). Any proportions should thus be read with caution. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 22 When asked how much they knew about their future career opportunities, almost half (46%) of KK customers felt they knew a lot or a fair amount compared with only around a quarter (28%) of non-customers (Figure 8). Figure 8: Knowledge of future career opportunities Base: All respondents 16 years of age and older (n=326/78) A similar trend was evident when girls were asked how much they knew about training opportunities. 47% of KK customers said they knew a lot or a fair amount of training opportunities, as opposed to 30% of non-customers. Overall sources of information on career and training opportunities Combining all sources of information girls accessed to find out about career and training opportunities28 , a rather different pattern emerges than for SRH and mental health, with the teacher being the most important source of information for both KK customers and non-customers alike. The KK is barely mentioned. This is consistent with feedback from the KKs themselves, who recounted sharing information on SRH and mental health, but not on career and training. Figure 9: Overall sources of information on career and training opportunities Base: All respondents to sources of information on career & training: B3.2 Scholarship Sources (n=630/146); B3.4 Competition sources (n=242/80); B3.5 Training Providers (n=30/12) B3.7 5 Future Career Sources n=326/78) 28 Scholarships, competitions, training provider, training and career opportunities. 22% 24% 36% 17% 10% 18% 41% 31% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% A lot A fair amount Not very much Nothing at all Customers Non-customers 21% 14% 1% 13% 19% 43% 0% 3% 0% 0% 0% 10% 20% 30% 40% 50% 60% Mother Father Grandmother and Grandfather Other family member Friends Teachers Doctor or Nurse The Kishori Kallyani Radio Leaflet KK customers Non-customers IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 23 In the FGDs, girls’ life ambitions and knowledge of future career and training opportunities were discussed with girls from all ages groups (10-19). Consistent with the survey, the FGDs evidence a rather different pattern of sources of information than those reported by girls on mental well-being and SRH. Overall, girls of all ages were less knowledgeable about their career and training opportunities than they were of their SRH. Among those that could comment on discussions they had had about their future career, the most common source of information were the girls’ teachers, father, sisters, brothers and uncles. According to one participant, “this type of [career] information [is mainly] provide[d] by my father, because he is always going outside the house and getting information from other sources” (Group 1, 10-13 Y). Similarly, one respondent said their teacher was the most important source of information “because he knows my career path and is helping me to create a bright future” (Group 2, 14-16 Y). Mothers were less frequently mentioned than when discussing SRH and mental health. The subject of career and training was notably also one where NGOs, specifically BRAC and Jagoroni, were highlighted as sources of information amongst girls aged 17 and older. The KK was not mentioned by any girls. FGDs also gave insight into some of the cultural barriers adolescent girls faced in pursuing a career, with some girls expressing concern that parents might stop them from completing their education as it was not perceived to be relevant for girls, and also exposed them to the dangers of public sexual harassment or sexual assault by men. For example, one participant said their “relatives told [them] there is no need for higher education for girls… [and that] girls should not move out of the house” (Group 2, 14-16 Y). With regards to public sexual harassment or assault, one participant explained that to avoid harassment, guardians opted to “organise the girl marriage instead” of sending her to pursue her education or a career (Group 2, 14-16 Y). 2.4 The Kishori Kallyani Service This section focuses on the KK intervention activities in her community. Namely: how customers have grown to know the KK, the products girls have bought directly from her, the referral services customers are aware of, their trust in the KK and the most important thing girls have learnt from the KK. Summary of findings on the KK products and services • Most customers know of the KK through her visiting their home, and only around a quarter know her through social or community networks. Most customers have known the KK for a few months and around half state the KK visits them every one or two weeks. These visits are more frequent that iSocial business model anticipated but may be a necessary part of establishing her business. KKs are nearly as likely to visit the mother as the adolescent girl. • Customers buy Orsaline and sanitary pads most often, as it is convenient and avoids embarrassment; and stationary, undergarments, Internet services and top-up cards least often. The most frequently cited reason for not buying from the KK is that she is not available when needed. • With respect to third party referrals, few customers had taken up services, with the most (a fifth) have taken up Aponjon. • Nearly all customers stated they trusted the KK and the information she shared, and eight in ten felt their behaviour had changed as a result of information shared. Four in ten girls stated the most important thing they had learned from the KK was advantages of using the sanitary pad. This was highest amongst girls aged 14 -16 years. • With respect to other ways the KK could help, four in ten girls sought more information and help on physical and menstrual health issues; and a similar proportion raised issues specific to the product range/prices offered by the KK; one in ten sought help with personal safety issues and a similar proportion sought help with career & training. 2.4.1 Getting to know the Kallyani KK customers were asked a series of questions about their history with and exposure to the KK. The majority of customers claimed having first heard of the KK because she came to either the house (80%) or school (7%) to introduce herself. Only 13% of girls stated having already known her, through family (9%), the village community (3%) or friends (1%). Where the KK was not previously known, she tended to introduce herself by visiting the households of adolescent girls across all age groups, though in schools KKs tended to target primary and lower IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 24 secondary schools (girls 16 and younger) more frequently. This survey finding aligns well with the self-described nature of door-to-door visits from interviews with the KKs (Section 2.2.1). The length of time that girls had known the KK varied greatly with a variance of 2.8 months, and the average duration of familiarity at the time of the interview was only 3.4 months. This was consistent across the districts, despite the earlier introduction of the KK in the districts of Jessore and Bogra (August and December 2017, respectively) than Jhinaidah/Kushtia (January/February 2018). However, it is consistent with the average length of time, of 4 – 6 months, the KKs we conducted KIIs with indicated they had been working for iSocial. This will be monitored and further explored during endline data collection, to include KIIs with KKs that no longer work for iSocial. In terms of frequency of visits, 83% of girls said that the KK came by the house at least once a month, with almost half stating that the KK came by every one or two weeks. Customers in Jessore and Bogra were more likely to state the KK visited every one or two weeks while those in Jhinaidah/Kushtia stating visits were monthly. The frequency of visits to the household did not correspond to a particular customer age group. Consistent with the KKs’ own feedback29 , only roughly half (55%) of respondents said that when the KK visits the household, she comes to see the girl herself. Forty-five percent of girls said that the KK actually comes to visit the mother. Three girls also specified that the KK primarily comes to visit their aunt. The likelihood that the KK was visiting the girl increased with age, from 35% of 10 – 13 year olds to 72% of 17 – 19 years olds as illustrated in the figure below. Figure 10: KK primary customer by age of adolescent girl (%) Base: All KK customers (n=745) 2.4.2 Adolescent girl products and purchasing behaviour Girls were asked about their, or their mother / guardian’s, purchasing preferences for a basket of adolescent girl products, in particular with regards to their preference to purchase these products from the KK. The basket of goods includes: orsaline30, sanitary pads, hand soap, stationary, undergarments, internet services and a SIM card. As can be seen in Table 8, girls / mothers tended to purchase mostly hygiene-related products from the KK, namely sanitary pads and orsaline (61% and 52%, respectively), with 37% also buying hand soap from the KK and 11% purchasing undergarments. More commonplace and neutral products such as stationary (pen or pencil), internet services and SIM cards were bought elsewhere. Twelve percent of customers had not bought any products 29 Detailed in Section 2.1. 30 Orsaline is a liquid formula made of salts and sugar which is used to compensate for the loss of excessive fluid from the body, such as during diarrhoea. 72% 57% 35% 38% 43% 65% 0% 10% 20% 30% 40% 50% 60% 70% 80% 17 - 19 years old 14 - 16 years old 10 - 13 years old My Mother Me IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 25 from the KK. Customers that had not bought products will have taken up health services or referrals. These findings are consistent with the KKs’ own records of their product sales. Table 8: Where do you (or your mother / guardian) normally buy the following items from? Don’t buy these goods Buy from Kallyani Don’t buy from Kallyani Don’t know Orsaline 3% 52% 44% 0% Sanitary pads 5% 61% 34% 1% Hand soap 4% 37% 58% 0% Stationary (pen or pencil) 5% 14% 80% 1% Undergarment 11% 11% 69% 9% Internet services 13% 10% 71% 7% SIM card/top-up card 6% 15% 77% 2% Base 745 745 745 745 According to the survey, the most important reason girls bought from the KK was convenience, with the next most important reason being to avoid embarrassment (Table 9). This is consistent with feedback from the KKs themselves. One KK shared: “In my working area, the mothers and the adolescent girls prefer to buy goods from me because they feel uncomfortable to buy those goods from the local market” (KII 6). Less often mentioned reasons for buying from the KK were local availability (8%) and price (5%). KKs themselves mentioned that they are competing with the local markets, who may be offering both better prices and credit facilities. These findings could raise important questions on the efficacy of KKs as a last mile distribution channel, as set out in iSocial’s ToC. We will investigate this further in subsequent data collection. Older girls were more likely to mention convenience as a reason for purchasing, while younger girls identified local access and price. The former could reflect how obligations change with age, in that as girls grow older they tend to become more occupied with work and / or tending to the household. Younger girls (13 and below), on the other hand, could perceive these products as being more expensive or unavailable due to having a general lack of purchasing power in the household, attributed perhaps to not only age but gender. Embarrassment to purchase the product(s) was mentioned more frequently by girls aged 14-16, which could correspond to heighted social awareness at that age. Table 9: The most important reasons for purchasing goods from the KK by age of KK customer Most important reason for purchasing from the KK Age of KK customer Total sample 10 – 13 years 14 – 16 years 17 – 19 years Convenience 57% 50% 56% 65% Too embarrassed to purchase in a shop 27% 21% 33% 25% Can’t obtain locally 8% 14% 7% 2% Good price 5% 10% 2% 4% Better quality than what is available locally 3% 6% 2% 3% Base 639 195 232 212 When respondents were asked about why they do not buy certain girl products from the KK, price, KK absence and local availability were identified as the most important reasons for not purchasing KK products by the majority of girls (Table 10). Reasons for not purchasing products from the KK were distributed evenly amongst age groups. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 26 Uncompetitive pricing and limited product availability were also cited by KKs as barriers to selling. Table 10: The most important reasons for not purchasing goods from the KK Most important reason for not purchasing from the KK Too expensive 20% KK not available when I needed the product 44% Can get locally 15% Out of stock 2% Poorer quality than what is available locally 6% I do not need these products 13% Base 89 2.4.3 Third party referral services As part of iSocial’s ToC, the KKs act as referral to third party services that are able to further promote girls’ overall well-being. Thus far, iSocial has succeeded in aligning with the following: Aponjon, Hello Doctor, Green Delta and Ajkerdeal, as detailed below. This is expected to increase over the evaluation. Respondents were asked about their knowledge of these services, with frequencies collated in Table 11 below. Aponjon Aponjon is a mobile health service for expecting and new mothers in Bangladesh. Four in ten (43%) KK customers had heard of Aponjon compared with nine percent of non-customers. Of those who were aware of Aponjon, most non-customers had learned about it through their mother (50%) or father (25%), while KK customers had heard about Aponjon from a doctor or nurse (62%) a family member (28%), or a friend (5%). Only five girls had heard of Aponjon from the KK. Amongst KK customers who were married, half (54%) of those aware of the service had bought it, this was consistent across those with and without children. This equates to 4% of the sample as a whole. None of the married non-customers had bought the service. Hello Doctor Hello Doctor is a mobile app through which customers can seek medical advice from a qualified doctor. The service is meant to make healthcare personalised, affordable and accessible to all. Most respondents, customers and non￾customers alike, had not heard about Hello Doctor. Only about a fifth (21%) of KK customers were familiar with the app, as opposed to only 11% of non-customers. Awareness increased with age, rising from 17% of 10 – 13 year olds to 29% of 17 19 year olds. Amongst KK customers who had heard of Hello Doctor, most heard about it from a doctor or nurse (44%), their mother (11%) or a friend (11%). By contrast, the primary source of information for non￾customers was a Hello Doctor leaflet. Of those who had heard of the Hello Doctor app, 38% of customers and 11% non-customers reported to have bought it. This was consistent across age and suggests younger girls may have been responding on behalf of their families. This will be further explored at endline. Green Delta Green Delta is a private, short-term insurance provider in Bangladesh providing products ranging from microinsurance for people in rural areas to insurance for garment workers, migrants and women. For the KK business model, we asked respondents about two specific Green Delta products: personal accident insurance and the Nibedita-Comprehensive Insurance scheme for women. Personal Accident Insurance Green Delta’s personal accident insurance is a product that provides compensation in the event of an accident causing injury or death, applicable to individuals aged 16-65 years of age. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 27 When respondents were asked about whether or not they had heard of the product, only 12% of KK customers and 6% of non-customers indicated they had. As with the previous third party health services, the main source of information for KK customers who were aware of the service was a doctor or nurse (44%), with the second most referenced source for customers being a friend (19%). The KK was only mentioned by one individual (1% of all customer responses). For non-customers, over half of girls had learned about the product from their parents. Amongst KK customers 16 years and upwards that were aware of the insurance (15%), a third (35%) confirmed they had bought the insurance scheme. This equates to 2% of the sample as a whole. None of the non-customers had bought the service. Nibedita A comprehensive insurance policy for women, the Nibedita-Comprehensive Insurance scheme provides affordable coverage to females in both urban and rural areas, applicable to women 18 – 65 years of age. Only 5% of KK customers and 3% non-customers had heard of Nibedita. Amongst KK customers, the main source of information was again a doctor / nurse (38%) or friend (21%), with only one girl citing the KK herself. Non-customers cited their father. Amongst KK customers 18 and over who were aware of the insurance (8.5%), 42% had gone on to buy it. This equates to 3.5% of all 18+ year-old KK customers and 1% of the overall sample. Ajkerdeal Ajkerdeal is an online shopping website through which KKs earn commission by providing delivery services on behalf of the website. 10% of customers and 9% of non-customers had heard of Ajkerdeal. KK customers tended to have found out about the service from a doctor / nurse (55%) while half of non-customers found out about the service through a family member. Only one KK customer had heard about the service from the KK herself. Of those that had heard about the service, 39% of KK customers and 13% non-customers had used it. This equates to 4% and 1% of the overall KK customer and non-customer samples respectively. Table 11: Awareness and purchase / use of third party referral services Aware Bought Customer Non-customer Customer Non-customer Aponjon 43% 9% 22% 0% Hello Doctor 22% 11% 8% 1% Green Delta Personal Accident Insurance 12% 6% 4% 1% Green Delta Nibedita Insurance 5% 3% 1% 0% Ajkerdeal 10% 9% 4% 1% Base 745 180 745 180 From sales records shared by KKs who participated in the baseline study, it was noted that referral sales of Aponjon were mentioned but no other services. Trust KK customers were asked to gauge their degree of trust in the information / learnings the KK shares with them. The overwhelming majority of customers (94%) agreed that they believe the information that the KK tells them, while most girls (84%) also believed that the information the KK shares with them has brought changes to the way they go about their life. The degree of trust and belief in the positive impact of KK information was consistent across all age groups. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 28 Figure 11: Trust in KK information Figure 12: KK information has changed life Base: all KK customers (n=745) Base: all KK customers (n=745) In terms of the most important learnings from the KK thus far, most girls’ responses were related to issues of physical health and menstrual hygiene, as detailed in Table 12 below. This finding is in line with the frequency that the KK was cited as a primary source of information on physical health as well as menstrual and SRH. Five girls also mentioned learning about problems related with early marriage, with another five mentioning learning to use the internet. Eleven girls said learning about the KK service in general was the most important learning from the KK thus far. Table 12: Most important KK learnings Most important thing learned from the Kishori Kallyani (self-reported) All girls 10-13 14-16 17-19 Knowing about the advantages of using a sanitary pad 42% 39% 48% 39% Knowing about physical health 13% 13% 10% 17% Learning about health and physical changes 8% 4% 12% 9% Learning about nutritious food 6% 12% 3% 4% Knowing about own physical weight 4% 5% 5% 4% Knowing about menstrual issues 4% 6% 4% 4% Blood test 4% 6% 2% 6% Hygiene practices 4% 4% 6% 3% Learning how to keep yourself healthy 3% 3% 4% 3% Base (all KK customers) N = 745 234 269 242 When asked about other ways in which the KK could provide help: • Four in ten (39%) girls sought further support with physical and menstrual health-related issues – including physical health (15%); nutrition and weight management (7%); BMI (5%); hygiene (4%). • 11% of girls sought support with personal safety issues, including early marriage (9%) and safety awareness (2%). • 9% of girls sought support with career and training, including with the computer / internet (3%); tailoring (3%); and study (2%). • Four in ten (41%) girls raised issues specific to the KK service itself, including offering cheaper prices, more products, and carrying more equipment and products to the house. 94% 3% Yes No Unsure Refused 84% 8% 7% Yes No Unsure Refused IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 29 2.5 KK intervention As detailed in Section 1, KKs provide information, counsel, products and services to promote girls’ health, safety and well-being through three primary activities: door-to-door (household) visits, yard meetings and school activation sessions31. Here, the KKs’ own experience of delivering these activities is described to illustrate the types of interventions expected to have been delivered to customers32. In Section 2.6 we detail the experience of Adolescent Girl KKs in iSocial’s value chain. Summary of KK Interventions • KKs are reaching far fewer households with more repeat visits than the iSocial business model assumes and this may have bearing on SPRING logframe assumptions of beneficiary reach. • While KKs target the adolescent girl, they talk of transacting with a range of other members of the household, most frequently the mother of the adolescent girl or other women in the households, but also boys. • Some success with sharing information and selling through yard meetings and school activation sessions is evident, though gaining access to schools is seemingly more complicated than had been anticipated. • To date, information sharing has concentrated on the areas of SRH, sanitation & nutrition and to a far lesser extent, mental health and career & training. It will be contingent on iSocial to finalise third party referral agreements to facilitate information sharing (and KK revenue generation) in all of the areas it outlines in its ToC. • A fine balance exists between sharing information to promote the KK business or to promote girls’ well￾being, and it is to be expected that customers will benefit more from KK information sharing than girls that fall within the KK catchment area but who do not transact with the KK. This may mean that the most vulnerable girls will not be benefitting from the KK services. 2.5.1 Door-to-door visits Door-to-door visits are the KK’s primary business activity. Households reached: By their own account, KKs visited between 5-15 households a week, with roughly half of these being repeat visits. Most33 KKs had reached between 200-300 households since they started with iSocial, though some had reached fewer than 100. One KK had reached only 35 households. This is lower than iSocial’s business model assumption that each KK will visit 15-20 households per day and approximately 300 in a month from a catchment of 1,200 households. Planning visits: The KKs spoke of different patterns of organising their working days. Some planned which customers they would target on a weekly basis. This was facilitated by their knowledge of the village and networks to identify which households had adolescent girls. When KKs “[did] not have any direct previous communication with [the household], in the worst case, [the KK would] directly go to the house and introduce [herself]” (KII 2). Other KKs spoke of planning each day according to when the adolescent girls would be available: “I know most of the villagers; if not, I collect information about those families from my relatives and friends. I visit the houses where there are adolescent customers in the late afternoon. Before then, I visit the other households that I know have female members and I try to sell different products to them” (KII 6). Some KKs highlighted a lack of mobility hindered them from venturing further afield to promote and grow their business. Social capital associated with the KKs local knowledge and presence can be expected to help grow her business. Direct and indirect customers: All KKs identified the adolescent girl as their target client but also recounted transacting with others in the household. Mothers, in particular, were often the purchase decision-makers in the household, so even when the KK sold to the adolescent girl, the actual transaction was with the mother. One KK 31 iSocial had planned to also offer a call centre for customers to contact and order products, as well as an anonymous hotline support service for girls, but neither of these services are in operation yet. 32 It should be noted that this is a qualitative account of the KK experience, taken from 10 KIIs with KKs. The primary aim of the KIIs was to gather information to evidence impact on adolescent girls that had been engaged in iSocial’s KK value chain. The KIIs include all three adolescent girls as well and seven KIIs with s a cross-section of all KKs. 33 Note, in this qualitative section where we write up findings from 10 KK KIIs, broadly: ‘Most’ = 8 – 10 KIIs; Many = 6– 7 KIIs; Some = 3-5 KIIs; and a few = 2-3 of 10 KIIs. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 30 justified that “adolescent girls do not have money in hand. That’s why I communicate with the mother. The adolescent girl’s wants and needs are met by her mother” (KII 3). Most KKs also provided products and services to boys, typically selling them mobile phone airtime but also selling toiletries such as shampoo, lotion and soap. Some KKs also mentioned boys took up health services, such as blood and diabetic tests. However, one KK described that as she was unmarried their she was limited in her ability to provide services to boys, that “I have ability to help the adolescent boys, but socially villagers will not take it positively because I’m unmarried.” (KII 4) KKs also mentioned being able to assess and recognise when someone would become a customer or not, saying, “when I talk to them, I can guess rather I will be able to sell my products to them or not” (KII 5). This introduces the element of beneficiary selection into the intervention, which is in some respects an inevitable consequence of delivering interventions through commercial activity. The issue of selection bias is further detailed at the end of this chapter. 2.5.2 Yard meetings A yard meeting is an iSocial term for a gathering of adolescent girls coordinated and planned by the KK, and delivered in her own or someone else’s home or yard, so that she can share a certain topic of information. Reach: By their own account, eight of the ten KKs interviewed had held yard meetings. These KKs had held on average 15 yard meetings each, with the exact number ranging from 2-40 meetings. Location: KKs spoke of meetings held at either their own home or, more frequently, that of another member of the community such as a community leader or Union Parishad, someone in the iSocial network (e.g. relative / hub manager) or a suitable customer. Content: Hub and Territory Managers were cited as providing content for the Yard Meetings and KKs worked closely with the managers to their script for the sessions. KKs recounted positive experiences of teaching girls about their health and of generating business at the same time: “All the yard sessions were successful. The girls’ interaction was very positive and enthusiastic. I delivered all the information correctly. I think the girls have learned a lot from me and I have got instant feedback on the subject matter from them. I also got orders for different products” (KII 4). Audience: KKs also highlighted that girls, their mothers and other relatives were regularly present for the meetings, in doing so widening the KK’s customer base: “The girls really learnt through this type of yard session. My observation is that the girls are always interested to learn about the physical changes at the adolescent ages. Generally, the mothers or guardians of the participants also listen from a distance about what type of information” (KII 6). This again reinforces that while the adolescent girl is the primary focus of the KK, other members of the family, particularly women, were regular customers. 2.5.3 School activation sessions A school activation session is an iSocial term for a coordinated information-sharing session delivered to adolescent girls while they are at school. The session typically consists of a presentation by the KK to share information, followed by a test of information acquisition (through a quiz). The sessions serve to improve adolescent girls’ awareness and knowledge while simultaneously raising the profile of the KK amongst potential customers. Reach: Only six of the ten KKs interviewed had completed a school activation session. The range of frequency varied greatly. Five KKs had completed one or two school activation sessions, while one KK had completed around 20. Permissions: The school activation sessions evidently took longer to organise than other activities (and even perhaps than iSocial had anticipated) as it required the iSocial Hub Managers to obtain relevant permission from government authorities before the KK could approach the schools. This meant KKs were dependent on the Hub manager obtaining the necessary permission in order for her to undertake the activity. As described by one KK: “I went to different schools with the permission letter and requested the Headmaster to provide a convenient date for conducting the blood testing program with awareness building and raising campaign among the adolescent girl students… In this regard, I needed support from the hub manager. The hub manager had to collect the permission letter from the Upazila Health Officer for conducting blood testing activities in different schools by mention my name on it” (KII 2). One of the KKs spoke of frustration that in her district, the permissions had not been obtained and she was thus unable to complete any school activation sessions in her area. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 31 Content: Hub Managers again were cited as providing content, including presentation and quiz material for the session and helping the KK organise it with the school. All of the KKs that had held activation sessions spoke of the positive impact on their business and willingness to organise more sessions. For example, one KK noted: “Such initiative has provided a positive impact on my business. My overall business has increased and it was good publicity for me amongst my target audience. I want to do this type of awareness campaign in different schools. I’ve detail plan on this issue. I will request iSocial to support me in future for organizing this type of event” (KII 2). Effect on sales: Not all information-sharing led to direct sales for the KK, as awareness raising led adolescent girls to purchase products elsewhere. KKs still perceived it to be good for their overall business. One KK shared: “I talked with the 11-16-year-old adolescent girls. I gave them my contact number. They never call me. I contact with them again. They have learnt a lot from me. They told me that they started to purchase the sanitary pads after words” (KII 6). 2.5.4 Topics of information shared When prompted to specify the topics of information they had shared in their door-to-door visits and information￾sharing sessions, nearly all KKs indicated that they had shared information on SRH, sanitation and nutrition; slightly fewer had shared information on mental health, and only half had shared information on career and training opportunities. As content was provided by the Hub and Territory managers, it is to be expected that iSocial controlled the topics of information shared. 2.5.5 Beneficiary or customer? The progression of information-sharing leading to service and product sales was implicit in KKs interactions with beneficiaries and customers. For example, an information-sharing session on menstrual hygiene and the use of sanitary pads instead of cloth would lead to greater sales of sanitary pads. This created a fine balance between sharing information purely to promote their business or to promote girls’ well-being. Encouragingly, most KKs identified strongly with both their intention to sell their products and services and with their role as an advisor and mentor to the adolescent girl. One KK stated: “I’ve found these types of discussion sessions very useful as the adolescent girls and women show their interest to buy different products from me… [but] I’m working as a Kallyani to solve the problems of the adolescent girls in my territory” (KII 1). This was also evident when the KKs were asked about what they saw as their primary role. Without prompting, they identified it as the provision of health and information services and product delivery to adolescent girls, with some highlighting their efforts in awareness￾raising in key areas of sexual and reproductive health (SRH) and mental health. For example: “As a Kallyani my primary role is to sell products and provide health services to adolescent girls. I also deliver various types of information to make them knowledgeable on basic health related issues. These are menstrual hygiene, nutrition, health services. I also advise them if they share any problems with me” (KII 6). The KKs’ own account of the products and services they had sold did suggest they were, in part, addressing the needs of the adolescent girl, as the most common items sold were sanitary pads, soap, shampoo, toothpaste and a range of health tests (BMI, blood, albumin, and less often, diabetes).34 Mention was also made of third party referral to Aponjon services35. KKs were evidently also meeting the needs of others in the household, as many of their products and services were relevant to a wider customer base. Other products sold included oral rehydration salts (orsaline), clothes washing powder and phone airtime. This is explored in more detail in Section 2.4. Based on the KKs’ account of their activities, one would expect to see some evidence of KK activity and some mention of the KK as a source of information in the areas of physical health, SRH and menstrual hygiene amongst KK customers, but less so in areas of mental well-being and career and training services. However, at baseline data collection, the aim is to capture and measure girls’ awareness and knowledge as close to the start of business intervention as possible to monitor changes over time. At this early stage in their business, it is not expected that KKs will be regularly referenced as a source of information or influence, though it is projected that this will increase at endline data collection. 34 To some extent, the nature of products sold would have been dictated by the ‘basket of goods’ the KK had available and this was a range of products that, through SPRING girl research, had been identified as relevant to the girl. 35 Aponjon is a mobile health service for expectant and new mothers in Bangladesh. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 32 2.6 Adolescent girl Kishori Kallyanis in the value chain iSocial’s ToC asserts that some adolescent girls will benefit from taking up the opportunity to run their own KK micro-franchise, and so learn, earn and enjoy improved well-being. iSocial did not purposively target adolescent girls to become KKs, and by May 2018, only three girls had taken up the opportunity to become a KK. The following section explores KKs’ experiences of working with iSocial, with particular emphasis on the adolescent girl. As iSocial had only recruited three adolescent girl KKs at the point of baseline data collection, adolescent girl feedback is combined with that of the other KKs and report on consistency and divergence from the experience of KKs overall. This chapter first details what KKs were doing before they joined iSocial, how they came to know about iSocial and their overall experiences and challenges so far. It then details what KKs have learned through iSocial, and finally, what KKs have earned. Summary of findings on adolescent girls as KKs in the value chain • At the time of baseline data collection, only three of the active KKs were adolescent girls. KIIs were undertaken with all three adolescent girl KKs and a further seven KIIs were undertaken with a representative spread of the other active KKs. To improve robustness of findings and provide anonymity, the experience of the adolescent girl KKs is presented in the context of feedback from KKs overall. • three of the active KKs were adolescent girls. KIere undertaken with all three adolescent girl KKs and a further seven KIIs were undertaken with a representative spread of the other active KKs. To improve robustness of findings and provide anonymity, the experience of the adolescent girl KKs is presented in the context of feedback from KKs overall. • Girl enjoy working as KKs and talk of improved self-confidence;enjoyment of information sharing ,and of improving customer well-being. Adolescent Girl KKs have also learned more about their physical, menstrual and mental health. • KII participants have acquired business management skills, though all worked before becoming a KK so this is not their first income earning experience. • Despite enjoying the job, all KKs, including adolescent girl KKs, do not feel they are earning enough for the effort they make to deliver their business. Many KKs, and all adolescent girl KKs, question the long-term viability of their business. • With reference to the iSocial business model, KKs are reaching far fewer households and beneficiaries than iSocial anticipates. KKs revenue is also currently reliant on sales of a high level of customer service to sell low unit priced products and services, with consequent small commission earnings. Some of this may change as the KK becomes more established and is able to expand her customer base and reduce the visiting time with each customer. • Digitisation of the service appears slow in taking off and currently imposes an additional layer of reporting activity on the KK instead of helping her to run her business. • Some aspects of the prototype have yet to be launched – hotline, call centre, paid activation sessions – these additional prototype services are important part of delivering the full suite of intervention iSocial is aiming to. 2.6.1 Access to employment Before iSocial Six of the ten KKs interviewed are married with one or more children. None of the adolescent girl KKs are married. Prior to starting with iSocial, the KK households’ primary source of income tended to be from farming (led by their husband of father) or a family / father’s small business. As described by one KK: “My father is a small businessman. He moves around the village with his rickshaw van for selling daily required food items like flour, rice, molasses, etc. He was the main source of income of our family and still he is. The economic situation in our family has not changed at all, even after I’m working as the Kishori Kallyani” (KII 6). Before joining iSocial, none of the KKs had been the main income earner. However, all except one of the KKs had done some type of income generating activity (IGA), and all of the adolescent girl KKs had been involved in an IGA before iSocial. These activities included: teaching (for BRAC schools); private tutoring; working in the family business; sewing; and one KK had worked for Dnet (iSocial parent company). As would be expected, prior to joining iSocial, the older KKs tended to have done more than one IGA in their lifetime, whereas the younger KKs IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 33 were more likely to have been studying as their primary activity. Some participants referenced monthly earnings. Prior earnings provided these KKs with a benchmark against which to assess their iSocial earnings. Finding out about iSocial KKs found out about the opportunity through word of mouth, with primary reference to members of the Union Parishad or friends and relatives who knew of and were involved with the enterprise. For example, one KK said she “heard about iSocial from the Hub Manager. The Hub Manager was my house tutor. He is known to me” (KII 8). Another shared that the Union Parishad Member told her she would be able to earn more money if she joined iSocial: “He inspired me and gave me the application form. I thought my financial problem would be solved if I could work at iSocial.” (KII 6). All of the adolescent girl KKs had heard about iSocial through the Union Parishad Member. The KKs recounted a structured and thorough selection process, including passport and qualification validation and an oral examination, all of which gave the organisation and role credibility and status. Once they had passed the tests, the KKs were required to buy the equipment and license from iSocial for BDT12,500 (USD150). For iSocial, this was a conscious step away from the legacy of the Infolady, where equipment was donor funded and provided at no cost. iSocial hoped that the fee would attract sales agents with a more entrepreneurial and commercial mindset. In practice, however, to raise this sum of money, many KKs, including two of the three adolescent girl KKs, spoke of having to sell family animals, take loans from other family members and even join a local savings cooperative: "I didn’t have money, so we have sold our domestic animals. My elder brother sold his three goats and provides the BDT 12,500" (KII 5). This meant that many of the KKs were in debt when they started working for iSocial. However, all KKs spoke of supportive family structures – mothers, fathers and/ or husbands – encouraging them to join iSocial and helping them to find the registration fee. One adolescent KK highlighted an advantage of iSocial was that she was able to work within the village and not need to seek work in other regions. Experience with iSocial All but one of the KKs described their overall experience of working with iSocial as good. KKs spoke of enjoying meeting new people and gaining confidence through door-to-door visits, activation sessions and by providing health services: “Previously, I couldn’t talk with people and felt shy. Now, I can talk freely with anyone anywhere. Previously, I couldn’t know how to sell products, provide information to people, and to provide health services to the people. Now, I’ve skill on these issues.” (KII 1) “I love working as a Kishori Kallyani. Previously, I don’t know how to talk with different types of people; how to communicate with them, sell goods to them. Now, I’ve learnt from my job about public relations. I’ve got the opportunity to hear their problems and advise them how to get rid of those crisis situations. I encourage them to follow my advice for changing their lives in the right directions. They listen to me, follow my words and I’ve received much respect from them.” (KII 2) One younger36 KK also highlighted the advantage of her age in being able to identify with and understand her customer base, saying that she enjoys her job working with iSocial because she has “very good communication with the adolescent girls [as] most of them are [her] age” (KII 9). KKs enjoyed learning about health and hygiene issues and being able to share this information with others in their community. KKs spoke of enjoying delivering health tests to their community, and a couple mentioned particular enjoyment in conducting blood tests. The role as KK gave them new status in their community, the Kallyani dress made them instantly recognisable, and the health focus of their work was a positive association. One KK said that she “always wear[s] the Kallyani dress” when going out in the village, because when she wears it, other “villagers call [her] ‘Doctor’, and [she] like[s] this approach a lot” (K11 10). When asked what they liked most about being a KK, opinions were fairly evenly split between enjoying their earning and enjoying the social aspect and status of the job, with the younger KKs more likely to site earnings over 36 Aged 21-25 years; so older than SPRING age band but younger than the other KKs. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 34 job characteristics. One KK said that with her earnings, “I’m meeting my basic needs. I also like the opportunity to talk with various types of people in the village and give them health-related advice and services” (KII 4). When asked what their greatest achievement had been to date, KKs most frequently cited awareness-raising amongst adolescent girls to improve their menstrual hygiene practice and, in particular, their success in switching girls from using cloth to sanitary pads. A few KKs also cited the improvement in WASH awareness at the household level and provision of health services (like blood tests) to their community. This was consistent amongst the adolescent girl KKs. Only one KK expressed a bad experience of working with iSocial overall. This was because of the poor earnings the KK had realised since joining, saying “It was a wrong decision for me to involve with iSocial because I was earning more before” (KII 6). Challenges KKs spoke of the biggest challenges they had encountered in their efforts to establish themselves in their communities. Despite the fact that they were often known within the village, they still encountered resistance. Most KKs managed to overcome these difficulties and accepted that not all households would want to be customers. One KK described: “At the beginning, the villagers didn’t cooperate and didn’t want to talk to me. But, I was confident and never disheartened. I kept continuing my job to talk with the adolescent girls and the female members of the households to convey relevant information to them and encouraging them to buy products from me. Of course there are some who do not buy goods from me as they buy from the local market. But, there are also some of the adolescent girl customers that always buy goods from me. It’s an open practice. Nobody told me that you should not come again” (KII 1). However, one KK felt the resistance to her business was too great, explaining that in her community the local shops provided credit to those in her village and allowed for repayment in monthly instalments. “The male head of the households [thus] discouraged their female members” from taking up the KK’s products and services (KII 6). The adolescent girl KKs in particular also encountered cultural barriers in establishing their business, as some of the villagers did not want their families to deal with an unmarried woman, especially when the potential customer was married herself. These married women were “explained everything [by their husbands]” and “[told that] they can buy products by themselves” (KII 4). All KKs also spoke of the large amount of equipment (including weighing scales, kits to deliver blood, diabetes and albumin tests, pregnancy strips, SIM cards, and products including sanitary pads, washing powder and toiletries), in addition to the long distances they had to cover, mostly on foot. This limited the distances they were able to cover. 2.6.2 Learning All KKs recalled a 5-day training on the iSocial products and services and information-sharing initiatives, customer relationship building and how to run their business. During training, they were also taught how to administer the various health services and tests they offered and how to complete forms associated with third party services they offered, such as Aponjon. Of note is that none of the KKs spontaneously mentioned training on targeting the adolescent girl. iSocial had introduced an online app to try to streamline the submission of sales data and stock control. The original idea was that the girls would take the tablet into the field and record the sales in situ, and send these records to iSocial when a signal was available. iSocial’s broader vision was that the tablet would facilitate the introduction of an additional revenue stream for the girls: of information gathering on behalf of other organisations. However, prior to baseline data collection, it was evident that there were issues with tablet uptake: not all KKs had access to a tablet, and even those that did were not collecting the expected fields of data nor providing sales updates via the tablet with the expected frequency, such that customer data was not available online as had been anticipated. In speaking to the KKs, it was evident that the tablet was not being used in field, but instead had become an additional layer of reporting, rather than the primary mode as intended. All KKs spoke of first completing all sales records on paper, then updating the tablet when they were back home. As described by one KK: “At first I keep records of all of my work on the sheet. After that I record all my sales-related information on the smart phone that is provided by iSocial. Though I should submit the sale report on a daily basis, I can’t make it. I try to submit the sales report after every 2 days. Sometimes, it takes 3-4 days due to many reasons, like no internet connection” (KII 2). Many KKs recounted submitting tablet updates on a weekly basis, with some providing the Hub Manager with IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 35 telephone updates in between. This pattern was consistent amongst adolescent girl KKs. One Kallyani spoke of having sent her tablet back for repair 2 months ago with no replacement, while another rejected use of the tablet altogether, saying her “normal practice” was to “take notes on a book, prepare the sales report, and submit [it] to the Hub Manager on a weekly basis” (KII 9). All of this suggests that the benefit of the tablet is somehow not being realised. This, in turn, raises questions over the ability of iSocial to monitor the number of customers the KKs are reaching as the manual records of KK activities are transaction-based. 2.6.3 Earning All ten KKs provided details of their monthly earnings since starting at iSocial. From this it was calculated that KKs earned, on average, BDT1,677 (USD20) per month and a median of BST1,214 (USD14). The three adolescent girl KKs earned on average BDT1,362 (USD16) a month. Despite the positive account of good customer interactions, growth in confidence and achievements in establishing their business with iSocial, the KKs were unanimous in their dissatisfaction with their earnings. KKs said that their income is “not satisfactory [as it] depends on commission, which is not much” (KII 1) and “is very low compared to the workload, time invested and physical stress” of the job (KII 4). One KK spoke of the difficulties in repaying the loan she took to start as a KK in light of her low earnings: “I thought my financial problem would be solved if I could work at iSocial. But the reality is very different. I took a loan [for the iSocial deposit]. Now, I’m struggling a lot to pay the monthly instalment because I need to pay weekly instalment of BDT 325 per week, which I cannot earn at iSocial” (KII 6). Many of those who had worked prior to becoming a KK reported to be earning less than in their previous employment. Others reported to be earning less than they had expected. iSocial operates on a 10% sales commission, and with the most commonly sold items cited as of relatively low value (i.e. sanitary pads, soap, shampoo, washing powder, orsaline), the KKs were struggling to generate a sizeable commission. One KK also perceived that iSocial product prices were not competitive: “The prices of the products are not significantly attractive… for instance, the price of a 12-piece shampoo pack is BDT 10 in the local market. But, iSocial told me to sell it for the price of BDT 11.40. So, no one shows any interest to buy it” (KII 6). A few KKs, including two of the adolescent girl KKs, stated they had been told by iSocial they would receive a monthly salary (of BDT2,500) but since starting had only received commission. All KKs received their iSocial earnings in cash. One KK even suggested being misled, saying that she “receive[s] monthly earning through cash from the Hub Manager though iSocial told [her] she would receive the money through bKash” (KII 2). The fact that payments are in cash does raise some concern over KK safety on receipt of the cash, given the lack of an audit trail that would be left by an alternative form of payment such as a formal bank or savings account. The (commission) payments were received a couple of weeks after KKs had submitted their monthly sales report. Some KKs were unhappy with the amount of time it took iSocial to turn around their commission payment, deeming the delay as “unacceptable” (KII 2). A few KKs were still involved in other income generating activity (such as sewing or private tuition) or were still studying. The adolescent girl KKs had been working for iSocial for between 5-6 months, and one of the girls was still working as a private tutor while the other two were studying alongside working as a KK. KKs’ primary expenditures were on transport costs to reach customers, money for airtime for their phones, contributing to household expenses, contributing towards the cost of their studies, and loan repayment. KKs spoke of barely breaking even between the costs of running their business and their earnings through commission. Poor earnings meant that many KKs questioned the long-term viability of being a sales agent. While they evidently enjoyed their job, they did not feel that their efforts were adequately rewarded. As one KK said, “If iSocial does not increase the present commissioning system, I may not continue my job for a longer period of time” (KII 5). This raises questions over the long-term sustainability of the iSocial business model. At endline the issues presented here could be explored in more detail. This could include a review of the contracts that KKs have signed. IMPACT EVALUATION – ISOCIAL BASELINE REPORT SPRING MONITORING AND EVALUATION – NOVEMBER 2018 36 3 Next Steps This baseline report will be shared with the donor and IP, then revised and finalised incorporating feedback. Once the report has been finalised, we will share feedback with iSocial and discuss any arising points of interest or questions. During the endline data collection scheduled for April 2020 we will pay particular attention to points of interest and questions identified during baseline data collection and also any subsequent developments in the prototype business model as identified by iSocial and their prototype progress reports (eg KPI Reports). As identified in this report, questions fall into two broad categories. On the one hand, there are questions around the KK customers’ perception of the intervention, including: the extent to which the KK meets the needs of others in the household; what adolescent girl customers understand of why and how they access referral services; how KKs are trained to detect mental health issues and whether KKs have different strategies according to girls age. On the other hand, there are questions with regards to the KKs experiences of the value chain, including: Why KKs are paid in cash and not through a safer source such as a bank account; why some KKs are under the misconception they should be receiving a monthly salary in addition to the commission payments; when iSocial anticipate the full suite of services will be available for KKs and what KK turnover has been, and if we can speak to KKs that have left iSocial. As identified in iSocial’s most recent KPI Report, there are questions around the sustainability of the prototype business model as while KK numbers are increasing significantly, the sales per KK appear to be falling. There is also a significant increase in the number of adolescent girl KKs in the most recent report, and we will seek to increase the number or adolescent girl KKs we conduct KIIs with accordingly. More broadly, endline data collection will trace the KKs progress towards iSocial’s envisaged impact of improving adolescent girls’ agency, ability to follow their aspirations, gain employment, delay marriage and be healthy. The following timetable provides an indicative timeframe for the activities indicated above: Task Completion date Donor /IP review October 2018 Amends and submission of final report November 2018 Feedback to iSocial December 2018 Endline April 2020 IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX A – THEORY OF CHANGE SPRING MONITORING AND EVALUATION – NOVEMBER 2018 37 Annex A. iSocial Theory of Change IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 38 Annex B – Customer Survey Centre For Research & Management Consulting SRG Bangladesh Limited House 12, Road 2/A, Banani, Dhaka 1213, Bangladesh Tel: 55040915-20 E-mail: srgbangladesh@gmail.com, info@srgb.org. Web Site: http://www.srgb.org iSocial Baseline in Bangladesh, 2018 Customer Survey Designation ID Name Signature Field Investigator (FI) Field Supervisor (FS) Quality Controller (QC) Interview Date (Day, Month, Year) Day Month Year Interview Duration (Start-Finish, Consider 24 hours) Start Finish Total House No.: ____________________________________ Road No.: _________________________ Mahalla/Village ______________________________ Ward/Union ________________________________ Thana/Upazilla ____________________________ District _______________________________ IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 39 Hello. My name is _____ and I work for SRGB research agency. I am here today to speak to girls that are customers of the iSocialKishori Kalyani. This [letter from Coffey] explains what I am doing today. Your KishoriKallyani will have shown this letter to you and you indicated that you were willing to take part in the research. The information you provide will help us understand how to build better programmes to improve the lives of adolescent girls. S1 Firstly, can I confirm you know [XXX], your local Kishori Kallyani? 1 Yes Continue 2 No Clarify who Kallyani is and if still does’t know, close the interview S2 Would you be willing for me to interview you? 1 Yes Continue 2 No close the interview Thank you for agreeing to talk with me. We will talk for about 45 minutes. This is not a test and there are no right or wrong answers. You can ask me any questions you have now or during our time talking. The responses that you provide will be stored and analysed by our researchers, and we may share them with others when we report on our findings from this research. However, we will never use your name, so nobody outside the researchers will know that it was you who gave this response. Before we start the interview: S3a Can you please tell me your name? (Parent/ Guardian) S3b And what is your name (adolescent girl)? S3c Can you please tell me how old you are now? (adolescent girl) Year NOTE IF OVER 20 YEARS OF AGE, THANK AND CLOSE Month If the girl is under 14 years of age, the interview will need to be conducted with the parent or guardian’s participation. IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 40 A signature of agreement will need to be obtained from all girls. A parent or guardian’s signature of agreement will also need to be obtained for girls under 18 years of age. If the girl is aged 20 years or older, thank her and close the interview. S4 Can I ask you please to sign here to show that you are happy to take part in this interview? Signature of Guardian Signature of Girl In a year and again in two years time, we will again be conducting interviews for our research. Would you be willing for us to contact you again then? S5 Would you be willing for us to contact you again then? 1 Yes Continue 2 No Thank & Close If yes to recontact: S6 Can you please give me your mobile phone number and that of another member of the household and also you full address Mobile phone number of guardian Mobile phone number of girl/another guardian Full address IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 41 Section A : Warm up & Well-being I’d like to start by asking a few questions about you A1 Are you enrolled in school /some kind of educational training? Education Level 1 No school/educational training Go to [A5] 2 Yes. Primary School Go to [A2] 3 Yes. Secondary / High School 4 Yes. College / University 97 Yes, Other (write in) 95 DK Go to [A5] 96 Ref IF ATTENDING SCHOOL/SOME KIND OF EDUCATIONAL TRAINING A2 What is the name of the school you are attending? A3 How many years have you attended school? A4 What is your highest level of education? [Single answer only] Education Level 1 Someprimary 2 Primary School Certificate (PSC) completed 3 Some years of Junior School 4 Junior School Certificate (JSC) completed 5 Some years of Secondary IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 42 6 Secondary School Certificate (SSC) completed 7 Some years of Higher Secondary 8 Higher Secondary Certificate (HSC) completed 9 Some years of Graduation (Pass Courses) 10 Graduation (Pass Courses) completed 11 Some years of University (Graduation Honors) 12 University (Graduation Honors) completed 13 Some years of Masters/Masters completed 96 Refused A6 Thinking of the last 12 months, have you worked for which you were paid? (Enerumrator is to code appropriate answers. Multiple answers are accpectable. Don’t read the answers.) 0 I didn’t do any work for which I was paid. 1 Sell things for somebody else 2 Planting/harvesting/farm labour 3 Paid for House Cleaning or collecting wood or water 4 Child Care 5 Run my own business (e.g. sell things for myself; tailoring/dress making for money 97 Other (write in) A7 How many people in total live in this household? Number A8 Who else (other than you) lives in your household? (Tick all that apply) 1 Mother 2 Father 3 Mother or Father’s partner 4 Grandmother and/or Grandfather IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 43 5 Aunt and/or Uncle 6 Husband or your partner/boyfriend 7 Brothers and/or sisters 8 My own children 9 Other children 10 Other adults A9 What is your religion 1 Muslim 2 Hindu 97 Other (Write in) A10.a Are you married? 1 Yes Go to A10_b 2 No Go to A11_b 96 Ref Go to A12 A10.b Do you have a children? 1 Yes Go to A10_c 2 No Go to A11_d 96 Ref Go to A11_a A10.c If yes to children, how many children do you have? (write in number)? Number A11_a IF MARRIED, at what age did you get married? (Write in years) Age Go to A11_c 95 DK IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 44 96 Ref A11_b IF NOT MARRIED, at what age would you like to get married? (Write in years) Age Go to A12 95 DK 96 Ref A11_c IF GIRL HAS A CHILD, at what age did you have your first child? (Write in years) Age 95 DK Go to A12 96 Ref A11_d [IF NO CHILD] At what age would you like to have your first child? (Write in years) Age 95 DK 96 Ref A12 I’d now like us to play a game, I have a picture here (SHOWCARD A) which shows lots of people doing different things around a big tree. Think of this tree as the tree of life. I want you to look at this picture and tell me which one of these people you think is you and why? (Record verbatim comments. Don’t say anything regarding the picture. Lets the respondent identifies by her own on which picture she found herself similar and asked why?) IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 45 A13 I now want you to look at this showcard for our next question, it is all about how you are feeling. For each I want you to tell me if you feel very poor, poor, fair, good, very good. Thinking of the past week, how have things been with: 1 2 3 4 5 95 96 Very Poor Poor Fair Good Very Good DK Ref 1 Your health 2 Your mood or feelings 3 School or learning 4 Helping out at home 5 Getting along with friends 6 Getting along with your family 7 Play or free time 8 Getting things done 9 Your love or affection (with your family and friends) 10 Getting or buying things 11 The place where you live 12 Paying attention 13 Your energy levels 14 Feelings about yourself 15 Overall how has your life been SECTION B – GENERAL KNOWLEDGE B1 NUTRITION, HEALTH AND HYGIENE B1.1 I would now like to ask you a few questions about what you know about your PHYSICAL health. B1.1.a First, Do you know your weight? 1 Yes Go to B1.b 2 No Go to B1.d IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 46 95 DK 96 Ref B1.1.b Do you know if this is considered underweight, normal weight or overweight? 1 Yes Go to B1.c 2 No 95 DK Go to B1.d 96 Ref B1.1.c If B1.b = 1, yes to either: How do you know this? (Showcard B) 1 Mother 2 Father 3 Grandmother or Grandfather 4 Other family member 5 Friend 6 Teachers 7 Doctor or Nurse 8 The Kishori Kallyani 9 Radio 10 Leaflet 97 Others (Specify) 95 Don’t know B1.1.d Do you know your BMI? 1 Yes Go to B1.1.e 2 No 95 DK Go to B1.2a 96 Ref IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 47 B1.1.e If B1.1.d=1, how do you know this (showcard B) 1 Mother 2 Father 3 Grandmother or Grandfather 4 Other family member 5 Friend 6 Teachers 7 Doctor or Nurse 8 The Kishori Kallyani 9 Radio 10 Leaflet 97 Others (Specify) 95 Don’t know B1.2.a Do you know your blood type/Group? 1 Yes Go to B1.2.b 2 No Go to B1.2.c 95 DK 96 Ref B1.2.b If yes, How do you know your blood type? (showcard B) 1 Mother 2 Father 3 Grandmother or Grandfather 4 Other family member 5 Friend 6 Teachers 7 Doctor or Nurse IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 48 8 The Kishori Kallyani 9 Radio 10 Leaflet 97 Others (Specify) 95 Don’t know B1.2.c Have you had an anaemia test? 1 Yes Go to B1.2.b 2 No Go to B1.3 95 DK 96 Ref B1.2.d If B1.c=1, then, Who gave you this test? (Showcard B) 1 Mother 2 Father 3 Grandmother or Grandfather 4 Other family member 5 Friend 6 Teachers 7 Doctor or Nurse 8 The Kishori Kallyani 9 Radio 10 Leaflet 97 Others (Specify) 95 Don’t know B1.3 Now thinking of your knowledge overall, how much would you say you know about? (SHOWCARD D) 1 2 3 4 IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 49 A lot A fair Amount Not Very Much Nothing at all a The reason why girls and women menstruate b Eating healthily during menstruation c What a normal menstrual cycle is d What to do if you are in pain during menstruation B1.4 Now, I am going to read a set of statements and I want you to tell me whether the statements are True or false 1 2 95 96 True False DK Ref a Menstruation takes iron from your body b Watermelon is a good source of iron c A normal menstrual cycle is 21 days d Using a sanitary pad during menstruation is as healthy as a piece of cloth e Anemia is the result of too little calcium in your body B1.5.a Have you heard of the Marie Stopes Centre? 1 Yes Go to B1.5.b 2 No Go to B1.6 95 DK 96 Ref B1.5.b How do you know about it? (Showcard B, but do not prompt) (Tick all that apply) 1 Mother 2 Father 3 Grandmother or Grandfather 4 Other family member IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 50 5 Friend 6 Teachers 7 Doctor or Nurse 8 The Kishori Kallyani 9 Radio 10 Leaflet 97 Others (Specify) 95 Don’t know B1.6 Have you received information about your menstrual and sexual and reproductive health, in particular from any of these sources? (Tick all that apply) (Showcard B, but do not prompt) 1 Mother 2 Father 3 Grandmother or Grandfather 4 Other family member 5 Friend 6 Teachers 7 Doctor or Nurse 8 The Kishori Kallyani 9 Radio 10 Leaflet 97 Others (Specify) 95 Don’t know B2: MENTAL WELLBEING – SUPPORT NETWORKS AND SOURCES OF INFORMATION Now thinking about your feelings, looking at this [SHOWCARD C]. B2.1 During the past 12 months how often have you…. 1 2 3 4 5 95 96 IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 51 Never Rarely Someti imes Most of the Time Alwa ys DK Ref a Been so worried about something that you could not sleep at night? b Felt lonely? c Felt good about your abilities compared to others? d Felt that you can be yourself around other people? [for translation: in presence of others do you ever feel you need to change who you are] e Felt useless? f Felt other people like you? (are you liked by others) B2.2 Who would you speak to in the following situations? [Showcard B: Source of information] (List all that apply) Showcards B DK REF a Changes to your body (e.g developing breasts or starting menstruation) b Arguments with your friends c Being bullied by others in your village d Unwanted attention from boys in your class/ neighbourhood e what to do or who to approach if you or a friend are feeling very worried, sad, lonely or afraid B2.3A Do you know of any organisations that could provide help and advice to you or your friends if they were feeling very worried, sad, lonely or afraid? 1 Yes Go to B2.3B 2 No Go to B3.1 95 DK 96 Ref IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 52 B2.3B Please mention the name of the organization(s). (Multiple answers are acceptable.) If the respondent knows the name of the organization(s), ask B2.3c, otherwise go to B3.1a. B2.3.c How do you know about it? (Showcard B, but do not prompt; multiple answers is acceptable.) 1 Mother 2 Father 3 Grandmother or Grandfather 4 Other family member 5 Friend 6 Teachers 7 Doctor or Nurse 8 The Kishori Kallyani 9 Radio 10 Leaflet 97 Others (Specify) 95 Don’t know B2.3.d Have you or one of your friends ever contacted it? 1 Yes 2 No 95 DK IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 53 96 Ref B3: CAREERS AND CONNECTIVITY I would now like to ask you a few questions about your education. ENUMERATOR NOTE: B3.1 – B3.3: ASK ONLY OF GIRLS STILL IN EDUCATION OR TRAINING Otherwise Go to B3.4 B3.1a Are you aware of any scholarships available for your education? (explain IF NECESSARY: a scholarship is a grant or payment to support a students education awarded on the basis of academic or other acievement? 1 Yes Continue 2 No Go to B3.4 95 DK B3.1b What is it called? (Write in) If the respendent know the name of the scholarships then ask B3.2, otherwise go to B3.4a. B3.2 How do you know about this scholarship? [Showcard B: sources of information] [ENUMERATOR TO RECORD ALL SOURCES MENTIONED] 1 Mother 2 Father 3 Grandmother or Grandfather 4 Other family member 5 Friend 6 Teachers IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 54 7 Doctor or Nurse 8 The Kishori Kallyani 9 Radio 10 Leaflet 97 Others (Specify) 95 Don’t know B3.3 Are you considering applying for it? 1 Yes 2 No 95 DK B3.4a Are you aware of any education related competitions (e.g Maths or Science Olympiads or spelling B) 1 Yes Go to B3.4aa 2 No 95 DK Go to 3.5a B3.4aa Please mention the name of the Compitition. If the respendent know the name of the scholarships then ask B3.4b, otherwise go to B3.5a. B3.4b Are you considering applying for it? (Or any of them?) 1 Yes 2 No 95 DK IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 55 B3.4c How do you know about these competitions? [Showcard B: sources of information] [ENUMERATOR TO RECORD ALL SOURCES MENTIONED] 1 Mother 2 Father 3 Grandmother or Grandfather 4 Other family member 5 Friend 6 Teachers 7 Doctor or Nurse 8 The Kishori Kallyani 9 Radio 10 Leaflet 97 Others (Specify) 95 Don’t know [ENUMERATOR NOTE: CHECK SC3 AND CONTINUE IF GIRL IS 16+ YEARS OLD, IF UNDER 16 YEARS OF AGE GO TO SECTION C] B3.5a Do you know of places where you could obtain further training to help you with your future work opportunities? 1 Yes Go to B3.5b 2 No Go to B3.7 95 DK B3.5b What is the name of the training provider(s)? (Write in) If the respendent know the name of the scholarships then ask B3.5c, otherwise go to B3.7. B3.5c How do you know of the training provider? [Showcard B: sources of information] [ENUMERATOR TO RECORD ALL SOURCES MENTIONED] IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 56 1 Mother 2 Father 3 Grandmother or Grandfather 4 Other family member 5 Friend 6 Teachers 7 Doctor or Nurse 8 The Kishori Kallyani 9 Radio 10 Leaflet 97 Others (Specify) 95 Don’t know B3.6a Have you applied to a training course? 1 Yes Go to B3.6b 2 No Go to B3.7 95 DK B3.6b Are you considering applying for a training course? 1 Yes 2 No 95 DK B3.7 Looking at the following (SHOWCARD B) have you obtained information about YOUR FUTURE CAREER/TRAINING OPPORTUNTITIES from any of the following [ENUMERATOR TO RECORD ALL SOURCES MENTIONED] 1 Mother 2 Father 3 Grandmother or Grandfather IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 57 4 Other family member 5 Friend 6 Teachers 7 Doctor or Nurse 8 The Kishori Kallyani 9 Radio 10 Leaflet 97 Others (Specify) 95 Don’t know B3.8 And looking at this showcard, [SHOWCARD D] How much would you say you know about the following, for each please tell me if you know a lot, a fair amount, not very much, nothing at all. 1 2 3 4 96 A LOT A Fair Amount Not Very Much Nothing At All Ref a Your future training opportunities b Your future career opportunities SECTION C – ADOLESCENT GIRL PRODUCTS ASK ALL C1 Where do you (or your mother/ guardian) normally buy the following items from? 0 1 2 95 Don’t buy these goods Buy from Kollyani Do not buy from Kollyani DK 1 Orsaline 2 Sanitary pads 3 Hand soap 4 Stationary (Pen or pencil) IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 58 5 Under garment 6 Internet services 7 Sim card / top up cards Instruction: If the respondent buys goods from the Kollyani (2) then go to C2a & c2b and then c4. If the respondent does not buy from the Kollyani (3) then go to C3a & C3b and then c4. C2a Why did you choose to buy these products from the Kishori Kallyani? (Tick all that apply) (DO NOT PROMPT, tick all that apply) (Multiple Response) Reasons 1 It was convenient 2 I can’t obtain the product locally 3 It was a good price 4 I would be too embarrassed to purchase it in a shop 5 It is better quality than the ones available locally 97 Other (Specify) C2b What was the most important reason? Reasons 1 It was convenient 2 I can’t obtain the product locally 3 It was a good price 4 I would be too embarrassed to purchase it in a shop 5 It is better quality than the ones available locally 97 Other (Specify) If girl did not buy anything from the Kishori Kallyani C3a Why have you not bought anything from XX {the Kishori Kallyani]? Multiple Answer Accepted IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 59 1 It is too expensive 2 I can get what I need locally 3 It is poorer quality than the ones available locally 4 They had run out of stock 5 The KK was not available when I needed the product 6 I do not need these products 97 Other (Specify) C3b What was the most important reason? (DO NOT PROMPT) [Most Important] 1 It is too expensive 2 I can get what I need locally 3 It is poorer quality than the ones available locally 4 They had run out of stock 5 The KK was not available when I needed the product 6 I do not need these products 97 Other (Specify) Ask Everyone C4 Have you heard about any of the following services? Name of the service 1 2 98 Yes No DK 1 Aponjon If YES, continue to C5. If NO, go to D1 2 Hello Doctor 3 Green Delta Accident Insturence 4 Green Delta – Nebadita Insurance 5 Ajkerdeal IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 60 C5 How did you hear about the service? (Showcard B) Name of the service Source of info (showcard B) Use Showcards Code 98 DK 1 Aponjon 2 Hello Doctor 3 Green Delta Accident Insturence 4 Green Delta – Nebadita Insurance C6 If bought, Did you buy it? (Tick if YES) 1 2 95 Yes No DK 1 Aponjon 2 Hello Doctor 3 Green Delta Nebadita Insurance 4 Green Delta – Nebadita Insurance TREATMENT GROUP ONLY SECTION D – GETTING TO KNOW THE KISHORI KALLYANI Now thinking about the Kishori Kallyani… D1 How did you first find out about her? 1 She came round to the house to introduce herself 2 She came to school 3 Already knew her through family 4 Already knew her through friends 5 Already knew her from village community 97 Other (Specify) IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 61 D2 How long have you known her now? Years Months D3 How often does she visit the house? (Tick only one) 1 Every week or two weeks 2 Every month 3 Every two or three months 4 Less often 5 Only visited once D4 Who does she visit in your home? (Tick all that apply) 1 Me 2 Mother 3 Father 97 Other (specify) D5 Thinking now about the information that the Kishori Kallyani shares with you… 1 2 95 96 Yes No Don’t Know Refused A Do you trust wht the kallyani tells you? B Hs the information provided by the Kalyani changed the way you go about your life? D6 What is the most important thing you have learnt from the KK? (Open Response) IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 62 D7 Are there any other ways she can help you? What are they? (Open Response) SECTION E: HOUSEHOLD DEMOGRAPHICS We are near the end of the survey. Lastly, I would like to ask a few questions about you and your household. E1 Thinking about the difficulties you may have doing certain activities because of a HEALTH PROBLEM, please look at this [SHOWCARD F] and for each activity, tell me if you have: ‘no difficulty, some difficulty, a lot of difficulty, cannot do at all’. 1 2 3 4 98 No difficulty Yes, some difficulty A lot of difficulty Cannot do at all Don’t Know 1 Do you have difficulty seeing, even if wearing glasses 2 Do you have difficulty hearing, even if using a hearing aid? 3 Do you have difficulty walking or climbing steps? 4 Do you have difficulty remembering things or concentrating on activities?? 5 Do you have difficulty with self-care (e.g. dressing and being able to feed yourself) E2 And can I confirm a few questions about your household (Tick one per question) E2.1 How many household members are 12-years-old or younger? Value 1 Three or more 2 Two IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 63 3 One 4 None E2.2 Do all household members ages 6-to-12 currently attend a school/educational institution? Value 1 No 2 No one 6-12 3 Yes E2.3 In the past year, did any household member ever do work for which he/she was paid on a daily basis? 1 Yes 2 No E2.4 How many rooms does your household occupy (excluding rooms used for business)? Value 1 One 2 Two 3 Three or more E2.5 What is the main construction material of the walls of the main room? 1 Hemp/Hay/Bamboo or other 2 Mud, brick or CI sheet/wood 3 Brick/ Cement E2.6 Does the household own any televisions? 1 No 2 Yes IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 64 E2.7 How many fans does the household own? 1 None 2 One 3 Two or more E2.8 How many mobile phones dose the household own? 1 None 2 One 3 Two or more E2.9 Does the household own any bicycles, motorcycle/scooters, or motor cars etc.? 1 No 2 Yes E2.10 Does the household own (or rent/sharecrop/mortgage in or out) 51 or more decimals of cultivable agricultural land (excluding uncultivable land and dwelling-house/homestead land)? 1 No 2 Yes Thanks for giving us your precious time and information. (Hopefully, we will talk to you next year again.) Instruction: Go to the 1st page and write down the end time on the box. Enumerator feedback NOTE: THIS SECTION IF FOR THE ENUMERATOR TO COMPLETE ONCE THE INTERVIEW HAS ENDED F1 Was guardian present throughout interview? 1 Yes, throughout the interview Continue IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX B – CUSTOMER SURVEY SPRING MONITORING AND EVALUATION – NOVEMBER 2018 65 2 Yes, for some of the interview Continue 3 No,but other people were around Continue 4 No, it was just the respondent and I Go to F3 97 Other (specify) Continue F2 If guardian/others were present; how much do you think did they influenced the respondent? 1 A lot 2 A little 3 Not at all F3 Why do you say this? (specify) F4 Do you have any other comments? (specify) Thank you IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEXES – TREE OF LIFE SPRING MONITORING AND EVALUATION – NOVEMBER 2018 66 Annex C: Tree of Life “I’d now like us to play a game, I have a picture here (Figure A) which shows lots of people doing different things around a big tree. Think of this tree as the tree of life. I want you to look at this picture and tell me which one of these people you think is you and why?” (Record verbatim comments) IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEXES – FOCUS GROUP DISCUSSION TOPIC GUIDE SPRING MONITORING AND EVALUATION – NOVEMBER 2018 67 Annex D – Focus Group Discussion Topic Guide iSocial FGD Guide for girls that participate in KK activation sessions - Baseline Purpose: This topic guide should be used to lead Focus Group Discussions with girls that have or will benefit from the Kishori Kallyani school activation sessions at baseline for the Impact Evaluation. Aim: The aim of this Focus Group Discussion is to speak to girls that have/will be part of school activation sessions to see what they know of sexual and reproductive health (SRH), mental wellbeing, and career and training opportunities, and identify their sources of information and support so that we can compare it at endline. Sampling: In total we will complete 6 FGDs with girls across in the districts where iSocial has introduced the KK services: The FGDs will be held at schools that are part of a KK catchment area and will be receiving activation sessions from the KK over the evaluation period. We will complete two FGDs in schools within each of the iSocial’s defined territories of work (Jessore, Bogra, and Jenanda/Kushtia), and a range of ages: • 10-13 years old, x2 • 14-16 years old, x2 • 17-19 years old, x2 Resources to have to hand: • Topic guide and show cards • Audio recording equipment. • Coloured sticky dots • A camera (to photograph the flowers). This can be a phone camera, but the photo should be of high resolution. Section 1. Introduction Enumerator to read to the respondents: Thank you very much for giving us some time to speak today. I’m from the research company SRGB. I am doing research to learn more about what girls know about their physical and mental health and dreams about their future. We would like to find out more about what you and your friends know and what you dream of for your life. We will talk for about 1 hour. If it is okay with you, I would like to record our discussion as well as take notes. Only myself and my close colleagues at SRGB, will hear the recordings. Nobody else will hear the recordings, and no one outside of this room will know what you have said. If at any time you are uncomfortable, we can stop or you are allowed to leave. If anything is unclear, please let me know and ask us any questions you have. Are you okay if we record this? (Yes/No). Section 2. Background / warm up (5 mins) First, to start, we would like to get to know each other. I would first like us to go around the room and if you can tell us your name, age, what year of school you are in, what you like doing most (e.g playing with your friends, reading, watching TV) and what your least favourite thing is. Enumerator who is facilitating should start first. IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEXES – FOCUS GROUP DISCUSSION TOPIC GUIDE SPRING MONITORING AND EVALUATION – NOVEMBER 2018 68 Section 3. Sexual and Reproductive Health (20 mins) I would like to start by talking about what you know of your physical health – in particular, what you know of menstrual health and hygiene. • 3.1 Who knows why girls menstruate? o Probe: How do you know this? • 3.2 What can you tell me about your menstrual cycle? o Probe: How long does it last? How many days between bleeds? What is normal pain? • 3.3 What about how to maintain good hygiene during menstruation? o Probe: is a piece of cloth as good as a sanitary napkin? Why/not? How often should you change it? Why? What else do you know about good menstrual hygiene? o And how do you know this? • 3.4 And do you know anything about the right foods to eat to keep yourself healthy during menstruation? o And how do you know this? 3. Flower map of support: SRH I’d like us to play a game – I want you to draw a flower, which represents you and your friends and the people you have around you to give you information or support when you need to know something about your physical and sexual reproductive / menstrual health and changes to your body. INSTRUCTION TO MODERATOR: Ask them to draw the centre of a flower which • represents them. Ask the girls to draw petals to go around the centre of the flower to represent which people they seek support from. The name/characteristic of the people (for example, parents, friends, teacher etc) should be written inside. Each girl can place a sticky dot in the petal for each source of support (ensures all girls are represented) Then discuss: • Which people are most helpful and supportive and why (characteristics)? • What are the kinds of support most sought? • Are there any kinds of support that are lacking? Section 4. Mental Wellbeing (15 mins) And now thinking about managing mood changes and what to do when you or your friends are feeling worried, sad, angry, lonely or afraid. • 4.1 Has anyone ever spoken to you about what to do when you are worried, sad, angry, lonely or afraid? o What did they say? 4. Flower map of support: mental well-being I would now like us to draw a flower representing you and your friends and the people you have around you to give you information or support when you or your friends are feeling worried, sad, angry, lonely or afraid. Then discuss: • Which people are most helpful and supportive and why (characteristics)? • What are the kinds of support most sought? • Are there any kinds of support that are lacking? Section 5. Career and Training opportunities (15 mins) And now thinking about what future opportunities of employment or training may be available to you as you get older: IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEXES – FOCUS GROUP DISCUSSION TOPIC GUIDE SPRING MONITORING AND EVALUATION – NOVEMBER 2018 69 • 5.1 Firstly, I would like us to go round the group and everyone tell me what they most hope for in their future o Do you wish to marry and stay in the village? If marry – do you have any idea what age you would like to marry? Have children? Any idea what age you would like to have your first child/children? o Do you wish to get a job and support your family? Or get a job and move away from the village? If a job, do you have an idea of what sort of job you might want? • 5.2 Has anyone ever spoken to you about your future career, employment and training opportunities? o What did they say? 5. Flower map of support: career and training opportunities I would now like us to draw a flower representing you and your friends and the people you have around you to give you information or support about their future career, employment and training opportunities. Then discuss: • Which people are most helpful and supportive and why (characteristics)? • What are the kinds of support most sought? • Are there any kinds of support that are lacking? Section 6: Comparing flower maps across the three fields (15 min) I would like us now to gather the three flowers together and compare across them to discuss: • 6.1 How do the support networks differ across the three areas? • 6.2 What are the areas where girls feel most supported and most in need? Section 7. Boys (10 mins) This set of questions asks girls about boys need for information on sexual reproductive health, mental health? This can apply to girls’ brothers, or if research is being conducted at a co-ed school, the boys in their class. I want to now ask you about the information that boys that you might know need. These might be your brothers (or boys you go to school with). • 7.1 In your community, do you think that boys need information on the changes in their bodies (voice breaking and facial hair growth)? • 7.2 And what about mental health? • 7.3 And future training or career opportunities? • 7.4 Do you know if boys already get information or support in any of these areas? Or if they need this as much as girls do? Section 8. Wrap up and close Does anyone have final comments or questions, or think something important has not been said? Enumerators to be sure to thank everyone for their time and for their participation. The enumerators should capture photographs of the map(s) when all the activities are completed. IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX E – DATA SHEET SPRING MONITORING AND EVALUATION – NOVEMBER 2018 70 Annex E – Data Sheet Data Sheet for Interviews with Kishori Kallyanis 1.1 About you. Below please write in an answer to each question Please Write in How old are you now (write in years) Are you attending school/training? (write yes or no) IF YES, what are you studying (write in full) What is your highest qualification (write in full) Are you married? (write yes or no) For how many years have you been married (write years & months) Do you have children? (write yes or no) 1.2 How long have you been working for iSocial? (Tick / write in one box only) 0-3 month 4-6 months 7-9 months 10+ months 1.3 Thinking about the DOOR TO DOOR (Home) visits you make: (Please write an answer for each question) Please write in How many girls to you visit each week? How many of these are repeat visits? How many of these are new visits? (meeting new customers) How many girls overall have you managed to visit since you started? How often do you return to each customer? How many anaemia tests have you done? How many blood tests? Any other health tests? (Please write in) IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX E – DATA SHEET SPRING MONITORING AND EVALUATION – NOVEMBER 2018 71 1.4 How many DOOR TO DOOR, SCHOOL ACTIVATION, and YARD MEETINGS have you done since you started working with iSocial? 0-5 6-10 11-20 20+ Door to Door School Activation Yard Meeting 1.5 What did you speak about in the information sharing/ activation sessions you held? (Please tick all that apply) Door to Door School activation Yard Meetings Sexual and reproductive health Mental wellbeing Training / career opportunities Nutrition Sanitation and Hygiene Other (write in) IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX E – DATA SHEET SPRING MONITORING AND EVALUATION – NOVEMBER 2018 72 1.6 In the past six months, roughly how much money are you earning at iSocial? And where does your earnings come from? (Note: we are not looking for full details of what you have sold, just what you sell the most of) Month Amount (Taka) What from (e.g. sanitary pads, Aponjon, Orsaline) 1 month ago 2 months ago 3 months ago 4 months ago 5 months ago 6 months ago IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX F – KEY INFORMANT INTERVIEW GUIDE SPRING MONITORING AND EVALUATION – NOVEMBER 2018 73 Annex F – Key informant interview guide iSocial Key Informant Interview Guide for Kishori Kallyanis - Baseline Purpose: This topic guide should be used to guide Key Informant Interviews with Kishori Kallyanis at the baseline for the iSocial Impact Evaluation. Aim: The aim of this Key Informant Interview is to explore KK experiences of working for iSocial. We are seeking to find out how KKs are progressing with their business, what they see and understand of their adolescent girl customers, what they have managed to deliver in the months they have been working and what they hope to achieve. Where KKs are adolescent, the guide also serves to explore girls’ experiences of being part of the value chain of iSocial. Please refer to the accompanying data sheet which outlines what the KK has delivered since she began working with iSocial. The interview needs to be structure accordingly to concentrate on the areas that the Kishori Kallyani has made progress. For new Kishori Kallyanis this is likely to be around door to door visits and establishing a customer base as they are unlikely to have done activation sessions. For more established Kishori Kallyanis, we are interested in finding out about all aspect of servicing they are doing. Sampling: 10 KIIs will be completed – and we will Resources to have to hand: • Topic guide • Audio recording equipment Section 1. Introduction Introduce yourself: I’m from the research company SRGB. I would appreciate being able to record our discussions as well as taking notes. Only myself and close colleagues at SRGB will listen to them and then we destroy the recordings in compliance with the data protection act. Is this OK? Yes/no to recording. Confirmation of purpose and outline structure: We are going to be talking about the experience you have had of working for iSocial. The interview will last up to one hour. First, I’m going to ask you a few questions about yourself and what you were doing before working for iSocial, then I will ask you a few questions about how you found out about working for iSocial and your experiences of working for them so far. Section 2. Background/warm up (5 mins) 2.1 Please can you tell me a bit more about yourself: • Probe: How old are you? Are you married? Do you have children? 2.2 What were you doing before you started working as a Kallyani? • Probe: Did you do any paid work before working for iSocial? What was this? What did you earn? Why did you stop doing it? 2.3 What was your household’s main source of income before you started working with iSocial? Has this changed? Section 3. Finding out about iSocial (10 mins) 3.1 How did you first hear about iSocial and how did you contact and apply to be a Kallyani? • Probe: Why did you want to be a Kallyani? Did you face any barriers/obstacles? If so, how did you overcome them? • Probe: Did your family support you? Why/ what were their reservations? How did you manage to find the money to invest in the business? Did you use your own money? Or did you get money from somewhere else? E.g. parents/husband? IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX F – KEY INFORMANT INTERVIEW GUIDE SPRING MONITORING AND EVALUATION – NOVEMBER 2018 74 3.2 Did you receive training to become a Kallyani? • Probe: Can you describe it to me? Has this helped? In what way? Are there areas that you still need training? What are they? Section 4. Working for iSocial (15mins) 4.1 How long have you been working for iSocial? • Probe: How many days a week to you work? How does it fit in with your life? 4.2 And overall, how are you finding working for iSocial? • Probe: good, supportive, fun; challenging? 4.3 What do you like most about working as a Kallyani? And what do you like least? What has been the greatest achievement for you? • Probe: are there any instances where you can see you taught a girl (or her guardian) something new and changed the girl/guardian’s behaviour? Can you describe it? 4.4 What do you see is your primary role as a Kallyani? • Probe: is it to provide products or services (like health tests) to girls that would otherwise not have access to them? Is it to provide information to girls and educate them on issues such as Nutrition, WASH and menstrual hygiene? Is it to act as informal counsel? Is it all of these? Which one the most? 4.5 Who do you speak to at iSocial? How often? What do you talk about? • Probe: Do you feel you get enough support from iSocial? Are there instances where you feel you need help and don’t have it? 4.6 How do you manage keeping track of your activity each day/week? • Probe: Do you record straight onto your iSocial smartphone or do you record in pen and paper first and then add the details to your phone? How often do you update the phone with your sales details? What information do you give to your hub manager? And how often do you provide your hub manager with updates? 4.7 Have you had any contact with the new call centre? Can you describe it to me? Section 5. Door to door visits (15 / 30 mins depending on KK data sheet) 5.1 You said you visited (XX homes – from datasheet) – can you describe a typical Household visit to me? • Probe: How do you first introduce yourself to the household? And how do you explain your business? • Probe: What equipment do you have to take with you for your visits? How do you manage to carry it? Do you cycle? Do you use the equipment? Are you comfortable using the equipment? • Probe: Who do you mainly speak to in the household – girl or mother/carer; and who makes the purchase decisions? • Probe: who do you see as your primary customer? how is relationship building is going? How do you know when things are going well? And not so well? 5.2 How do you decide which households to visit? Do you already know these people? How? E.g from friends networks? From neighbourhood (local) knowledge? Cold calling? 5.3 And how would you say the home visits are working? Do you have enough time to visit the girls? Are you making enough contact? Building relationships? 5.3.1 How much time (on average) do you spend with each girl customer? 5.4 What do you tend to talk about? IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX F – KEY INFORMANT INTERVIEW GUIDE SPRING MONITORING AND EVALUATION – NOVEMBER 2018 75 • Probe: have you done any targeted information sharing? For example, information on menstrual hygiene or mental wellbeing [INTERVIEWER TO REFER TO Q1.5 OF DATA SHEET] and ask for up to three areas: • What did you say (what information did you share)? How did you know what to day (Who told you what to say?) Who in the household did you share this information with? What was their response? Do you think they are learning from you? How do you know this? (any examples where girls/carers behaviour has changed?) 5.5 Have you had any experience of girls not wanting to be a customer? Who are most open to being customers? And who least? Are there any households you no longer visit? Are there some you visit less frequently than before? • Probe: do you think it is an adult (mother/guardian) in the household that decides you should not visit again or is it the girl? 5.6 Have you had any experience of boys wanting to be customers? • Probe if yes: Do you feel able to support these male customers? Section 6: Activation sessions (15mins/skip if KK has not done any activation sessions yet) SCHOOL ACTIVATION SESSIONS 6.1 You said you had done XX SCHOOL activation sessions (taken from 1.4 of data sheet), can you describe your last activation session to me: Who at iSocial told you how do it? • Who provided the information? How did they provide the information/content? Did they train you on what to say? Did you feel comfortable with the information you were sharing? • Organising: How did you organise the activation session? How did you inform the participants? 6.2 How did it go? What was the reaction? Do you think the girls learnt anything from it? How do you know? Did you get any business out of it? Do you think you will get any business out of it? Do you see the point in doing them? What is it? Do you plan to do more? YARD MEETING 6.3 You said you had done XX YARD MEETINGS (taken from 1.4 of data sheet), can you describe your last activation session to me: Who at iSocial told you how to do it? • Who provided the information? How did they provide the information/content? Did they train you on what to say? Did you feel comfortable with the information you were sharing? • Organising: How did you organise the activation session? How did you inform the participants? 6.4 How did it go? What was the reaction? Do you think the girls learnt anything from it? How do you know? Did you get any business out of it? Do you think you will get any business out of it? Do you see the point in doing them? What is it? Do you plan to do more? Section 7: Earnings (15mins 7.1 How much money are you earning at iSocial? Take me through each month you have worked – what did you earn in xx, and what was this from?; and then in xx what did you earn? And what was this mainly from? • Probe:: What do you make the most money from and what the least? What are the favourite ways you like to earn money? Why? And what are which activities do you like the least to earn money? Why? 7.2 What affects your earning? Do you get a lot of repeat business? On what items? 7.3 What do you spend your money on? 7.4 Thinking about the money you earn: • How often are you paid (weekly or monthly or other)? • How are you paid? (cash/bank account) IMPACT EVALUATION – ISOCIAL BASELINE REPORT / ANNEX F – KEY INFORMANT INTERVIEW GUIDE SPRING MONITORING AND EVALUATION – NOVEMBER 2018 76 • If bank account – is this your own bank account? Whose name it is in? (your own or husband/parent/other) Does anyone else in your family have access to it? If not mentioned before: 7.5 Are you earning as much as you expected? Why? Do you think your earnings are fair for the work you do? 7.5 Are you doing any other work alongside working as a Kallyani? What else do you do? 7.6 Are you still managing to run your household and all other aspects of your life? Do you get any support from others in your family? 7.7 What is your long-term plan? In one years time – what do you want to be earning /selling/ doing? • And 2 years time? • And 5 years time? • Do you think you will still be working with iSocial or will you want to work for yourself once you have saved enough money from iSocial activities? Section 8: Recommendations (5 mins) 8.1 Have you recommended iSocial / becoming a Kallyani to others? Why (or why not)? Have any become Kallyanis on your recommendation? 8.2 Is there anything we’ve not spoken about today that you would recommend to iSocial to improve their business? THANK AND CLOSE