Cohort 2 Programme Performance Evaluation Report SPRING Monitoring and Evaluation The Department for International Development SPRING Monitoring and Evaluation March 2018 Coffey International Development Ltd The Malthouse 1 Northfield Road Reading Berkshire RG1 8AH United Kingdom T (+44) (0) 1189 566 066 F (+44) (0) 1189 576 066 www.coffey.com Registered Office: 1 Northfield Road Reading Berkshire RG1 8AH United Kingdom Registered in England No. 3799145 Vat Number: GB 724 5309 45 This document has been approved for submission by Coffey’s Project Director, based on a review of satisfactory adherence to our policies on: • Quality management • HSSE and risk management • Financial management and Value for Money (VfM) • Personnel recruitment and management • Performance Management and Monitoring and Evaluation (M&E) Heidi Ober, Programme Director Signature: Cohort 2 Programme Performance Evaluation Report COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 Abbreviations and Acronyms BoP Base of the Pyramid BPE Business Performance Evaluation CEO Chief Executive Officer COO Chief Operating Officer DFAT Department of Foreign Affairs and Trade (Australia) DFID Department for International Development (UK) HCD Human Centred Design IP Implementing Partner IE Impact Evaluation KPI Key Performance Indicator M&E Monitoring and Evaluation NGO Non-Governmental Organisation PPE Programme Performance Evaluation TA Technical Assistance ToC Theory of Change ToR Terms of Reference USAID United States Agency for International Development USD United States Dollar VfM Value for Money COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 Contents Executive Summary i 1 Introduction 1 1.1 Background and context of SPRING 1 1.2 PPE evaluation approach and methodology 3 2 Key findings 4 2.1 SPRING’s Theory of Change 4 2.2 Relevance Has SPRING selected the right businesses to achieve programme goals? 6 2.3 Effectiveness How effectively were the different SPRING accelerator inputs designed and delivered? 11 2.4 How effective was HCD at helping businesses refine their SPRING product, service, or business model? 22 2.5 To what extent has SPRING adapted and with what effect? 25 2.6 Efficiency To what extent does SPRING represent good value for money? 28 3 Conclusions 30 4 Lessons and Recommendations 32 Annexes Annex 1: SPRING Theory of Change (February 2017) Annex 2: Programme Performance Evaluation approach and methodology Annex 3: Overview of SPRING businesses in Cohort 2 Annex 4: Summary of key changes made to the SPRING programme between Cohort 1 and Cohort 2 Annex 5: PPE research tools Annex 6: Summary of bootcamp survey findings Annex 7: List of stakeholders interviewed Annex 8: List of documents reviewed and references COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 i Executive Summary Background to SPRING SPRING is a £16 million1 programme supporting the economic empowerment of adolescent girls in developing countries. The programme is being implemented by a consortium of partners led by Palladium involving brand and industrial design experts (fuseproject), a team of global thematic experts, and affiliated country managers. The programme will support four cohorts of businesses to reach vulnerable girls in nine countries spread throughout East Africa and South Asia. Each cohort comprises a number of different processes including rigorous selection process, a girl and business landscaping process, a nine-month accelerator programme consisting of two bootcamps and a business oriented research-in-context phase, and an end of cohort event. Purpose of the Cohort 2 PPE Report Coffey is the independent Monitoring and Evaluation (M&E) Partner for SPRING, conducting an ongoing evaluation of SPRING programme. The evaluation strategy has three interlinked processes; a Programme Performance Evaluation, a Business Performance Evaluation and an Impact Evaluation. This report focuses on the Programme Performance Evaluation of the implementation of the second cohort (Cohort 2) and reports on the relevance and effectiveness of the programme design and delivery process. This report also provides key lessons learned and recommendations from Cohort 2, some of which have already been actioned and implemented for Cohorts 3 and 4. The report builds in evaluative research activities undertaken from September 2016 – August 2017, which includes surveys conducted at bootcamp and end-of-cohort events, key informant interviews with IP team members and Cohort 2 businesses, as well as participation in SPRING￾hosted events. Key Findings On the SPRING Theory of Change A Theory of Change (ToC) assists in clearly mapping out the possible prospective path, with regular reviews to determine if the actual implementation has diverted from the planned route, and why this diversion occurred. Between Cohort 1 and Cohort 2, the SPRING programme underwent significant adaptations, including to the outreach process leading to the selection of businesses, the addition of a second bootcamp and the refinement of the bootcamp curriculum, the expansion of human centred design (HCD) research to all businesses in the form of the “Research-in-Context” phase, and the ways in which funds are disbursed through the Prototype Development Fund (PDF). While the rationale for decisions on changes is often recorded in programme documentation, the processes to complete the programme learning are left undone if they are not also reflected in the ToC revisions. While SPRING continues to deliver on, and often exceed its logframe targets, as an iterative, innovative, learning programme, it has not yet made full use of the ToC learning opportunity, to test hypotheses and assumptions for future iterations of similar programmes. Relevance: Has SPRING selected the right businesses to achieve programme goals? SPRING has implemented a considerably evolved selection process for Cohort 2. Part of the learning between Cohorts 1 and 2 focussed not only on the type of businesses selected, but also on the process the programme followed to identify and interact with these businesses. Cohort 2 included more mature businesses, but maturity was not always the best fit for SPRING. During selection process for Cohort 2, some selection criteria remained from Cohort 1 (medium sized businesses, social component, expansion goals, etc.), while other criteria were added including, business longevity, profitability, team make-up, and history of innovation. An important selection criterion was the inclusion of the existence of a defined prototype. The application of business typologies and the weighting of selection criteria continues to evolve. 1 The implementation contract is worth £ 13 million; the M&E partner contract is worth £3 million. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 ii Although no longer the main criteria for the programme, defining and measuring the extent to which girls reached are from the base of the pyramid (BoP) proved difficult in South Asia. As part of this reasoning, the IP expanded its targets to girls beyond the BoP , and broadened the description of their target group to “vulnerable” girls, to encompass populations of girls identified as vulnerable, but not strictly defined as BoP. 2 While this provides some leeway to businesses to identify a wider range of girls as potential beneficiaries, this continues to be a tension for the selection of businesses for the programme. In Cohort 2 selection, SPRING moved away from selecting (but did not exclude) businesses whose prototypes involve girls in the value chain. While there is still opportunity for the involvement of SPRING businesses whose prototypes involve girls in the value chain, another tension lies in the challenges that these prototypes bring for the programme in achieving its quantitative girl reach target within the programme timeframes. SPRING encountered challenges in meeting an extremely diverse set of business needs across Cohort 2. Given the range of levels of maturity of the Cohort 2 businesses, matching business needs with tailored support provide fruitful but challenging. Effectiveness: How effectively were the different SPRING accelerator inputs designed and delivered? SPRING accelerator inputs can be categorised into three categories: technical assistance (focusing on the HCD process as part of Bootcamps 1 and 2 and Research-in-Context, investment readiness support, and in-country support provided throughout the cohort; financial assistance, through the prototype development fund; and additional support, including in-kind, low, and pro-bono technical and advisory services, and mentoring. • Technical assistance: The shift to two bootcamps has allowed for more effective delivery of technical assistance, particularly with sequencing the HCD process to allow for iteration. The two bootcamps provided important and increased touch points between SPRING and businesses. This was further supported by the rollout of HCD research to all Cohort 2 businesses, to support the prototype iteration and refinement process, although this was hampered by difficulties in the coordination of the aims of and logistics for HCD research. The investment readiness support also underwent restructuring, with revised timelines for the support provided and the addition of an online course. However, the effectiveness of investment readiness will require more than nine months to be realised. The roles and responsibilities of country managers have evolved and become more defined in Cohort 2, which has supported greater effectiveness across a number of other different elements of technical assistance. However, there is now a greater reliance on country managers to provide support outside of bootcamps, in order for businesses to achieve SPRING goals. • Financial assistance: Learning from Cohort 1, SPRING moved towards a differentiated funding model, selectively providing financial assistance to SPRING businesses. These changes have helped SPRING focus on generating benefits from technical assistance by focusing funding (and thus activities) on prototype development and launch. However, challenges remained in ensuring that the messaging to businesses on this shift was clear from the outset. • Additional support: SPRING’s approach to additional support is now more focused and tailored. Additional support was provided ad-hoc through a mixture of global and local efforts, although ultimately the extent of support provided for Cohort 2 was relatively limited in comparison to the other types of SPRING inputs. However, a positive unintended product of SPRING inputs has been the formation of peer networks across cohort participants. Effectiveness: How effective was HCD at helping businesses refine their SPRING product, service, or business model? SPRING utilises HCD as a method to guide businesses through identifying adolescent girls as beneficiaries of their businesses, and to assist them in understanding this group’s real-life behaviours, attitudes, and needs. SPRING has undergone a number of adaptations since Cohort 1 that help the programme better deliver technical assistance for SPRING businesses to use the HCD process. Prior to the SPRING programme, most businesses had little to no understanding of HCD. Beyond the SPRING prototyping process, entrepreneurs were generally positive regarding HCD, explaining how the approach and the related support has helped them to think differently to understand the needs of the end user, 2 Coffey, SPRING Programme Performance Evaluation – Cohort 1 Review, May 2016 (Section 3.3) COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 iii to generate ideas, and solve design challenges. While the general overall sentiment towards HCD amongst entrepreneurs was positive, some entrepreneurs did note the limitations they faced in using HCD. HCD was recognised as being time and cost intensive, and only able to focus on small groups of users. Effectiveness: To what extent has SPRING adapted and with what effect? Building on the mechanisms that the programme has put in place for continuous formal and informal learning, the SPRING consortium has adapted the programme based on learnings from Cohort 1 ahead of the implementation of Cohort 2, and during the process implementing Cohort 2 as well. SPRING’s adaptations have been driven by the needs of businesses to engage with the programme to launch prototypes. Changes to selection criteria focus on selecting businesses that are more likely to benefit form, or take up, the HCD process including being able to iterate on prototype development. The roll out of HCD research across all businesses, and the implementation of a second bootcamp are also focused on improving the effectiveness of businesses’ use of the HCD process to develop prototypes. However, there has been less focus on adaptations that address concerns about girl impact. There are three barriers to adaptation that persist: leadership and coordination; logframe indicator target numbers, and the timeframes imposed on the programme. Efficiency: To what extent does SPRING represent good value for money? A full Value for Money (VfM) analysis will not be possible until full financial, impact, and beneficiary data are collected, which will not be available until the final cohort of the programme. As part of a preliminary VfM narrative, we have identified key factors affecting VfM in the programme thus far: • Economy: This includes the increase in cost drivers, notably a second bootcamp and HCD research for all businesses. The changed approach to the distribution of funds and fundraising has also meant that that these are needs-based and differentiated, targeted specifically towards prototype piloting and launching. • Efficiency: In the context of short time frames, SPRING has been able to deliver a large amount of programming. Several inefficiencies did emerge, including inefficiencies at bootcamps (e.g. too many staff present) and through the Research-in-Context phase, where in some cases, research did not necessarily yield insights. • Effectiveness: The majority of businesses expressed satisfaction with SPRING’s technical and financial assistance, citing positive effects on their understanding and use of HCD – despite some frustration regarding the quality of HCD research conducted between bootcamps. While there is strong evidence of the programme-level reflexive and iterative adaptation both between and during cohorts, these decisions are driven by selecting businesses who are likely to succeed in the programme (such as in selecting businesses determined by their size and maturity). • Equity: SPRING has continued to learn from the ways in which girls are impacted through business interventions, although adolescent girls from the BoP are no longer strictly the target beneficiaries in Cohort 2. SPRING has broadened its reach to ‘vulnerable’ girls; however, there is currently no precise definition. Lessons Learned and Recommendations As a “critical friend”, the M&E Partner endeavours to provide regular and constant feedback to the IP. Lessons learned and recommendations are provided informally throughout the cohort and formally in evaluation reports. The IP has taken on board many of the recommendations set out in the Cohort 1 PPE report, and continues to apply recommendations as they emerge. Summaries of lessons learned and recommendations are provided below, with more detailed information provided in the relevant sections. They are organised by OECD-DAC criteria. Cross-cutting 1. The Theory of Change is a tool for implementation, learning and evaluation. The programme ToC can be used as a guiding tool for programme implementation and as a reflective tool for charting progress against the planned path of action and plays a more central role in the decision-making process, even if this role is only the recording of changes made. SPRING should more regularly record changes and details relating to the implementation on the programme level ToC. This will give SPRING opportunities to reflect on the planned route of implementation and the rationale for variations to this route. It also provides background for SPRING to test programme changes and iterations against the theory of change assumptions. As the programme heads COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 iv towards its last planned cohort, SPRING should ensure that there is an opportunity to reflect on the ToC before the programme ends to capture learnings from all cohorts and to match evidence against it as well as identify evidence gaps. This recommendation was implemented after the Cohort 2 PPE emerging findings workshop in July 2017. SPRING is in the process of revising the ToC and should continue to implement this recommendation. This recommendation should also be taken up in the design of future programmes. Relevance 2. A multi-engagement and iterative selection process is foundational in the formulation of anticipated outcomes during the 9-month engagement process. The refinement of the selection process includes relying more on local knowledge and switching to a series of interviews rather than a one-off pitch process. SPRING should continue to maintain its multi-engagement selection process. Future programmes should also utilise a series of engagements in the selection process, rather than relying on a one step application process. SPRING has continued to implement (with further iterations and improvements) these principles for the selection process for Cohorts 3 and 4. 3. Key implementation targets should be identified and acknowledged as programme drivers. Programmes face considerable pressure to meet logframe targets, even when they are largely on track to deliver. While clearly defined metrics have a role to play in determining whether programmes have achieved the desired level of success, funders need to be aware of inherent dangers of skewing the incentive for experimental, innovative programmes having to achieve numerical targets. When designing programmes of experimental nature, consideration should be given to the kind of, or range of metrics against which the programme implementation and success are assessed, rather than the use of definitive targets. This recommendation is not for implementation during the lifetime of SPRING and should be taken up in the design of future programmes. 4. With SPRING’s model of using HCD to help prototypes scale, there are limitations and challenges in working with large, mature businesses. In Cohort 2, two of the more established, mature businesses did not participate or benefit as fully as possible from the programme. In selecting and supporting mature businesses, SPRING should develop measures to manage expectations and ensure that all cohort participants receive (and are seen to receive) equitable levels of input and support. SPRING should continue learning from what works and what does not in engaging with larger, more mature businesses. SPRING has already identified the challenges of working with large businesses and have amended their selection criteria to include ‘leadership’ to reflect the need for the SPRING participant to be influential to implement HCD in their business. Funders and future implementers should consider these challenges in the design of future programmes. Effectiveness 5. Tight timeframes (e.g. between cohorts) are a constraint to in-depth learning. The tight cohort timelines have restricted the programme’s ability to reflect and learn as much as possible. While the programme has placed a high value on learning, and works to constantly adapt, these adaptations have not necessarily been tested against a hypothesis but have rather been reactions to circumstance. Future iterations of the programme might consider a longer timeframe between cohorts to allow for more effective programme learning, particularly to test new approaches or are experimental in nature. This recommendation is not for implementation during the lifetime of SPRING and should be taken up in the design of future programmes. 6. Country managers play a central role in the selection process, M&E process, and in providing 1-on￾1 mentoring by bridging the gap between key SPRING touchpoints. The utilisation of local knowledge through the country managers, especially in the selection and support processes was an important component in these aspects of the SPRING programme. SPRING should ensure that the role of country managers is clarified, and that they are supported with adequate resources to support these areas of their role. Improved resources could include increased remuneration, increased hours, and an increased, weighted role in selection. This COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 v recommendation has been implemented. Learning from Cohort 2, SPRING has already clarified the role of the country manager for Cohort 3 and moving forward, and should continue to better resource the role. 7. Programmes with multi-faceted areas of input should acknowledge the multi-faceted outcome areas that can result. SPRING provides three areas of support to its businesses: technical training on the use of HCD, business accelerator support including investment readiness, and a focus on girls. SPRING can provide support to its businesses in all 3 focal areas; however, a danger lies in businesses being more biased in their ability to absorb support in areas with which they are already familiar e.g. investment readiness rather than girl focus. All SPRING businesses receive a level of girl centred programme support. However, given the centrality of girl focus to the programme design, SPRING should bolster its girl-focussed support to businesses and that businesses receive this support post-Bootcamp 2 period and in alumni support. This recommendation should be considered before the end of the lifetime of the programme. 8. Using research within the HCD process can require specialist resources, and requires adequate time and resources dedicated to supporting businesses in this. The challenges included ensuring that moderators are sufficiently trained and well versed in HCD research methodology, that fieldwork is conducted with the right respondents, and that findings are synthesised in time for Bootcamp 2. SPRING should provide greater clarity on the goals and purpose of HCD research to participants, as well as its limitations. This applies to not only the businesses and research moderators delivering the research, but also to the research coordinators. Given the difficulty in implementing well-coordinated but tailored research activities, future programmes should account for this in the design and resource allocation. SPRING continues to iterate on the best way to resource external logistical support for delivering HCD research. Learning from these experiences has been applied for Cohort 3 and ahead of Cohort 4. Part of this recommendation is not for implementation during the lifetime of SPRING and should be taken up in the design of future programmes. 9. Investment readiness support should be both tailored to the different level of needs of businesses, but also be scalable. SPRING has recognised that the entrepreneurs are all in different states of readiness regarding their potential for outside investment, impacting on their ability to absorb and make use of investment readiness support. To address this, SPRING has provided both tailored support from the Investment Director, but also a scalable solution by supplementing this with an online course. SPRING should continue with the provision of one on one support to businesses that are able to benefit at the time of the bootcamp. SPRING should consider allowing all businesses to access the online course support offered by Duke, to be used when the businesses are able to benefit. To facilitate this process SPRING should look to providing some investment readiness support in its alumni programme / local network support. The M&E system should also ensure that it is documenting the investment readiness journey more clearly. Future programmes should consider the need to extend the timeframe for businesses to adequately improve their investor readiness. This recommendation has been implemented and should continue to be considered before the end of the programme. Sustainability 10. To embed HCD in SPRING businesses, HCD training should go beyond the one SPRING entrepreneur. SPRING invests considerable support, time, and resources into training business entrepreneurs in how to use the process of HCD in their businesses. To embed HCD beyond the entrepreneur in SPRING businesses, SPRING should examine how best to provide additional resources and toolkits to entrepreneurs that allow them to share them with their teams. An optional module on how to ‘teach’ HCD could also be added to the bootcamp curriculum or as alumni support for those entrepreneurs who are interested. The is recommendation should be considered before the end of the lifetime of the programme. 11. Sustaining the impact of SPRING on businesses will benefit from clearer plans for alumni support and networks. An alumni component was developed as part of the initial design of SPRING but has never been clearly articulated. SPRING should finalise its post-SPRING alumni support process and communicate this to the businesses in all its cohorts. Objectives of the support could include building business capacity in HCD, COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 vi reinforcing the centrality of girls within the businesses mind-set, and continuing to improve the investment readiness of the businesses. Future programme might consider ensuring that post-programme support is designed and tested earlier in the programme, to allow iterations and adjustments to be made. This recommendation should be considered before the end of the lifetime of the programme COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 1 1 Introduction 1.1 Background and context of SPRING 1.1.1 Policy context and rationale for SPRING The SPRING Business Case estimates that around 250 million adolescent girls (i.e. girls aged 10-19 years) are living in poverty worldwide3 . The Business Case, and SPRING’s subsequent raison d’etre, suggests that access to assets (e.g. physical or financial resources, technology, insurance, or savings) can contribute to the economic empowerment4 of these girls and promote growth. Therefore, the Department for International Development (DFID) and the Nike Foundation, in partnership with USAID and Australia’s Department for Foreign Affairs and Trade (DFAT), are funding the £16 million SPRING accelerator to support the economic empowerment of adolescent girls in developing countries. SPRING aims to catalyse the design of innovative products, services, and business models that can help girls increase their earnings, savings, learning, safety, or well-being, and to encourage wider investment in these areas. SPRING aims to reach 200,000 girls by 2019. SPRING is designed to be an innovative, experimental programme to contribute to ongoing learning. SPRING is being implemented by a consortium of partner organisations including Palladium and fuseproject, an international industrial design and brand development firm and experts in human-centred design (HCD). The Implementation Partner (IP) team includes: a full time SPRING chief executive officer (CEO) and core project team based in London, additional part time support from other Palladium staff, a team of global thematic experts providing short term technical assistance; and local Country Managers. SPRING has supported two cohorts of 18 businesses through a nine-month accelerator programme to date. Cohort 1 included businesses in Kenya, Rwanda and Uganda in 2015, and expanded into South Asia (Bangladesh, Nepal and Pakistan) for Cohort 2. Implementation of Cohort 3 with 19 businesses in East Africa (Kenya, Rwanda, Uganda, and the addition of Tanzania and Ethiopia) will conclude in February 2018. The selection process for Cohort 4 (in South Asia, including the three Cohort 2 countries and the addition of Myanmar) will take place in January 2018, with the cohort scheduled to begin in March 2018.5 1.1.2 Purpose and scope of SPRING M&E Coffey is the independent Monitoring and Evaluation (M&E) Partner for SPRING. Focusing on both learning and accountability, SPRING’s M&E Partner aims to fill gaps in the international knowledge base around what works (and what does not) and why in promoting the economic empowerment of adolescent girls. The M&E of SPRING generates evidence and learning through three components, applied to each cohort: • The Programme Performance Evaluation (PPE) assesses the effectiveness of the programme, provides regular recommendations to adapt the SPRING design and delivery between cohorts, and synthesises findings and lessons learned on programme design and delivery. • The Business Performance Evaluation (BPE) assesses how business performance (e.g. in terms of sales, investment readiness, and number of girls reached) is changing as a result of participation in SPRING. It also generates lessons about what works and what does not in reaching adolescent girls. • The Impact Evaluation (IE) uses longitudinal and quasi-experimental evaluation designs (where possible) to assess whether and how adolescent girls have benefitted from SPRING products, services, or income￾generation opportunities. It will cover two SPRING businesses per cohort and involves tailored research at baseline (i.e. towards the end of the cohort phase) and a follow-up after the end of the cohort (with the exact timelines tailored to each business). Through these three components, the SPRING evaluation adopts a theory-based approach, meaning that the evaluation design and implementation are framed by the SPRING Theory of Change (ToC). We assess whether 3 http://www.girleffect.org/learn/faq 4 Where economic empowerment is defined as a process that increases people’s access to and control over economic resources and opportunities including jobs, financial services, property and other productive assets (from which one can generate an income), skills development and market information. 5 See the SPRING Annual Report September 2016 – August 2017. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 2 SPRING has delivered the types of change set out in the ToC, and achieved its objectives. Taking a theory-based approach also enabled us to explore which aspects of SPRING worked well, which worked less well, and the reasons for success or failure. The M&E of SPRING runs parallel with programme implementation, but will continue for another three years (until June 2022) after the programme itself has ended to capture longer-term impacts. The evaluation is guided by an evaluation matrix, which has identified the core evaluation questions to be answered as part of the evaluation. The core questions were mapped to the OECD-DAC criteria, and to the relevant evaluation component, (PPE, BPE or IE). A set of sub-questions were developed for each evaluation question. These sub-questions are revisited and refined during the course of the evaluation, but the core questions remain intact. 1.1.3 Purpose of the SPRING Programme Performance Evaluation Report SPRING is an innovative and experimental programme which learns and evolves from the challenges it has faced or anticipates it will face. This PPE report, as part of the overall evaluation, tracks and reports on these changes at a programme level, the reasons for these changes, and records, to the extent possible, the impact these changes have had on the implementation of the programme.6 Our PPE aims to: • Assess the effectiveness of the programme design and delivery process; • Document the programme’s journey, including lessons learned and adaptations made; • Capture learning and recommendations that can inform adaptations to SPRING cohorts; • Generate transferable knowledge about what works for policy-makers and practitioners. Building from the previous Cohort 1 PPE report (first draft submitted May 2016 and finalised August 2016), the review period covers the implementation phase of Cohort 2 (September 2016 – August 2017). Within this period, implementation of Cohort 3 began (in May 2017), but will not be covered in this report. In addition, we refer back to early design documents such as the SPRING Business Case that were finalised before the start of the programme, to understand what it was expected to deliver, how, and why. The structure of the report follows the core questions pertinent to the PPE according to the OECD-DAC criteria. These can be found in Table 1 below. Table 1: PPE evaluation questions and structure of the report Key PPE evaluation questions Sections Relevance: How successfully did SPRING target businesses to achieve programme goals? Has SPRING selected the right businesses to achieve programme goals? Section 2.2 Effectiveness: How successfully have SPRING inputs been delivered? How effectively were the different SPRING accelerator inputs designed and delivered? How effective was HCD at helping businesses refine their SPRING product, service, or business model? To what extent has SPRING adapted and with what effect? Section 2.3 Section 2.4 spacing Section 2.5 Efficiency: Was SPRING delivered efficiently and does it represent good Value for Money? To what extent does SPRING represent good value for money? Section 2.6 Key conclusions are then presented with recommendations. Annexes include a detailed discussion of the methodology and its limitations, research tools, and a list of sources. Additional research with selected businesses for the BPE and IE are under way. Sustainability findings are addressed in the BPE report. Impact findings are addressed in the Impact Evaluation Report. While this report has 6 This is distinct from the impact the programme has had on business which is reported in the BPE, and on girls which is reported in the IE. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 3 been informed by the findings that are included in the BPE report, at the time of writing, baseline studies for the IE of Cohort 2 have not yet been conducted. 1.2 PPE evaluation approach and methodology This PPE report triangulates data and evidence from a number of different sources. They include: • Document review (complete list of documents reviewed can be found in Annex 8). • Bootcamp and End-of-Cohort surveys administered to Cohort 2 businesses (sample surveys can be found in Annex 5 and a summary of results in Annex 6). Bootcamp surveys were completed during the final day of both bootcamps (September 2016 and February 2017), while the End-of-Cohort survey was administered as an online survey from August – September 2017. • Key informant interviews (KIIs) with members of the IP team, donors, Cohort 2 businesses (master topic guide can be found in Annex 5, and a list of respondents in Annex 7). KIIs were conducted from May – June 2017. • Other sources of data include observational notes from participation in bootcamps and the End of Cohort event, and ad-hoc discussions with SPRING IP members. A full discussion of the PPE approach and methodology can be found in Annex 2. 1.2.1 Methodological limitations and mitigation strategies • At the start of the SPRING accelerator programme, businesses are often unclear about Coffey’s role as independent evaluators of the SPRING programme, and therefore may have misunderstood or misinterpreted the purpose of their participation in evaluative activities. This was mitigated by providing an introductory presentation to businesses at the start of Bootcamp 1, and by ensuring continuous contact points with key members of the evaluation team throughout SPRING touchpoints. This allowed businesses to become more familiar with the M&E Partner role. • Efforts were made to complete as many interviews in person. This was made possible by timing evaluative activities to the key touchpoints of the SPRING programme (such as bootcamps and the End of Cohort event); however, due to the pressing schedules of the events, facilitating this was not always possible. KIIs were additionally conducted via telephone / skype. This was particularly true for activities with some SPRING businesses; instances in which KIIs or surveys were not completed with businesses are explained in Annex 2. With regard the programme’s VfM, this report provides a summary of the results of the in-kind verification process and other activities to assess the effectiveness of SPRING’s performance against their key VfM indicators. As a full VfM assessment will require full costing information and impact data, this will be conducted towards the close of the programme. Further explanation of this process is detailed in the Efficiency section of this report (Section 2.6). COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 4 2 Key findings This section will begin with a review of SPRING’s Theory of Change (ToC). Following this, we present our findings, structured according to the evaluation questions and organised by the OECD-DAC criteria. Each section provides an explanation of the criteria, and the sub questions / indicators that inform each section. 2.1 SPRING Theory of Change, assumptions, and accelerator design A ToC maps, describes, and illustrates how a programme anticipates change will happen within a particular context, detailing process steps, assumptions and influences. It focuses on why certain actions will invoke or influence change, and ventures what conditions must be present for this change to occur. It records the details of the “missing middle” between a programme’s interventions or activities and the anticipated change and resultant impact. 2.1.1 SPRING Programme Activities This section describes the activities and inputs for Cohort 2, and some of the motivations for the programmatic changes, as a result of learnings from Cohort 1. A comparison of Cohort 1 and Cohort 2 timelines can be found in Annex 4. Further adaptations have been made for the implementation of Cohort 3. SPRING Cohort 2 was preceded by an outreach phase. SPRING was advertised to entrepreneurs through various channels, particularly through its Country Manager network, who reached out to potential candidates to build a pipeline of applicants. During outreach, the team also studied the local ecosystem for entrepreneurs (entrepreneur landscaping) and the needs and lives of adolescent girls (girl landscaping) to inform the selection process and the design of the accelerator curriculum. Outreach was followed by a selection process during which shortlisted candidates participate in a series of (telephonic) interviews with Country Managers and subject-matter experts, leading to the selection of the 18 preferred businesses for the cohort.7 A final list of 18 businesses and several alternates were put forward for a due diligence and commercial landscaping process, before being fully admitted to the programme. The Cohort 2 accelerator programme began with a two-week residential Bootcamp in Kathmandu in September 2016. At Bootcamp 1, SPRING businesses, primarily represented by their founders, co-founders, or CEOs (referred to as ‘entrepreneurs’) were introduced to SPRING’s consortium of global experts, programme team, and country managers. Entrepreneurs were provided with different modules of technical assistance, which included: • A module on HCD and understanding end users, encouraging businesses to think about who their users are, what they need, and how their needs could best be served; • A module on understanding girls as end users, helping entrepreneurs understand the specific needs of adolescent girls, the market size, and opportunities that these girls represent; • A module on understanding the participant’s business ecosystem, to assist them in analysing their competitive landscape and upstream and downstream value chains; • Other HCD modules focusing on business strategy, such as managing stakeholders, mapping the competitive landscape, identifying growth opportunities, and branding; • Training on business prototypes and financial modelling and management; and • Training on SPRING M&E (focusing on KPIs) and developing a business specific Theory of Change. Following Bootcamp 1, businesses entered a research phase (“Research-in-Context”) 8 of gathering data and testing and refining their prototype of a product, service, or business model that could benefit adolescent girls. In a departure from process implemented9 in Cohort 1, businesses worked with local moderators and a research 7 SPRING typically selects a number of additional businesses as ‘back-ups’ in case any of the preferred candidates do not pass due diligence. 8 This phase was titled ‘Research-in-Context’ for Cohort 3. For Cohort 2, this was sometimes referred to as ‘HCD and Girl Research’ in SPRING reports and timelines. This refers to the same phase, but the term ‘Research-in-Context’ is maintained in this report. This is further discussed in Section 2.3.1. 9 The planned process for Cohort 1 was similar to the implemented process in Cohort 2 COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 5 coordinating firm to conduct HCD research to test pre-defined aspects of their prototype. In addition, several businesses were provided with small research grants to help augment their research. The challenges relating to this research process are explored more fully in Section 2.4. Businesses then brought their research insights into a second bootcamp, held again in Kathmandu in February 2017. At Bootcamp 2, businesses worked with SPRING to refine their prototypes based on research findings, and to develop actionable plans to implement, launch, or scale up their SPRING prototype, including investment readiness preparation. Throughout the accelerator programme, businesses received continuous technical assistance through consortium partners, and their local Country Managers. They received both global and locally-based support with regards to investment readiness. Further analysis of the effectiveness of SPRING’s technical assistance is discussed in Section 2.3. Businesses were eligible for financial assistance, primarily in the form of the Prototype Development Fund (PDF) grant. This departed from the model used in Cohort 1, in which a flexible grant of £50,000 was provided to each business. For Cohort 2, funding decisions for each business for the PDF were made by the SPRING consortium, based on each businesses’ needs and capacity, with businesses submitting additional funding applications following Bootcamp 2. A further refinement was that the funding was limited to prototype development rather than allowing a business to spend it on wider business operations. Fourteen businesses successfully applied and received PDF funding, with grant amounts ranging from £23,000 to over £74,000. The PDF process is further discussed in Section 2.3.2. The cohort programme ended with a two-day learning event in each of Cohort 2 countries. Entrepreneurs shared their learnings around engaging adolescent girls, their progress to date, and feedback on the SPRING accelerator. 2.1.2 Changes to the SPRING Theory of Change A ToC should assist in clearly mapping out the possible prospective path, with regular reviews to determine if the actual implementation has diverted from the planned route, and why this diversion occurred. The hypotheses underpinning the ToC should be re-examined and, if necessary revised, based on programmatic learning. In February 2017, the IP revised the programme level ToC10, following input from a December 2016 workshop attended by the donors and the external evaluation team. This ToC reflects two streams of programme activity: the first working with businesses and the second working with a wider “ecosystem”. The revised ToC equates the programme inputs into the businesses and demonstrates that through an application of HCD, these will result in an improved understanding of girls’ needs and improved girl reach. There is little clear indication of the level and intensity of business-related support to be provided to the businesses during the programme. In the first activity stream, the ToC identifies a number of inputs including; research, funding, HCD, prototyping, mentoring and technical support. These inputs are then utilised by the businesses to refine their business model, their product or service. The inputs also aid in the businesses improved understanding of girls’ needs, an improved reach of this target market and improved investor readiness. SPRING has undergone a number of changes in implementation from its original design. Despite that, the main focus has remained to improve the economic lives of adolescent girls. However, the preferred pathway to achieve this objective remains ill-defined. In the PPE emerging findings workshop (July 2017), the M&E Partner presented a number of suggested options to the IP, which serve as pathways to frame and refine the ToC. 11 These pathways12 speak to the design of the inputs, and ultimately the possible (and desired) outcomes. These include the emphasis that SPRING is: • a business accelerator promoting businesses to focus on girls as a potential market; • a girl-focused programme, using businesses as vehicles to impact on their economic lives An HCD initiative, promoting HCD as a means to assist businesses to more readily engage with girls as a market; meeting their needs and improving their lives. 10 See Annex 1 for SPRING’s ToC (Updated February 2017). 11 Since the initial drafting of this report the IP has presented a more refined ToC in the December 2017 donor workshop. 12 In a subsequent discussion with the SPRING team it was discussed that the programme could move on all three directions. The evaluation team agrees with this, pointing out that different businesses may need different levels of engagement, and that the three paths are not identified in the ToC. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 6 • Is the TOC predictive or reactive? The changes applied to the February 2017 ToC speak to programmatic changes and learning resulting from differences in the original planned and actual implemented processes. These recorded changes, however, do not reflect if they are as a result of testing and proving (or disproving) programmatic hypotheses or which hypothesis (and assumptions) have or have not been tested, and are (or are not) still valid. As a result, these ToC revisions record implemented changes, but not whether these have impacted (and to what extent) on the planned delivery processes of the programme. The purpose of the ToC, especially in a learning and iterative programme like SPRING, is to “blaze a trail”, to map out a route from the starting point to the end goal, detailing why changes have had to be made, and the effect that this might have on programme processes. While the rationale for decisions on changes is often recorded in programme documentation, the processes to complete the programme learning are left undone if they are not also reflected in the ToC revisions. In addition, some activities and changes are not clearly recorded in the current ToC and yet, substantial changes have been made to these process as a result of programme learning. Subsequently, the programme changes and factors that influence changes to activities are then also not recorded in the ToC. An example of this is the initial outreach phase, which precedes any cohort activity and influences the selection process. 13 Similarly, changes were made to the prototype funding criteria and process, the research and prototyping process, and the inclusion of a second boot camp, none of which were reflected in the February 2017 ToC, nor are the consequences or perceived impact of these changes on the programme recorded. Following the emerging findings presentation, we re-traced the ToC based on their own understanding of the programme. Through this exercise, the three suggested programme pathways, or priorities (outlined above) were explored, allowing us to examine the ways in which: • SPRING provides business-related technical assistance (as per a traditional accelerator). • SPRING provides the businesses with girl focused expertise. • SPRING provides the businesses with HCD expertise. The distinction between the three pathways speak to the multiple objectives in the design of the programme, which at times are in tension with one another. As a result of these competing routes, the pathway to achieve the ultimate impact of the programme, improving the life of adolescent girls, is not clearly defined or balanced with the other alternatives. While this distinction may appear to be minor in terms of programme design, the implications in terms of selection of businesses, emphasis of content during engagement with the businesses, focus of the research within the different cohorts, and so on all flow from this lack of clarity. While SPRING continues to deliver on, and often exceed its logframe targets, as an iterative, innovative, learning programme, it would be useful for the programme to make full use of the ToC learning opportunity, to test hypotheses and assumptions for future iterations of similar programmes. 2.2 Has SPRING selected the right businesses to achieve programme goals? This section examines the evaluation question for the PPE that falls under the OECD-DAC criteria of ‘relevance’. In this section, we will explore two aspects of the relevance of SPRING: were the right businesses selected, and to what extent has SPRING been relevant to address the needs of its selected businesses. A further evaluation question on the relevance of SPRING businesses (and therefore of SPRING as a programme) to adolescent girls, falls under Cohort 2 BPE Report Section 2.1. 2.2.1 Were the right businesses selected, and in the right way, to achieve programme goals? Businesses are at the core of SPRING’s Theory of Change. The logic of the programme reflects that sustainable, innovative solutions to improve the economic life of adolescent girls will be applied as a result of commercial motivations. It is these same motivations that will assist the businesses to apply their prototype solutions to scale, reaching larger numbers of girls, even as the programme moves on. Business are, in effect, the implementation tools of SPRING. The selection of the correct businesses is therefore essential to the programme’s success. 13 See Table 2 in Section 2.2 for more detail regarding these changes. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 7 SPRING has implemented a considerably evolved selection process for Cohort 2 As a learning programme SPRING has sought ways of reflecting on and improving their own internal processes. Part of the learning between Cohorts 1 and 2 focussed not only on the type of businesses selected, but also on the process the programme followed to identify and interact with these businesses. The differences between the cohorts are detailed in Table 2. Learning from Cohort 1, the IP made the Cohort 2 selection process more efficient, and identified seven specific businesses types for consideration in SPRING. The selection process was also changed; emphasis in Cohort 1 had been on the pitch camp whereas Cohort 2 selection accented a series of interviews with the decision makers in the businesses, and technical experts in SPRING. Greater involvement of Country Managers in Cohort 2 selection also introduced a greater level of local knowledge. Table 2: Selection process differences between Cohorts 1 and 2 Cohort 1 Cohort 2 Outreach emphasis on generating wide interest, including unlikely applications, and meeting targets for quantity of applications. Earlier start allowing more time for applicants Greater emphasis on engagement of CMs in curating a pipeline of strong applicants. Prioritisation of quality over quantity applicants (reflecting a change to the logframe indicator). Scoring criteria based on: viability, potential for impact, and strength of leadership and team Scoring criteria expanded to include: viability, scale potential, interest in girls, innovation, leadership, strength of supporting team, and alignment to SPRING’s objectives A number of start-up businesses / concepts included in the mix, with a focus on idea-stage or validation stage businesses. Application of different criteria (growth oriented, mature businesses, commercial oriented NGOs) Businesses completed online applications. Application forms streamlined, using a more responsive and simpler website. Primary source of final selection took the form of a Selection Camp, where 50 shortlisted businesses pitched their ideas. All 50 shortlisted businesses subject to due diligence, but no due diligence conducted on perceptions of business or risks of the sector. A rolling shortlisting process was applied, using the application of scores through (telephonic) interviews, with subject-matter experts. Final selection decisions made at Selection Workshop, subject to due diligence. Part of the learning between Cohorts 1 and 2 focussed not only on the type of businesses selected, but also on the process the programme followed to identify and interact with these businesses. These were supported by a paper￾based application process, rather than a pitch process that anchored Cohort 1 selection. As one IP respondent reflected, “Cohort 1 selection had been based on a pitch; Cohort 2 selection was based on a series of interviews, scored interviews, and it was driven far more by the country managers and their knowledge of the countries.”14 IP members reflected that the involvement of country managers in the build up to the selection process worked well. CMs for the most part felt that their opinions were taken into consideration, although at times they also felt their opinions, especially when it came to selection decisions, were overlooked. Cohort 2, based on engineered changes to selection process criteria, included more mature businesses, but maturity was not always the best fit for SPRING In the transition from Cohort 1 to Cohort 2, there was substantial debate within the SPRING team about selecting the businesses that were most appropriate for programme purposes, reflecting the tension outlined above. On review, not all of the business selected for Cohort 1 were a good fit for SPRING’s objectives. In a 2016 quarterly report, SPRING reflected: “Cohort 1 businesses [were] primarily early stage, founder-led social enterprises, with many questions about their core business models, which … hampered their ability to reach scale.”15 The selection 14 KII Respondent 1. 15 SPRING Quarterly Report, June 2016 COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 8 process evolved to try and identify more appropriate or “correct” businesses, with an emphasis on scale and girl impact. During selection process for Cohort 2, some selection criteria remained from Cohort 1 (medium sized businesses, social component, expansion goals, etc.), while other criteria were added including, business longevity, profitability, team make-up, and history of innovation. An important selection criterion was the inclusion of the existence of a defined prototype. While Cohort 1 had very few mature businesses and many start-up/ early concept businesses, the makeup of Cohort 2 was evenly divided between mature, growth and start-up/ early concept businesses. This, however led to its own set of challenges including ensuring the bootcamp curriculum was applicable to such a diverse audience. A small number of participants felt that the bootcamp curriculum did not speak to the diversity of businesses and that multiple sessions geared for different business types might have been an appropriate solution. 16 In Cohort 2, it was apparent that for two businesses, the businesses and the programme were mismatched: Engro and Easypaisa. Both of these businesses were of significant size and partnered with SPRING for the development of a small addition to the larger business model. Initially, their size and reach was seen as a boon to SPRING, “because they will bring real scale potential but also potential resources as corporate partners,”17 but the companies’ participation did not live up to expectations. While the businesses were significantly larger than others in the cohort, they were also perceived as being less committed and less responsive to the challenges faced as a result of SPRING participation. Reasons for this include the changing of responsible personnel between bootcamps, a lack of commitment from business decision makers and a perceived lack of buy-in to the SPRING and HCD process; all of which are explored in more in the BPE report. While this might not be indicative of larger businesses in general, SPRING should be aware of these issues to effectively engage with larger, more complex players in future. Identification of selection criteria for businesses in Cohort 2 continue to be refined. While it is difficult to identify exact criteria that a business should meet (example length of time in business, annual turnover figures, qualifications of key staff members, etc.) SPRING have circumnavigated this issue by identifying typologies of businesses where it felt SPRING is able to provide support.18 The application of these typologies and the weighting of selection criteria is not permanent and continues to evolve. Although no longer a main goal, targeting girls at the base of the pyramid proved difficult in South Asia Towards the end of Cohort 1, SPRING realised that their then beneficiary target of Base of the Pyramid (BoP) girls was problematic. “[We] thought we could really reach the poorest or we’d make an effort to reach the poorest and that that has been more challenging than people wanted it to be”.19 Using businesses as a mechanism to reach girls (if not part of the value chain), enshrines the principles of profitability and efficiency, in the reach process, forcing the target beneficiaries to either a) have access to funds or b) be part of a cross-subsidy or social enterprise model. “We’re obviously very passionate about impact, that’s why I’m dedicating my life to this cause. At the same time, I did not want to build a non-profit”. 20 Partially as a result of this reasoning the IP expanded beyond the BoP principle and broadened the description of their target grouping to “vulnerable”.21 While this provides some leeway to businesses to identify a wider range of girls as potential beneficiaries, and this is considered in the selection process, some respondents still feel that this is problematic for the programme: “the focus is more on girls in a more generic way but, by doing that, the risk is that you may support businesses that are really only catering to the girls who belong to better-off families … So you are caught … You may want to service girls from poor families but the reality is that those families don’t have any purchasing power.”22 16 See Section 2.3 and for a comprehensive overview of the business’ perceptions of boot camp, see the BPE Report. 17 SPRING Annex 2: Effectiveness of Outreach KPIs 18 KII Respondent 1. 19 KII Respondent 12 20 KII Respondent 31. 21 Coffey, SPRING Programme Performance Evaluation – Cohort 1 Review, May 2016 (Section 3.3) 22 KII Respondent 6 COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 9 This dilemma is repeated multiple times when examining the Cohort 2 businesses targeting of girls23 and intensified within the South Asia context as a result of cultural and socio-economic sensitivities; including employment of girls and limitations to girls’ movement outside the home. [In] South Asia, we found that there were lots of barriers in reaching girls and the poorer girls would be in more rural areas, perhaps more traditional, more religious, and, so, had a few more gatekeepers that would obstruct us or our businesses from actually engaging with them.24 Girls and the value chain – In? Or out? Donor sensitivity impacted on the selection of businesses in Cohort 2 with some IP respondents feeling that the most appropriate businesses were blocked from selection.25 In Cohort 2 selection, SPRING moved away from selecting, (but did not exclude), businesses whose prototypes involve girls in the value chain, for a number of reasons, most notably the possibility of exploitative labour and concerns regarding the safety of the girl beneficiaries. A donor respondent noted, “The businesses that are about reaching … girls in the value chain or girls as employees, … come with a whole host of other reputational issues “ 26 Many respondents felt that the Cohort 1 lesson, of involving girls in the value chain was important, if only to find out that implementing the process was more difficult than anticipated, testing a programmatic hypothesis. 27 However, some respondents felt that the error was not the inclusion of businesses which involved girls in the value chain, but rather that the maturity and depth of those businesses, “I think we went too early on the first round and we took ventures that were really too far away from being at the place where we could see any kind of skill being achieved.”28 While a number of Cohort 2 businesses benefit girls in the value chain29 the IP has now introduced a bootcamp module on girls’ safety and has had discussions with the M&E Partner on the International Labour Organisation (ILO) distinction between “child work” and “child labour”, where the latter is seen as exploitative and/or unsafe. This leaves the opportunity open for the involvement of SPRING businesses whose prototypes involve girls in the value chain. However, the remaining tension lies in the challenges that these prototypes bring for the programme in achieving its quantitative girls reached target within the programme timeframes.30 Tension continues between selecting businesses for depth versus breadth The experimental nature of the programme has led to some tension between the recruitment of businesses which might deliver significant numbers of girl beneficiaries, allowing the programme to achieve its identified numerical targets, and the identification of other businesses which might have a more significant impact on a smaller client base. The IP and the programme donors are well aware of this tension, and appreciate that the “right” businesses will display a mix of both growth and impact potential. This focus on scale has also subtly shifted the emphasis of the programme away from a need for businesses to focus solely on girls, limiting their customer scope, to one where the focus is rather on businesses providing disproportionate girl benefit. An IP respondent stated, “you want a business to be focused on business and understand that there is a significant opportunity to address girls. You don’t want a girl-focused business.”31 It has led some respondents to recognise that SPRING might be driven by the logframe target of scale, although some argue that this emphasis is not disproportionate to other drivers such as quality and depth of impact.32 This tension prevented some businesses that might have had deep impact on a smaller number of girls’ lives from being selected as a result of them showing limited opportunity to achieve scale. One IP respondent reflected, “[it] has inhibited us from testing things where we might have found some really interesting learnings to think about girl 23 See Cohort 2 BPE Report Section 2.1 for further discussion. 24 KII Respondent 1. Some key learnings in terms of accessing these rural girls are explored in the Cohort 2 BPE Report Section 2.1. 25 “We’ve struggled consistently because most of the promising business in Bangladesh we couldn’t actually bring in because of Nike’s or DFID’s constraints.” KII respondent 7. 26 KII Respondent 14. See also KII respondents 8, 10, 12 and 15. 27 KII Respondents 8, 10, 12, 14 and 15 28 KII Respondent 15. 29 See BPE Report for a detailed discussion of cohort 2 business’ thinking in this regard. 30 KII Respondent 16. 31 KII Respondent 7. 32 KII Respondents 15, 16, and 21. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 10 impact that didn’t have the potential for the immediate scale but that, in terms of really understanding what more robust girl impact looks like, could have given us some good insight.”33 2.2.2 Has SPRING correctly identified business’ needs to achieve its goals? Part of the selection process revolves around the extent to which the SPRING applicant will contribute to overall programme goals. SPRING assists the businesses to do so by providing or facilitating access to technical support in a variety of areas. This technical support however, needs to be appropriate to the businesses and match their needs. A mismatch between the needs of businesses and the provisions of SPRING could lead to a wasted opportunity, with a business dropping out or over-reaching its current capacity; both of which would impact on SPRING achieving its goals. Of the 18 Cohort 2 businesses, 15 of the businesses were working with adolescent girls in some way, prior to SPRING.34 It was found of those 15 businesses, seven reported that they did not have a strong understanding of how they were targeting adolescent girls. • 5 included girls in their business models as employees; • 5 had launched a product or service to specifically reach girls as users or customers; • 3 had refined an existing product or service to specifically reach girls as users or customers; and • 2 had tested a specific product or service with girls as users or customers. When businesses were asked about their motivations for why they wanted to work with adolescent girls, most businesses reported that furthering social impact and reaching new girl customers were strong motivations.35 Businesses also reported that engaging adolescent girls was a means for reaching new female customers overall, as well as recognising that a focus on adolescent girls as a means or way to attract impact investment. SPRING struggled to meet extremely diverse business needs The diverse maturity range of businesses in Cohort 2 created a challenge; one IP respondent stated, “I think this is one of the challenges of SPRING… it’s not like you can just put a template on and say all the businesses need X, Y, and Z. …it’s literally like having 55 different projects and they all have individual needs.”36 In spite of this, business respondents felt that SPRING identified their business needs, for the most part, for where they were at the time of Bootcamp 1 and were able to address or contribute substantially to their business needs. “[In terms] of the help we got, yes, they were pretty spot on. I would say they were on it for seventy percent. I feel like, in terms of the technical help, they were spot on.”37 Given the range of levels of maturity of the Cohort 2 businesses explored above, matching business needs through a single, unified approach was challenging. There were a lot of people who really benefitted from it, like [company name] and other friends which were there with me, but … if they had factored in the fact that a company has different requirements and a corporation has different expectations, it would have been probably better at the selection phase rather than later on finding out.38 A further observation relating to the two more mature businesses was made by one of the respondents. It was felt that although these businesses benefitted from the programme, SPRING may have struggled to understand and engage with the businesses representatives that attended, “[I felt] there was very little interest [from SPRING]… in those two larger businesses. They just didn’t get it. They just couldn’t connect”.39 This might have been as a result of a combination of the SPRING curriculum and format for engagement, together with the businesses’ own challenges of personnel commitment leading to a mismatched attempt at engagement. This will be explored later in this report under Section 2.4. 33 KII Respondent 15. 34 Coffey, Cohort 2 Bootcamp 1 Survey. 35 See Section 2.1 and Section 2.6 of the Cohort 2 BPE report for a more detailed discussion of business motivations. 36 KII Respondent 19. 37 KII Respondent 65. 38 KII Respondent 22 (emphasis added). 39 KII Respondent 2. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 11 2.3 How effectively were the different SPRING accelerator inputs designed and delivered? This section is the first of three sections that explores the OECD-DAC criteria of ‘effectiveness’. In this first section, we will examine the effectiveness of SPRING technical assistance, financial assistance, and other support throughout the nine-month accelerator programme (summarised in Section 2.1). These categories of inputs can be summarised as the following: • Technical assistance: focusing on the HCD process, which includes both providing training in HCD, as well as technical assistance in applying HCD for prototype development. This process mainly occurs as part of Bootcamp 1 and 2, with Research-in-Context in between. Key technical assistance provided also includes Investment Readiness support, which is the focus following Bootcamp 2, and regular in-country support throughout the cohort. • Financial assistance: SPRING businesses have access to grant funds to support research and he development of their prototype. Funds were disbursed following Bootcamp 1 to support Research-in￾Context, and through the PDF application process following Bootcamp 2. • Additional support: includes a variety of other forms of support which are catered to the needs of individual businesses. This can include in-kind, low, and pro-bono technical and advisory services, through the form of mentoring. Finally, this section will examine which inputs (or combinations of inputs) were most effective in supporting businesses performance improvement. Although this section will make reference to important changes and adaptations made to the accelerator design since Cohort 1, these will be more fully discussed in Section 2.4, while Annex 4 provides an overview of the key features of the Cohort 2 programme, with comparisons to Cohort 1. 2.3.1 How effective was technical assistance to help businesses develop prototypes to meet girls' needs? Technical assistance forms a crucial part of SPRING’s offering to businesses. While much of the technical assistance provided is in the form of both providing training on how to use the principles of HCD in one’s business as well as support to use HCD to design a prototype, SPRING provides crucial technical assistance in a number of other areas including investment readiness. The shift to two bootcamps has allowed for more effective delivery of technical assistance, particularly with sequencing the HCD process to allow for iteration One of the most important shifts to in the design of Cohort 2 has been the implementation of two bootcamps. Cohort 1 included a singular two-week residential bootcamp in June/July 2015, followed by a Prototyping-in￾Context phase in October 2015. Prototyping-in-Context was conducted in each of the three Cohort 1 countries over a span of 3.5 weeks. It included six to eight days of support in each country, including conducting research with NGO partners for select businesses (five in total), a one-day HCD workshop for all businesses in each country. Finally, it provided 1-on-1 design hub support with innovation strategists and designers from fuseproject, as well as local accelerator partner support. The shift to the Cohort 2 format of two bootcamps, with a period of Research-in-Context in between, was prompted by firstly the Cohort 1 learning that all businesses should undertake research with girls, necessitating the second bootcamp to help businesses take up research insights to ensure further iteration on their prototype. As one IP respondent reflected, “we created these potential solutions… but we recognised that we needed to equip participants to test what they left Bootcamp 1 with, and then iterate upon it… we needed to build another touch point where we could iterate and refine [the prototype].” 40 As a result, businesses were more able to systematically refine their prototypes, with the support of fuseproject, during Bootcamp 2. The changes, extent of change, and types of change in prototypes as a result of HCD, are further discussed in the Cohort 2 BPE Report - Section 2.2. The addition of the second bootcamp also allowed for the design-hub support to be better utilised, as they were able to design building from user insights: “It meant that the designers who design, design based on direct input 40 KII Respondent 11. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 12 from girls from the research. So… it was a much more honest HCD experience because people were designing with insights on users.”41 It also mean that designers were able to work with more developed prototypes; as fuseproject reported, “we tried prototyping-in-context with designers first time around, but quickly realised, these businesses, some of them don’t have products so there’s not much use sending an industrial designer to field for a few days.”42 Two bootcamps provided important and increased touch points between SPRING and businesses; however, it has implications for who could participate, and for SPRING resources While the shift to two bootcamps had important positive repercussions for the prototype development process, there were also implications of this change. The bootcamp surveys revealed that feedback from Cohort 2 businesses overall on the time pressures of two bootcamps was mixed; while some businesses appreciated the intensity of bootcamps, others recognised the trade-off with balancing the on-going demands of their businesses. The addition of a second bootcamp helped place a greater priority on the HCD process and prototype development, it served to alienate larger corporate businesses from the process. Ironically, given that they are more likely to be able to absorb the cost of human resources, larger, corporate businesses struggled with the amount of time needed for attendance at the bootcamp: If we’re talking about the entrepreneur whose job is only to set up his business and definitely can spend two weeks/three weeks at Nepal on a regular basis. If you talk about corporations, then it has to be structured differently. I think three weeks or two weeks spending time cut off from office can get you in trouble. [Maybe] there should be a mixture of video conference, maybe, using technology more and a mixture of being present there. … But I think, for corporates, it has to be wired differently.43 This comment reflects other statements made by IP respondents about the shift of the cohort structure towards a much more time and resource intensive process. This view was echoed by one IP respondent who did not see the value add of HCD research in context: “the prototyping research, the thing that happens between the bootcamps…it just feels like a really heavy, heavy, heavy programme and, therefore, expensive and I think it needs a complete rethink in terms of the value-for-money question.” 44 Another IP respondent, who while recognising its functionality, commented on its expense and the resulting trade-offs: “[one] of the challenges is that [SPRING] is a lean programme…a lot of money goes to HCD.” 45 While these comments are not necessarily reflections on whether the shift to two bootcamps improves the effectiveness of the delivery of HCD, but that these changes did not automatically translate into greater business impact. This will continue to be explored in the ongoing programme evaluation. I think that was done in a very heavy, expensive way that had limited, limited impact. … I think it needs a complete rethink in terms of the value-for-money question: what actually has impact? How best to work with entrepreneurs in developing countries?46 Another important change earlier in the programme speaks to a similar type of programme change. This is the shift from an incremental change in the number of businesses per cohort, to a more standard number (between 18-20) and then the addition of a fourth cohort. IP respondents reflected that this change was driven by the capacity of fuseproject to deliver bootcamps. A respondent reflected, [W]hat do we know about what the resources we need to run this programme, based on the experience of the first cohort? In the original design of it, it was 18 in the first cohort and then 36 in the second and then 58 in the third. So it expanded which, obviously, doesn’t work ... that was obvious to all of us from going to the first bootcamp that fuse[project] and their curriculum for bootcamp, you probably couldn’t do that with more than 20 people at one time because you need one-to-one attention.47 41 KII Respondent 9. 42 KII Respondent 4. 43 KII Respondent 22. 44 KII Respondent 2. 45 KII Respondent 5. 46 KII Respondent 2. 47 KII Respondent 14. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 13 The rollout of Research-in-Context was also designed to augment the effectiveness of the HCD process for businesses In the context of Cohort 2, research with girls was part of the process of girl landscaping research, conducted ahead of the start of the cohort to help inform its design. During the Research-in-Context phase between bootcamps, businesses conducted HCD research more broadly, which did not exclusively (but in almost all circumstances) involve research with adolescent girls. Both of these phases were designed to ensure that the perspective and insight of girls would be embedded not only throughout the programme, but also in the prototypes designed by businesses. Ahead of the implementation of Cohort 1, extensive girl landscaping research was conducted in East Africa by the SPRING Technical Director to understand the needs of girls potentially targeted by SPRING businesses. A similar exercise was undertaken ahead of Cohort 2 in order to inform SPRING’s understanding of South Asia. This “situational analysis of girls” 48 provided useful insights, but its relevance to businesses in the form of its presentation was problematic. “I think that kind of information is incredibly useful for the rest of the ecosystem, the donors, academics… for the businesses, it wasn’t right.” 49 Another IP respondent reported, “it was interesting, but not as relevant as it needed to be.”50 Research is also undertaken by businesses, as part of the HCD process, in order to better understand the users, or potential users, of their prototypes. In Cohort 1, this research process was only provided to five businesses51 and focused on allowing entrepreneurs to gain insight into the day-to-day lives of girls, and for girls to take part in the prototype design process. It also allowed entrepreneurs to gain practical skills in implementing HCD research methodologies. The selection for research involved an assessment of the business’ overall viability and potential to reach scale, as well as the impact that research would have on the businesses’ design. This research process was reformulated for Cohort 2, firstly to be bookended by the addition of a second bootcamp, and secondly the research opportunity was extended to all 18 businesses. This decision was made to reflect the needs of the HCD process: In order to understand girls… how can we say that we’re using …HCD but only some businesses get to go out and do it? … If it’s at the centre, the core of what we do is understanding girls and designing for girls and their needs, then, surely, we’ve got to do that. We can’t just say we do it and do it with five as some sort of little experiment. It’s got to be core to our approach.52 In Cohort 2, the research shifted from exclusively “girl research” in Cohort 1 to be more specifically known as “HCD research” throughout Cohort 2.53 This meant that for HCD research, research would be conducted more broadly with the relevant purchasers and end users, who may not be exclusively girls.54 However, IP respondents made it clear that despite the reference to girl research, “it was very divided, the girl landscaping research and the HCD research was not connected”55 As one IP respondent clarified, [T]his is strategic design research, basically, that is necessary for the research and testing phase. The girl insights …for Cohort 2, it’s broader, it’s a much broader array…There’s not much overlap but [girl landscaping] feeds the businesses knowledge and understanding of the girl in their country context, but then [HCD research] provide[s] the actual in-depth research with their girls. So it’s different.56 This is particularly relevant to the target participants for both sets of research. Whereas girl landscaping explored the direct perspectives and challenges faced by girls, HCD research is sometimes instead conducted “… with the people who are actually going to be purchasing and using a product that may benefit girls. But girls might not be a direct consumer or even a direct user of the product, but hopefully, most of the businesses, it’s getting…[to] the girls.57 48 KII Respondent 8. 49 KII Respondent 10. 50 KII Respondent 8. 51 The five businesses were: Haute Baso, Jibu, Africaqua, Ensibuuko, and KadAfrica. 52 KII Respondent 8. 53 See Footnote *, above. The research phase was still referred to as “girl research” by some businesses, but more accurately speaking specifically refers to the HCD research conducted in the Research-in-Context phase. 54 An example of this circumstance might be when the end user of a product is a girl, but the relevant purchaser is a parent. 55 KII Respondent 1. 56 KII Respondent 4. 57 KII Respondent 8. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 14 The success of Research-in-Context was put at risk due to poor coordination of research Based on the experience of Cohort 1, SPRING learned two important applicable lessons for Cohort 2, namely that a research partner would be needed to ensure research logistics (including quality control, girl safety assurance, and a final debrief) were consistent across countries, and that research logistics required greater resources to ensure that research was conducted in the relevant geographical context. These lessons were particularly pertinent with the decision to roll out HCD research amongst all 18 businesses, in three new SPRING countries. To address these learnings, SPRING adopted an outsourcing model for the research and recruited a research partner, TNS58, to coordinate and implement the research with SPRING businesses and specially recruited SPRING research moderators. SPRING businesses worked with fuseproject to design research in Bootcamp 1. The research partner was then responsible for recruiting participants for the research using the parameters set by SPRING businesses (such as age, location, economic bracket). The research itself (primarily using HCD methodology, involving participator methods in groups) was conducted by moderators recruited by SPRING and trained by fuseproject, who then provided research reports back to the businesses. Cohort 2 businesses across all three countries provided a lot of feedback on the results of HCD research, in both KIIs and in bootcamp surveys. Businesses provided candid feedback on surveys, such as, “research recruitment with TNS and having researched with the wrong target market” and, “The business prototype research did not go right.”59 Generally, the research partner was described non-responsive and lacking professionalism. Businesses were able to expand on what went wrong: I was not happy with the research part… our [moderator] really understood and worked really well with me and put in a lot of effort….but the [research partner] that was arranging all of this, I was extremely unhappy with them. I think their coordination was awkward, they wasted a lot of time, they could have definitely given us access to more girls, research could have been a lot more extensive and a lot more productive.60 Another business, from a different country, reported: “I would say the struggle was with the prototype research… [it] became very cumbersome and multiple parties are involved, and there’s a lack of coordination among the parties.”61 Some businesses expressed disappointment with the quality of the submitted research, revealing that girls from the incorrect market segment had been interviewed. Others were satisfied with the outcomes of the research, but concerned about the research process. “The research execution was a disaster…They were not … prepared.”62 Some companies identified problems with this research process early on and ran parallel HCD research at their own cost. Additionally, interviews with moderators revealed concerns they had for their own safety at times, as well as for the safety of their girl respondents. Business also expressed concerns about the safety of the girls, with girls reporting that they did not know who was “collecting them for the girl research”, making them feel unsafe. “Girl research was very helpful [but] the moderator was confused, we were confused, the [research partner] was confused, and… they hired a Nepali company and that company was not that good, so they could not get the right candidates for the research”. 63 This issue was flagged in only one instance and not representative of the remainder of the HCD research experiences; however, it flags that girl safety can be an issue resulting from the poor coordination of research, even in the presence of strong girl safety protocols and well-trained moderators. Issues with Research-in-Context stemmed from both the research partner procurement process, and the lack of alignment on the purpose of research across the SPRING consortium Several explanations were identified to account for the issues that emerged during HCD research, related to confusion that emerged over the coordination of the research process, and the poor performance of the research partner in crucial issue such as participant recruitment. First, as SPRING adapted from Cohort 1 to Cohort 2, they had very little time to implement the changes to the plan. One IP respondent described this process: “SPRING is a very fast-moving programme, so I think it needs to have that recognised, but plans coming into place, it was difficult 58 Also interchangeably referred to by SPRING businesses and respondents as ‘Kantar’ – the company TNS was rebranded as TNS Kantar in September 2016. 59 Coffey Cohort 2 Bootcamp 2 Survey. 60 KII Respondent 28. 61 KII Respondent 61. 62 KII Respondent 30. 63 KII Respondent 66. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 15 to get aligned…between the different players as to what research would really entail and what it really required.” 64 Generally, the process involved in recruiting an external firm to oversee the research coordination was difficult and slow, resulting in the late recruitment of the research partner: So I think that the problem with [TNS] was … I think that they were poorly recruited, they were brought in at the last moment, and, therefore, there was too much pressure to have them understand the research programme that we had designed and not enough time. So they were procured in the wrong way.65 The late procurement of the research partner meant that there was less opportunity to ensure that TNS understood the task set out. Poor coordination within the SPRING team, partially as a result of changing personnel, and within the TNS team, especially with regard to logistics, exacerbated the problem. I think they were briefed in the wrong way, and I also think the problem is that research…human-centred research is different from research …by the time [TNS] figured out that this was a much more unique and bespoke process, they had probably under-budgeted it and underprepared their management for it.66 This issue was further compounded by the lack of strategic oversight across the SPRING consortium, as reflected in this statement from an IP respondent: “No one was aligned. Visions were stuck in heads, constraints and budgets were stuck in other heads, and things weren’t shared transparently, or in a [sic] ordered fashion. So we were constantly scrambling.”67 Both businesses and SPRING acknowledged that Research-in-Context was a difficult learning process, but ultimately, research – when it did take place – contributed valuable insights for businesses. “[T]he result was a strange mixed bag whereby…on the one hand… the research was much more successful than I anticipated, some of the insights we got was [sic] great given that we were using unexperienced moderators and given that….On the one hand… We get over the line, we get the…most of the information that we need and we get to our target, but the process is always so exceptionally painful.68 “It was very messy. In the end, I think valuable. It generated value for many of the businesses. Not all, certainly, but, going through the process, I think provided value to them all anyway.”69 Problems with delays in receiving the research findings meant there was less time for follow up work prior to Bootcamp 2. Some businesses mentioned challenges concerning costs and funding, including the costs of human resources, and transportation. Changes to investment readiness support was adapted for Cohort 2 to effectively meet the needs of relevant businesses, and early signs suggest this has enabled businesses to increase their investment readiness In contrast to the changes made to the roll out of HCD research across SPRING businesses, SPRING learned from Cohort 1 that more catered and individualised support was needed in order to increase “investment readiness” across SPRING businesses. Furthermore, investment readiness support has largely been implemented through SPRING’s Investment Director, limiting the scalability of the process to all cohort businesses. Lastly, the SPRING logframe indicator’s requirement to improve businesses’ investment readiness also recognises that not all businesses are yet in a position in which they seek investment, or require investment. Therefore, the process for the investment readiness programme for Cohort 2 was simultaneously designed to be more structured, but also personalised for businesses. This includes access to the Duke University’s CASE Business School course on capital raising, as well as expert advice and coaching from the Investment Director, the creation of a new role in South Asia for a Regional Investment Support Manager, and further additional support (provided by in-kind, low, or pro-bono support) in financial modelling, and the design of investment prospectuses and pitch decks. Rather than provide support through a series of investment readiness modules (as in Cohort 1 bootcamp), investment readiness support was provided through a series of 1-on-1 sessions in Bootcamp 2, and through 64 KII Respondent 4. 65 KII Respondent 9. 66 KII Respondent 9. 67 KII Respondent 4. 68 KII Respondent 9. 69 KII Respondent 4. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 16 individualised support after Bootcamp. Support was structured to help businesses improve their investment readiness according to 10 key areas: Box 1: Cohort 2 key areas of investment readiness support 1) The quality and completeness of the entrepreneur’s / team’s experience, in terms of being able to run the business and the investment / capital-raising process 2) The entrepreneur’s / team’s understanding of requirements for the capital-raising process (including time and resource commitment) 3) The level of clarity around valuation logic (if appropriate, or comparatively, if available) 4) Whether the company structure is investable or if it needs restructuring 5) Clear definitions of the share ownership structure and capital-raising structure 6) The existence of a board and governance structure, or clarity of requirement 7) The completeness and quality of financials and other investment diligence documents. 8) Whether the entrepreneur / team is pitch ready, including pitch deck and executive summary 9) Clear investor shortlist, with potential investor types and rationale 10) Contact with potential investors In Cohort 2, investment readiness support was limited to nine businesses who were deemed by SPRING to have the potential to raise capital. With the introduction of Bootcamp 2, investment readiness support was largely shifted towards Bootcamp 2 and after to allow the first phases of the accelerator programme to focus on prototype development. This meant that businesses had a condensed period in which they were able to improve their investment readiness and apply learnings to pursue leads and attract investment. Amongst the nine businesses who received investment readiness support, seven were actively being supported to attract investment, while as of September 2017, two had received investment.70 However, the effectiveness of investment readiness will require more than nine months to be realised SPRING’s ToC continues to hold the assumption that businesses will improve their investment readiness within the 9-month accelerator programme. SPRING sees an increase in investment readiness as a result of the whole package of SPRING, and not just investment readiness input, yet, a continuing weakness in SPRING’s ToC lies in making explicit the pathways and causal chains involved in a businesses’ ‘investment journey’, and conversely, the pathway required to connect “investor warm up” to the “crowding in of new market actors and investors”. While in Cohort 1 it was found that investment readiness was provided either too early, or to businesses who were too early stage to be investor-ready, adaptations to the investment readiness programme for Cohort 2 were designed to address these. One of the noted limitations of the investment readiness angle has been the ability of the programme to address not simply investment readiness, but investor readiness. This is reflected in the implicit assumption in the SPRING ToC that for SPRING businesses to translate investment readiness into investment, there must be a supply of investors who themselves are ready to invest in the SPRING businesses in the nine SPRING countries. In the case of Cohort 2, the space for foreign investment in Nepal is significantly limited due to the regulatory hurdles in introducing foreign capital into the country. As one IP respondent noted, “All the money that’s gone into entrepreneur cultivation and entrepreneur support has been great but not if it’s not matched but also bolstering the investor side.”71 A further reflection on the limitations of investment readiness concerned the lack of availability of intermediary funds or engaging local investors, both of which may be well placed to provide smaller amounts of investment to businesses. Such funds would help businesses as …they go through their fundraising process where they’re kind of in a valley of death. They’re prototyping new ideas, they don’t have enough traction and they don’t have enough data to show that it’s working yet, they’re running out of cash or they’re going to run out of cash if they grow in the way we want them to grow, 70 As reported in SPRING’s 2017 Annual Report. The two that have received investment are Drinkwell and R&D. As a figure of comparison, at the end of Cohort 1, 3 businesses had been successful in attracting investment. Subsequently, 6 Cohort 1 businesses have attracted investment since the close of Cohort 1 (including the original 3 businesses attracting further investment). 71 KII Respondent 15. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 17 but the investors are saying come back when you’re further along and, so, many of them are in danger of just running out of cash while they’re in the process of working with us.72 The role and responsibilities of country managers have evolved and become more defined in Cohort 2 The role and structure of country managers has evolved since Cohort 1, where country managers were linked with local incubators to provide a coordination and mentorship role. Beyond this, the role of the country manager was not necessarily defined. Country managers were considered to play a key role in providing or facilitating in-country technical support, as both the regular liaison between SPRING businesses and the global/London team throughout the duration of the accelerator programme, but also in providing important context and local knowledge back to the rest of the SPRING team and experts. This was envisioned as a light touch position, given the resources dedicated to this role for Cohort 1. The approach and role of country managers has evolved in Cohort 2 and become much more defined. As with Cohort 1, country managers in South Asia were responsible early in the process in outreach and developing a pipeline of Cohort 2 applicants. In this sense, the role of country managers as the key source of important contextualisation for the rest of the SPRING team in the selection process, as well as in the early diagnostic sessions at Bootcamp 1, has been re-affirmed. The clearer sequencing of the HCD process (diagnostics in Bootcamp 1, research and testing in between bootcamps, and refining and iterating in Bootcamp 2), alongside the investment readiness phase post-Bootcamp 2, has helped clarify what it is country managers need to support businesses to do, at each stage of the accelerator. IP respondents defined the role of the country manager in Cohort 2 as being “completely hands on, real guiders and advisors to the businesses every step of the way… and also… a critical friend to check things and to give them the tough love when they need it. Particularly around budget building and workplan building” 73and “the single point of contact, the main relationship builder.”74 Another IP respondent described the importance of the role of country manager as with “having a strong person on the ground who is in constant contact, who is a valuable resource for the businesses.”75 Changes to the accelerator programme mean there is now a greater reliance on country managers to provide support outside of bootcamps, for a business to succeed As one IP respondent said, “[T]he stronger the relationship between the country manager and the business, the more likely they… really benefit from the full package of support that SPRING can offer.”76 The changes made to the accelerator programme have placed a reliance on businesses to achieve certain amounts of progress at key stages of the cohort timeline – notably in research and testing in order to refine prototypes at Bootcamp 2, and then to launch prototypes after Bootcamp 2 and to focus on investment readiness. In this sense, the technical assistance provided by SPRING does not simply require the businesses to optionally absorb the trainings, but to actively participate and graduate through the phased sequence of events. For businesses to do so, there is a greater reliance on the role of the country manager to provide regular support to businesses outside of the two bootcamps, as well as to document and evidence this progress as part of the SPRING monitoring system. “[T]he challenge with bootcamps is it happens at a certain time. So for us, it happened in September, and a challenge will come up for a business in December or January… and the bootcamp is irrelevant for that. You have to have someone on the ground.”77 Country managers are also responsible for identifying and driving the use of additional local support, particularly in mentoring (see Section 2.3.3 below). Although this has primarily concerned supporting businesses (or finding support for businesses) in other areas of business planning and functions (areas outside of the prototype development). However, IP reflected that one of the biggest limitations to the role has been the under-resourcing of the position. This includes the need for country managers to work as part of teams, particularly in countries that have more than 5 businesses. As one member of the SPRING London team reflected, “Our team have really agreed that the country manager role is really important and… we can’t scrimp on it, we can’t cut out any more than how lean it [is] 72 KII Respondent 15. 73 KII Respondent 1. 74 KII Respondent 7. 75 KII Respondent 11. 76 KII Respondent 8. 77 KII Respondent 2. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 18 already… having a good team would be ideal.”78 Another IP respondent reflected, “I would invest much more… resource on the in-country staff to do on the on-the-ground work, the follow up, the on-to-one, the pushing, the challenging, keeping them on track.” 79 2.3.2 How effective was SPRING financial assistance to help businesses develop prototypes to meet girls' needs? Learning from Cohort 1, SPRING moved towards a differentiated funding model, selectively providing financial assistance to SPRING businesses As with the provision of technical assistance, SPRING implemented a number of changes to the ways in which it provided financial assistance to Cohort 2 businesses. In Cohort 1, SPRING found that the provision of uniform amounts, and for unrestricted purposes, did not necessarily facilitate businesses’ in developing and launching their prototypes. As discussed before, this issue was compounded by the fact that many Cohort 1 businesses were in early stages of their development, and therefore dedicated the funding they received towards setting up financial and administrated structures, and not necessarily on their prototype. For Cohort 2, SPRING trialled a new model for providing grant funding, renamed the Prototype Development Fund (PDF). The PDF is a differentiated funding model, meaning that funding would be allocated through a revised needs-based application and budget, and targeted specifically towards prototype piloting and launching. Therefore, businesses would not be eligible for funding if was deemed that they would not launch a viable prototype. Following Bootcamp 2, businesses were asked to submit revised prototype plans and budgets, which were assessed by a committee with the SPRING team. In Cohort 2, 14 businesses successfully applied for the PDF grant, receiving on average £38,925, and whereby 5 of the 14 businesses received the full amount of the budget requested.80 Funds were then disbursed between March – May (months 6 to 9 of the accelerator programme). Additionally, during the cohort it was decided that businesses would also benefit from financial support to help facilitate and enable more research during the Research-in-Context phase. Ultimately, 10 of the 18 Cohort 2 businesses received research support funding, ranging from £2,433 to £11,20781 although three of these businesses were unable to utilise these funds as they did not have users with which to test their prototype. This is part of a process, in which SPRING is trying to towards a technical assistance programme where businesses seek grant funding as part of their SPRING journey. As one IP respondent reflected, I think that businesses are much better able to use it for their needs rather than just being given £50,000 and sink or swim. I think that the focus being taken away from the grants and being much more on the offering of SPRING, I personally feel that it was an excellent way of changing the strategy, really, because I think before it had been seen only as a grant programme with a bit of TA. Now I think it’s much more TA with a bit of grants and I think that that is really useful for businesses.82 These changes have helped SPRING focus on generating benefits from technical assistance; however, the messaging to businesses on this shift was often unclear Throughout the process of evolving the financial assistance component of the programme, SPRING struggled to ensure that the messaging of this component remained clear to businesses, in spite of ongoing communication from the programme to the businesses. Businesses stated that at the application stage and up until Bootcamp 1, they were not clear on how the financial assistance component would function, as demonstrated in the two quotes from Cohort 2 businesses below: When we applied, it just said you can get up to $80,00083 of grant funding, but it wasn’t set out what that would be for, that it would come in two tranches, or that, first, you’re only getting $10,000 or however much, and then you’re going to get the rest. That wasn’t clear, and I think it became more clear once we went to bootcamp and we heard about it.84 78 KII Respondent 1. 79 KII Respondent 2. 80 See SPRING 2017 Annual Report, Annex 8 Prototype Development Fund Alignment Deck. 81 See SPRING 2017 Annual Report, Annex 8 Prototype Development Fund Alignment Deck. 82 KII Respondent 20. 83 This figure is roughly interchangeable with the £50,000 amount of funding originally advertised as part of Cohort 1. This expectation does not necessarily reflect misinformation provided by the programme, but rather, expectations held by some of the businesses coming into SPRING. 84 KII Respondent 29. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 19 At Bootcamp 1… we found out that there’s actually money, there’s actually funding available for the prototype development. And, even then, we had no idea how to go about it and what it is that we have to do… what amount of money, and is everybody getting it or are we getting it? So there was a little bit of confusion around that.85 This was similarly reflected by IP respondents: Somehow, the message [the businesses] got while applying was they were hoping [for] $80,000 or more…. So some even just applied for money… You get the fund of course, but it is to develop your prototype, not to run your whole business.86 However, while some of this became clearer to businesses at Bootcamp 1, the introduction of funding available to businesses after Bootcamp 1, in the form of research support funding, added further confusion. The decision to allocate research funding to 10 of the 18 businesses appears to be an adaptive response to address emerging challenges faced by businesses ahead of the Research-in-Context phase, and was effective in filling research gaps. However, this was not communicated clearly to businesses. One IP respondent reflected: “For additional research, there was no process, and it was messy, and it was confusing for the businesses, it was confusing for the whole team.” 87 Ultimately, the problems with communicating these changes did not result in further issues. Businesses have noted that they would have liked clarity greater clarity, but were content with the process for PDF applications: I think… people want… a clear indication of how you’re going to get that money… Those things could be clearer, but I wouldn’t say that I had a problem with it all. 88 The way the budget was prepared jointly [by SPRING and the business] and the support amount identified, I think it’s very democratic.89 SPRING has also acknowledged the importance of clear messaging to manage the expectations of businesses for future cohorts.90 2.3.3 How effective were additional sources of support to help businesses develop prototypes to meet girls' needs? SPRING’s approach to additional support is now more focused and tailored; the extent of support provided for Cohort 2 remains relatively limited While the focus of SPRING’s offering to businesses as an accelerator derive from its core technical assistance support and financial support, SPRING also offers ‘additional’ support, which derives from in-kind, low, and pro￾bono support facilitated by SPRING, as well as mentoring support. While the original design of SPRING envisioned a comprehensive global and local mentorship scheme to be applicable and useful to all businesses, the experience of Cohort 1 demonstrated that a much more tailored plan, where additional support would be provided as needed, was a more efficient use of resources and effort. The provision of additional support is also more closely linked to SPRING’s partnership building efforts and targets towards logframe Output Indicator 4.2.91 Across Cohort 2, six businesses reported that SPRING had linked them with mentoring or additional support, in addition to the legal support provided by Hogan Lovells.92 In addition, Deutsche Bank provided financial modelling support to several businesses.93 The effectiveness of the additional support component of the programme depends both on the demand for the support, as well as the support supplied. Only one business reported that they had asked, or required, additional support that had not been satisfied; other businesses remained satisfied either with the support they received, or did not require further support. For those that did receive additional support, the response was positive, as SPRING was able to match needs with support that had both the right expertise as well as availability. 85 KII Respondent 27. 86 KII Respondent 3. 87 KII Respondent 1. 88 KII Respondent 29. 89 KII Respondent 61. 90 See SPRING 2017 Annual Report, pg. 40, as well as Annex 8: Prototype Development Fund Alignment Deck. 91 Cumulative amount of value-add (in-kind and cash) contributions raised by SPRING implementing partner. 92 The six businesses who reported support are: Ariia, Sehat Kahani, SmartPaani, Uptrade, Drinkwell, and iSocial. Hogan Lovells provided legal support 1-on-1 sessions at Bootcamp 2, which were available to all businesses. 93 To Drinkwell, KGG, and Sehat Kahani. See SPRING Annual Report 2017. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 20 One area that was not addressed was on-going girl expert support. As identified earlier in this report, a gap resulting from the change in the accelerator design was the lack of further girl-expert support beyond Bootcamp 1 and outside of M&E support. This was further identified as an area businesses identified that they would like to receive more support. Although efforts were not made to provide a balance of global and local support, this ultimately was not an issue for SPRING businesses Most of the additional support provided to Cohort 2 businesses was global (e.g. London-based) in nature; both donor and IP respondents reported scepticism of this approach, but also reported success thus far: I think SPRING should, in terms of mentoring, try to develop networks within the chose countries of mentors and businesses… SPRING would add more value if they connect the businesses more with local mentors and local experts.94 I was really sceptical about this in the beginning. And I thought it’s even more challenging if people are trying to do it between different countries. But, in some case, it has worked really well.95 In most cases, when businesses were asked about any additional locally-based support, they continued to report satisfaction with the role played by their country manager as mentor. 2.3.4 Which (combination of) accelerator inputs were most effective in improving the performance of businesses? Part of what makes SPRING unique, versus other accelerators, is the holistic package of support it provides to businesses.96 This includes the combination of technical assistance in both HCD and on adolescent girls, investment readiness training, in-country support, financial assistance, and local/global mentoring and additional support. As one country manager put it, [T]he other accelerators only cares about investing in a high volume of money… they don’t give the knowledge part, human-centred design, the whole approach or research part. So [the businesses] just feel that [SPRING] is a whole package… it’s a package of knowledge, it’s a package of mentoring, it’s a package of networking, they connect you to all these amazing people.”97 SPRING’s combined effect of components aim to achieve both an increased number of girls reached, as well as increase business performance in the form of sales. The Cohort 2 BPE Report provides a greater discussion of the extent to which those goals have been reached, this section provides a brief summary of the effectiveness of each SPRING input in contributing to, or combining, towards the performance of SPRING businesses. Cohort 2 differs from Cohort 1 in that while HCD support is now received by all businesses (particularly in the form of HCD research), key SPRING inputs such as grant funding (the PDF) and investment readiness support are provided on both the basis of needs-based (with tailored support) and as well, as well as likelihood of prototype launch. Additional support was only provided in instances where gaps remained in business needs, after the three other sets of inputs. Therefore, receiving technical assistance (in the form of using HCD for prototype development) became a prerequisite for receiving further inputs of PFD and investment readiness. Five businesses ultimately received all four elements of inputs: Drinkwell, Goats for Water (Uptrade), iSocial, Smart Paani, and Sehat Kahani, while another eight received the three main inputs (see Table 3 below). 94 KII Respondent 6. 95 KII Respondent 2. 96 See Coffey’s End of Cohort 1 Programme Performance Evaluation Report, Section 3.12 for further discussion of SPRING’s unique attributes as an accelerator. 97 KII Respondent 3. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 21 Table 3: Summary of inputs received by SPRING businesses, and outputs and outcomes achieved Business Name AA DL DW EP EN GW IS JB KG NS PT RD RL SH SP SN SW SK Inputs received Received HCD support ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Received PDF  ✓ ✓   ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓  ✓ ✓ Received Investment readiness  ✓ ✓   ✓ ✓    ✓ ✓  ✓ ✓   ✓ Additional support (e.g. mentoring) ✓  ✓   ✓ ✓        ✓   ✓ Outputs and outcomes achieved Applied HCD  ✓ ✓   ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓  ✓ ✓ Launched prototype  ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓  ✓ Increase girls reached ▲ ▲ ▲ ▲ ▲ ≡ ▲ ▲ ▲ ▲ ▲ ≡ Increase sales ≡ ▲ ▲ ▲ ▲ ≡ ▲ ▲ ▲ ▲ ▲ ▲ Attract external investment ✓       ✓     Key Not applicable ✓ The business received this input / achieved this output or outcome  The business did not receive this input / did not achieve this output or outcome ▲ The business reported an increase ▼ The business reported a decrease ≡ No data available, or nothing to report Note: The table shows the results for within the 9 month period of Cohort 2. AA: Ariia; DL: Dot and Line; DW: Drinkwell; EP: Easy Paisa; EN: Engro Foods; GW: Goats for Water; IS: iSocial; JB: Jeevan Bikas Saamaj (Nawa Bihani); KG: Kalpavriksha Greater Goods; NS: NexSource (U-Sehat); PT: Paritran (Fightback); RD: R&D; RL: Rooster Logic, SH: Shreenegar; SP: Smart Paani; SN: Sunaulo Parivar Nepal (Marie Stopes International); SW :Switch (Zoya); SK: Sehat Kehani. Source: As reported in the SPRING Annual Report 2017. Across the different inputs received, there was no particular bearing on the outputs and outcomes achieved by the business, other than (by definition) needing to have launched a prototype. This demonstrates that SPRING has moved beyond providing a ‘basket’ of inputs to businesses, to instead providing a tailored set of support and technical assistance, designed and delivered in a complementary way. A further, unintended product of SPRING inputs has been the formation of peer networks Although not necessarily a planned outcome, both IP respondents and SPRING entrepreneurs reported benefits of being part of a cohort of businesses. These included potential long-term businesses collaborations, sharing contacts and mutual learning from each other’s prototypes and experiences. One business mentioned that they had hired a member of staff based upon a referral from another SPRING business.98 Entrepreneurs reported that during bootcamps, SPRING encouraged peer networks, reporting: “It has benefited us a lot to know other 98 Coffey Cohort 2 Bootcamp 2 Survey. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 22 businesses’ prototypes. Yes, SPRING has facilitated this interaction, this is a way of networking / partnering with each other” and “SPRING has always encouraged us to partner with other businesses at Bootcamp”.99 I think, actually, the cohort cohesion and the partnerships that have come up have gone very, very well. There’s been a very lively post-Bootcamp 2 continuing engagement amongst the companies that bodes well. I think there’s now country networks, certainly in Nepal and Pakistan, that are going to be very beneficial in a way that Cohort 1 never really achieved.100 Both donors and IP respondents reported that they believed that the creation of in-country networks would form an important legacy for SPRING: I think the most powerful thing we’re able to build with this programme are tightly linked networks in-country that can actually challenge each other and support each other as they grow and serve as a model for other businesses in-country. These are fairly small communities… Cohort 2, we had people saying things like I never knew there were entrepreneurs like this in Pakistan. Cohort 2 and 3, we had partnerships springing up where people are not trying to solve everything on their own but actually trading ideas, assets, networks, connections, investors. That, to my mind, will be the most valuable legacy of SPRING and it could be quite powerful but it’s not going to happen overnight.101 I think SPRING should, in terms of mentoring, try to develop networks within the host countries of mentors and businesses, even…this community could be very small, because you are talking about five or six businesses in a country, and a few mentors but, even if it’s a very small group, I think, in terms of sustainability, SPRING would add more value if they connect the businesses more with local mentors and local experts.102 2.4 How effective was HCD at helping businesses refine their SPRING product, service, or business model? This section further explores the OECD-DAC criteria of ‘effectiveness’, with a particular focus on the role of HCD. While this section examines the effectiveness of SPRING’s delivery of HCD to help businesses refine their prototypes, Cohort 2 BPE Report - Section 2.2 more closely examines the businesses’ uptake of HCD and how effectively HCD has been used to both better understand girls’ needs and improve their reach of adolescent girls. HCD is key to SPRING as both an output of technical assistance, and an approach to programme design and implementation. Its focus on empathy and user consideration places users at the centre of the design process to create products, services, or business models that suit users’ needs. SPRING utilises HCD as a method to guide businesses through identifying adolescent girls as beneficiaries of their businesses, and to assist them in understanding this group’s real-life behaviours, attitudes, and needs. SPRING entrepreneurs are introduced to HCD thinking and principles at Bootcamp 1, through workshops and modules that focus on girls, end users, and stakeholders to identifying their behaviours, attitudes, and needs. These learnings are then translated into the Research-in-Context phase between the bootcamps. In this phase, businesses work with local moderators to apply their HCD thinking by designing research that tests hypothesis about girls and/or end users during Bootcamp 1. Insights from this research are then examined during Bootcamp 2, to help inform both the refinement of prototypes as well as further design and additional support. SPRING has made several changes to the accelerator programme to improve the effectiveness of HCD SPRING has undergone a number of adaptations since Cohort 1 that aim to help the programme better deliver technical assistance to help SPRING businesses use the HCD process. These were discussed in Section 2.3.1, and can be summarised as: • The selection of more mature businesses, to ensure that businesses have end users with whom they can test their prototype; • The roll out of HCD/girl research across all 18 businesses, to ensure that all businesses have the opportunity to collect user insights; 99 Coffey Cohort 2 Bootcamp 2 Survey. 100 KII Respondent 7. 101 KII Respondent 7. 102 KII Respondent 6. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 23 • The addition of a second bootcamp, in order to allow businesses to take advantage of the Research-in￾Context phase, and return to reflect on user insights and to work with fuseproject to further refine and iterate their prototypes; and • Although not directly related to the HCD process, the shift in the sequencing of investor readiness support to Bootcamp 2 and beyond allowed businesses to first focus on the design and iteration of their prototype, and then receive support (both technical and financial) to facilitate its launch or scaling. Cohort 2 businesses who had no prior experience of HCD became enthusiastic practitioners Prior to the SPRING programme, most (15) businesses had little to no understanding of HCD. This included three entrepreneurs who had never heard of the HCD process, four entrepreneurs who were aware of HCD but did not understand what it is, and eight entrepreneurs who said they had limited knowledge of HCD. Only three entrepreneurs said they had a fairly or very good understanding of the approach.103 80% of businesses replied that since the start of their SPRING cohort, they have used HCD principles to refine their girl-focused product, service, or business model (See Figure 1 above). 104 Of those businesses, most businesses used HCD principles to design their prototype, conduct girl research, and map their end user journeys. As one entrepreneur explained, I would say I started focusing more on girls. One big change, that I’d never done before. And I got to understand the lower-income people’s mind set. What do they prefer?… So I knew that high-income group will always go for quality but I always thought that middle-income group will not give emphasis on the quality but, after doing a survey … I realise that they want quality as well.105 Overall, the HCD focus of SPRING has been well received by the entrepreneurs. All but one of the entrepreneurs see HCD as a useful approach to improving their business model, product or service, with 13 of the 17 entrepreneurs describing the approach as ‘very useful’ and three as ‘fairly useful’ (See Figure 2). 103 Coffey Bootcamp 1 Survey. Total respondents = 18. 104 Coffey Cohort 2 End of Cohort Survey. Total respondents = 16. 105 KII Respondent 25. 0 2 4 6 8 10 12 14 Mapped out the end user journey Defined a design challenge Designed a prototype Tested a prototype Synthesised findings and learning from the research between Bootcamps 1 and 2 Captured learning from the prototype development period after Bootcamp 2 Conducted research with adolescent girls Conducted research with other stakeholders Figure 1: Uses of HCD principles to refine businesses' girl-focused product, service, or business model COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 24 All entrepreneurs were asked to consider which aspects of HCD they found most and least useful. Reflecting the overall positive response to the HCD approach, respondents focused on the most useful aspects, especially understanding the needs of the users and seeing things from their perspectives. Respondents also focused on the usefulness of the approach in service design and business processes. Businesses’ use HCD beyond bootcamp – and not just to make profits Beyond the SPRING prototyping process, entrepreneurs were generally positive regarding HCD explaining how the approach and the related support has helped them to think differently to understand the needs of the end user, to generate ideas, and solve design challenges. Exposure to a different business tool, using an interactive methodology assisted the businesses in absorbing the methodology and applying it in contexts outside of SPRING. Entrepreneurs provided comments on our bootcamp surveys such as, “[w]e had never experienced HCD before this bootcamp. After knowing it, it has helped us to solve our complex design challenges in the entire business” and “Loved the HCD approach, it was explained well enough to me to influence my thinking completely.”106 Further comments include: [Our] business is… developed through HCD. Assessing the needs and then tailoring the solution, looking at the need of our customers which are very different from being able to see things from their perspective. So, in terms of how [HCD has] been helpful, it formalised the process which we do, I think, instinctually. So it formalised the process of how to evaluate things from the HCD perspective.107 So, for us, just getting feedback out of them and feedback that is unbiased and…especially product feedback. So that was very important. And I don’t think the results would have been that great or that comprehensive if we hadn’t applied the HCD model.108 So all the design challenge, and understanding the user journey, understanding their perception, we had never gone in that depth. The human-centric design really helped to understand the likes and dislikes of our users… the method that I learned from SPRING was great. So I will be definitely using it in my other product as well… So I will use it in my business… There are a hundred shopkeepers who needs [sic] to understand girls’ eating habits. So I will train them to understand girls’ human-centric nature and design...109 More mature businesses struggled to use HCD to develop prototypes While the general overall sentiment towards HCD amongst entrepreneurs was positive, some entrepreneurs did note the limitations they faced in using HCD. HCD was recognised as being time and cost intensive, and only able to focus on small groups of users. On this, entrepreneurs reported, 106 Coffey, Cohort 2 Bootcamp 1 Survey (emphasis added). 107 KII Respondent 65. 108 KII Respondent 27. 109 KII Respondent 25. It's a very useful approach to improving my business model, product, or service. 76% It’s a fairly useful approach to improving my business model, product, or service 18% I can see how it is useful for some businesses but it is not useful for my business 0% I do not think it is a useful approach to developing business models, products, or services 6% Figure 2: Usefulness of HCD to improving business model, product, or service COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 25 It was time-consuming. I would not say it’s easy research, but it was a very helpful research which gives us the in-depth insights of the girls’ interest.110 I think it’s very time consuming and it’s also costly … It’s definitely an investment.111 One entrepreneur reported, “it’s extremely valuable to listen to your customers but, at some point, you need to stop this thing and you need to start implementing that in your business. So, for me in particular, finding that cut-off line or that sweet spot, that was a bit of a challenge initially.”112 As indicated in Section 2.3.1, larger corporate businesses struggled with the multiple bootcamp format of the programme, designed to facilitate the HCD process. It was not that “these businesses couldn’t engage with the bootcamp… They were very engaged, the participants that came,” but rather, “[a]t that level, the participants come across as perhaps a little less like entrepreneurs, a bit more like leaders of large organisations… They’re managers, I suppose.”113 2.5 To what extent has SPRING adapted and with what effect? This final section exploring SPRING’s ‘effectiveness’ reflects upon SPRING’s ability to adapt and learn. SPRING has continued to demonstrate that it is an adaptive and iterative programme, through a great number of changes made to improve the effectiveness of its components for Cohort 2, detailed throughout Sections 2.1-2.4. Building on the mechanisms that the programme has put in place for continuous formal and informal learning,114 the SPRING consortium has adapted the programme based on learnings from Cohort 1 ahead of the implementation of Cohort 2, and during the process implementing Cohort 2 as well. One purpose of including HCD as a learning process within SPRING was to identify its applicability as an approach within a developmental context. SPRING as a programme has continuously utilised the HCD process as an iterative, internal process for programme adaptation. As a learning programme the IP has, after events; paused, reflected, pivoted or innovated and then re-implemented. In particular, the HCD method was used to facilitate adaptation between Cohort 1 and Cohort 2. One IP respondent discussed this, in the context of coming up with the solution of two bootcamps: [W]e each had a board and we… did the user journey for the participants. And we had post-its and we each came up with ‘what’s a potential method for what we could do, what would the challenges be? How do we change that?’ We had: ‘you could make two boot camps... you could have a bootcamp and then go visit each company.’ We created these potential solutions and this was the one that we arrived on.115 Table 4 summarises of some of the key adaptations made in SPRING, while Annex 4 contains a more detailed description of the key changes between Cohort 1 and Cohort 2. 110 KII Respondent 25. 111 KII Respondent 27. 112 KII Respondent 28. 113 KII Respondent 2. 114 See Coffey’s End of Cohort 1 Programme Performance Evaluation Report, Section 3.15. 115 KII Respondent 11. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 26 Table 4: Summary of key adaptations made in SPRING Learnings from Cohort 1 – Cohort 2 Learnings going forward Selection process Focus selection more on developing a pipeline of businesses; selection process to be streamlined into individual interviews and committee decisions, instead of in-person pitch fuseproject and SPRING M&E to integrate on participant landscaping, so that at the start of the cohort, SPRING has a better understanding of the business (e.g. draft ToC already prepared) Selection criteria Pick a mix of businesses and experiment with more mature businesses, as start-up businesses struggle to launch prototypes. Move away from selecting businesses proposing a prototype with girls in the value chain Still getting ‘goldilocks’ of business size right – not too big and not too small; still picking a mix, but need to figure out what allows a mature business to pivot Bootcamps Necessity for a second bootcamp as an opportunity to allow for iteration based on HCD research insights Tweak bootcamp curricula: Sequencing of investment readiness support – not to leave diagnostics until Bootcamp 2, where there is very little time to act on what is needed, after Bootcamp 2; greater forward-thinking on scaling prototype to be built into prototype design and refinement at bootcamp; ensure that ‘girls’ stay at the forefront of processes after Bootcamp 1 HCD research Need coordination and logistics of research; guaranteeing consistency Need to have greater clarity on goals and purpose of HCD research, with research partner and moderators; need to have alignment across SPRING team on goals of research. Grant funding Unrestricted grant funding encourages grant￾seeking businesses and does not produce sustainable prototypes; businesses to apply for funding needed and to demonstrate case for commercial viability of prototype Need to ensure the purpose and process of PDF funds is communicated to businesses from the start. Investment readiness Need for tailored investment readiness; but also consider the prerequisites for investment readiness and not all businesses are ready for investment readiness Investment readiness requires greater resources – look to online programmes for more basic areas of support, and draw in in-kind, pro, or low-bono support where needed; use of country managers and regional leads to identify local investment landscapes Role of country managers Local (in-country) support is key to provide tailored and relevant support; need for clarity on roles of country managers and support for local mentoring Continually clarify and modify the role of country managers, to fill gaps in support to businesses, and role in SPRING M&E; role of country managers should be made clear to businesses from the start; M&E Role of M&E in not only building knowledge products / evidence-base, but in coordination and alignment on understanding of key business elements, such as prototypes and girl impact M&E in important role to continually monitor and promote girl impact pathways throughout duration of prototype refinement Improvements to SPRING’s M&E system have allowed it to further learn and adapt An important limitation to adaptation and learning in Cohort 1 was the tight timelines in which SPRING operates, and therefore the lack of opportunity to document processes and reflect. However, SPRING has addressed this by improving their monitoring and evaluation (M&E) system. The M&E system allowed the SPRING team to better COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 27 document its processes (and therefore learn from them) as well as to create alignment across the team, making decision-making easier. An important component of this has been the development of girl ToCs for each business, ensuring a common understanding on the precise prototype being developed by the business during bootcamps, as well as its potential girl impact. The other piece [that has improved] is around the [key performance indicators] and the M&E side of things. [The M&E team has] done an amazing job in building up the system and this process by which we can actually document the journey of the businesses from cradle …to graduation, let’s call it… So, actually being able to document not only just having better KPIs and a better way of monitoring them. There’s a wider… story of the business and I think that, through Cohort 2 that has been an efficiency that has really helped further on down the line, particularly as we start to build out and deliver knowledge products... 116 SPRING’s adaptations focus on improving the effectiveness of HCD to refine prototypes SPRING’s programme stakeholders were generally satisfied with SPRING’s responsiveness and proposed changes. As one member of the donor team explained, I think whether the [SPRING] team have managed to execute the changes they agreed to, yes, they have. They said they’d do two boot camps and a more refined selection process and a better process on the grants and they’ve done all of those things. I think their innovation on the girl theories have changed to be used in selection but then also at boot camp with the businesses to identify KPIs. [It] has been really, really effective in helping them to articulate what the impact on girls will be of each business but then also, from that, ‘what do we need to measure’? So I think that fed into both selection and will continue to feed into the monitoring of the programme. So I think that’s been really effective.117 As discussed in Sections 2.3.1 and 2.4, SPRING’s adaptations have been primarily driven by the lens of prototype development and its role as a business accelerator. In other words, SPRING’s adaptations have been driven by the needs of businesses to engage with the programme to launch prototypes. Changes to selection criteria focus on selecting businesses that are more likely to benefit, or take up, the HCD process including being able to iterate on prototype development. The roll out of HCD research across all businesses, and the implementation of a second bootcamp are also focused on improving the effectiveness of businesses’ use of the HCD process to develop prototypes. However, there has been less focus on adaptations that address concerns about girl impact. Concerns were raised about the shift to two bootcamps, where one IP respondent remarked about the Bootcamp 2: “One thing I would say is that the girl piece seemed to be a little less at the forefront”118 Bootcamp 2 noticeably did not include the attendance of the Technical Director. As another IP respondent reported that Bootcamp 2 “needed somebody to say ‘girls’ every five minutes, because the businesses had prototyped away from girls… and needed dragging back again”119 The same applies for the role of the country managers, who play an important role as part of the M&E process. As the programme looks to gather more evidence, it is “imagined that the country managers… from the M&E perspective, have more ownership of the M&E process so that… they would actually understand what all the indicators were and would be able to spot if there were some issues with the data being reported.”120 The role of the country manager in both gathering evidence of girl impact, and reinforcing the girl angle as businesses develop and iterate their prototypes is limited by their lack of resources. Barriers to adaptation still persist, concerning leadership and coordination, targets, and timeframes While SPRING has generally had the freedom and culture to continually adapt, SPRING IP respondents and donors have identified a set of barriers that have limited some of the extent of the programme’s ability to change: • Leadership and coordination was mentioned as a barrier to change. The geographic spread of decision￾makers and an opaqueness of decision-making processes at the top levels of SPRING management have made it difficult for suggestions to be heard when they come from lower levels. There is also a lack of 116 KII Respondent 1. 117 KII Respondent 14. 118 KII Respondent 1. 119 KII Respondent 8. 120 KII Respondent 21. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 28 governance structure and decision-making structure across the top level of SPRING. This is also reflected in issues such as the amount of influence and resources available to country managers. • As an innovative and experimental programme, the need to reach certain indicator targets (such as applications received, girls reached, etc) have posed difficulties to the programme’s ability to adapt nimbly. However, there is some scope to review and if necessary amend these targets on an annual basis, which is not always conducive to the nine-month cohort schedule. • Timeframes and timing posed an additional barrier to adaptation, as the tight timelines (and overlapping nature of the cohorts) have made it difficult for the programme to reflect and absorb learnings between cohorts. One donor commented, “The rapid transition is really a risk for SPRING. For a small implementation team to handle all these transitions - the recruitment, orientation… it must be very challenging, I think, and it’s all got to do with timing.”121 Two of the above barriers identified concern SPRING’s design, rather than implementation. 2.6 To what extent does SPRING represent good value for money? The goals of the VfM analysis are to examine whether SPRING has delivered value for money, addressing key questions around cost-effectiveness, considering SPRING’s added value in terms of relevance, efficiency, effectiveness, impact and sustainability, as well as to provide lessons learned and recommendations on how best to generate value at the least possible cost to inform iterative adaptations on the SPRING programme. It should be noted that a full VfM analysis will not be possible until full financial, impact, and beneficiary data are available, which will be collected until the final cohort of the programme. Within the context of SPRING, the first step in a VfM analysis is the production of a VfM narrative, which outlines the key design, development and delivery factors that have driven decisions about the value that the programme intends to generate, and the different types of costs incurred as a result. It is against these key cost drivers that inputs can be judged to have incurred the least cost necessary. These considerations include: a proper consideration and analysis of options; sufficient and appropriate use of evidence and information; and iterative, adaptive and responsive programming. As part of a preliminary VfM narrative, we have identified key factors affecting VfM in the programme thus far: Economy • Cost drivers: The IP have cited extremely tight budgets, stating that “the only leeway we’ve had has been on the savings front where we’ve done, I think, a very credible job of maintaining very, very tight cost discipline on everything.”122 Despite this, SPRING has been able to negotiate budgetary changes for key adaptations in the programme, including the addition of a second bootcamp. Nevertheless, there is a divergence of opinion within the IP consortium as to whether spending on events such as bootcamp is justified in terms of the ‘value’ derived. While bootcamps have increased the effectiveness of some of the technical assistance, others believe it to be a bloated cost that could be streamlined. • Changed approach to funds and fundraising: A key change has been in the allocation of funding as more needs-based and differentiated, targeted specifically towards prototype piloting and launching. This pivot has provided SPRING with a level of flexibility to respond to businesses’ needs and simultaneously hold businesses more accountable in their spending. In tandem, the broader fundraising approach has seen a shift from seeking cash contributions in corporate partnerships to acquiring in-kind donations, or through pro and low-bono support in strategic partnerships, in an effort to raise funds for a fourth cohort. This decision was made in order to utilise corporate partners’ skills and expertise to help unlock businesses’ potential and help them scale while offering partners an attractive opportunity to gain insights into emerging markets, as well as to engage employees. As part of this process, SPRING has surpassed its fundraising target. Efficiency • Extremely tight timelines: In the context of short time frames, SPRING has been able to deliver a large amount of programming. Several inefficiencies did emerge, including inefficiencies at bootcamps (e.g. too 121 KII Respondent 6. 122 KII Respondent 7. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 29 many staff present) and through the Research-in-Context phase, where in some cases, research did not necessarily yield insights. There are still risks in the rapid transition, both between cohorts and between regions, based on design demands rather than businesses’ needs. In one example, one donor voiced concerns that: [Businesses] may start developing a new product or a service or they may increase the girl market, they may extend it and diversify it, and then all this takes time. And then, when they’ve started to do it, they may lose SPRING’s support because SPRING has now moved on to the next cohort. It would have been much better if there was a budget…to support the grantee for the longer period of time [and] provide a different kind of support. 123 A member of the IP team further commented, “It’s not just [focussing] on one country at a time. It’s focussing on two continents at a time. So that brings gaps to the country.”124 Effectiveness • Tailored support and delivery of inputs: The majority of businesses expressed satisfaction with SPRING’s technical and financial assistance, citing positive effects on their understanding and use of HCD – despite some frustration regarding the quality of prototype research conducted between bootcamps. Throughout Cohort 2, SPRING has moved away from providing disjointed, general support towards providing a tailored set of support that has been designed and delivered in a complementary way. • Constant changing and adaptation of the programme: While there is strong evidence of the reflexive and iterative adaptation of the programme both between and during cohorts, these decisions are driven by selecting businesses who are likely to succeed in the programme (such as in selecting businesses determined by their size and maturity). Yet, as six businesses had not launched after 12 months, the selection criteria has not necessarily been justified in the end. Still, given these shifts, SPRING has been considered to have been executed efficiently, although there remain concerns about “productivity leakage” from coordinating the programme virtually (from the UK) – as a function of budget constraints – as well as from being bound by strict targets and timeframes. Equity • Changes to types of girl impact: While SPRING has continued to learn from the ways in which girls are impacted through business interventions, girls from the bottom of the pyramid have been removed as target beneficiaries in Cohort 2. SPRING has broadened its reach to ‘vulnerable’ girls; however there is currently no precise definition. More broadly, in selecting businesses, there were clear tensions between selecting businesses that were able to reach scale and those that were directly targeting girls. In order to further build a VfM narrative, we will continue assessing key decision-making processes of each cohort, contributing to a summative evaluation of SPRING’s VfM that takes into consideration impact through the BPE and IE. We will continue with the verification of SPRING’s in-kind contributions in order to ensure that all reported contributions are acceptable types of contributions, that calculations to estimate the value of pro-bono inputs are based on sound logic and evidence and that sufficient evidence exists to prove savings made. As of 19 January 2018, we have been able to fully verify a total of £729,994 of the total of £1,081,811 recorded and raised by the IP. The remaining value is still undergoing final verification. The bulk of the in-kind support has been provided by Hogan Lovells, who have provided businesses with pro-bono legal support and advice since Cohort 1. Pending the final verification of these figures, the IP have been successful in meeting the related logframe targets to date. 123 KII Respondent 6. 124 KII Respondent 3. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 30 3 Conclusions SPRING was designed as an innovative and experimental programme, and has continued to revisit its processes and activities, making changes and pivoting in implementation in direct response to challenges. Lessons from Cohort 1 were applied to implementation in Cohort 2 ranging from scoping and business identification and selection, to programme delivery and business support. This section summarises the lessons learned in the programme level evaluation of Cohort 2. Programme Design SPRING continues to learn and adapt, which is a strength in its implementation, but its institutional learning might be better recorded. SPRING has often implemented changes with little recorded reflection on ToC. While the programme continues to evolve and learn, these learnings are often not placed within the larger experimental context for the hypotheses and assumptions at a theoretical level, which is encapsulated in the ToC. The motivation behind programme changes is not always clear, as the programme does not always clearly test hypotheses and report results. As a result it is not clear whether SPRING is solution-driven in trying to solve an existing problem, or trying to identify problems in need of solutions. As an illustrative example: SPRING has made a number of alterations from Cohort 1 to Cohort 2 evidenced in the significant Cohort 2 adaptations supporting the delivery of HCD training (e.g. two boot camps). These changes were largely driven by the perceived need of businesses to effectively learn and use HCD following reflection on Cohort 1 weaknesses. As a result SPRING changed their delivery methodology and process. However, these changes do not necessarily reflect, or correlate with, creating more (i.e. greater or deeper) girl impact. An ongoing learning area for the programme is “What constitutes its primary focal area?” - that of a business accelerator, providing business assistance to businesses which have a focus on girl-related products and services; a girl-focused accelerator providing assistance to businesses to identify girls as a potential market; or an accelerator assisting a range of businesses to improve the products and services to the client base, some of whom are SPRING target girls, by providing instruction in human centred design. Selection Process It has become increasingly evident that selecting the correct companies (and the correct mix of companies within a single cohort) is key to the success of the cohort and the cohort’s contribution to SPRING success. The centrality and gravity of identifying the most suitable businesses makes this an ongoing area of discussion and debate. In Cohort 2 SPRING opted for an equal mix of more mature businesses, growth oriented businesses and early stage businesses, following a learning from Cohort 1 of having too many concept / idea or early stage businesses, and this impacting negatively on the programme’s potential to realise its targets. The lessons from Cohort 2 in this regard are that while mature businesses may be more suitable for SPRING’s purposes to achieve scale; larger, more complex corporate style businesses, are not generally suited to the “hands-on” SPRING approach, and are more disinclined to implement HCD as a product or service development process. Implementation In Cohort 2 the role and importance of the country managers increased significantly; from initial networking prior to selection, to building an ongoing pipeline through the selection process, to providing support to the businesses during their prototype phase, and ongoing support and liaison between the businesses and the programme. The lack of resources allocated to this role, which is becoming increasingly central to the success of the programme, is an area of concern. While the implementation of a two boot camp model comes with an increased cost in terms of programme resources as well as substantial investment in terms of time and personnel commitment from the selected businesses, the additional interaction and support provided at this second touch point allows for more comprehensive and cohesive support. A change in the allocation of PDF funding from a set allocation to a more competitive, business oriented “challenge fund” provides the programme with the level of flexibility to respond to business needs. This revision also ensured that businesses use this resource as planned, prioritising their funding requests as well as holding them more accountable; a further lesson from Cohort 1. Some of the business participants criticised this competitive pitch COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 31 process, but the revised model increased programme control over this resource, allowing for more nuanced and guided support. The process and messaging around the PDF process continues to evolve, as SPRING seeks to place less emphasis on financial assistance. One area where there is a need for increased and ongoing support to the businesses is that of girls support. If the programme wants the business to continue to innovate products and services applicable to girl beneficiaries, these businesses need ongoing support in this area. This is especially true if the programme seeks the selected businesses to increase their girl impact and scale post-SPRING. Impact on Businesses The motivations for business involvement in SPRING tell an important story and may provide some guidance in identifying additional selection criteria. Businesses reported that the primary motivations included the furthering the social impact, identifying adolescent girls as a means to attract impact investment support as well as increasing the business market scale by reaching new girl customers. The vast majority of businesses selected for Cohort 2 expressed satisfaction with the SPRING processes, inputs and technical assistance, although there were some grievances raised regarding the quality and usefulness of the prototype research conducted between boot camps. Business also reported that they generally found HCD useful and have used the approach to design, and refine their business prototypes. Some businesses have also reported using the approach in other, non-SPRING related areas of their business. However, there is not yet any concrete evidence that the businesses using this approach will continue to do so or that the approach has increased the businesses’ ability or tendency to improve the provision of products and services to the girl target audience. An assumption within the SPRING theory of change is that selected businesses will increase their investment readiness within the timeframe of the SPRING programme. However, the process for ensuring that this happens within the anticipated timeframe is not explicit and makes numerous assumptions for example, the availability and willing state of investors. There is also no clear explicit link between the success of facilitating investment within SPRING businesses and the creation of a wider, more diverse “investment friendly ecosystem” for post-SPRING businesses to access. Cohort 2 businesses appeared to be more investment ready than their counterparts in Cohort 1, improving the chances of the businesses to achieve scale. This improved readiness was largely due to both the status of the businesses to attract and utilise investment support as well as the change in the delivery of investment readiness support. All businesses, including those who might need some time to be investment ready, were provided with access to online support from Duke University’s CASE Business School and mentoring support from the Regional Investment Support Manager, improving the likelihood of all businesses attracting some investment. However, at the time of writing, it is too early to tell if investment readiness support has been effective, as many businesses were still in the process of attracting and negotiating investment, with only two businesses of the cohort having finalised investment. The provision of investment support is a further area that needs increased resource allocation if the programme seeks to increase and improve the girl focussed outreach of the businesses, post-SPRING. Impact on Girls The programme continues to debate and learn from the differing ways in which girls are impacted through business interventions, by scale and by depth. In moving from Cohort 1 to Cohort 2 the programme expanded beyond BOP girls as a target audience. This is allowed the programme and the businesses more latitude in Identifying, developing and prototyping appropriate products and services. However, the programme still has to define “vulnerability” to clearly ensure that it meets logframe targets. The lack of the current definition of this term may prove to be an Achilles Heel for the programme learning. After Cohort 1 SPRING opted to move away from including girls in the value chain. However, some Cohort 2 businesses have continued to pursue the involvement of girls in the value chain, although admittedly on a smaller scale than planned in Cohort 1. The programme considerations of girls in the value chain regarding programme scale and impact over the longer term continue to remain an area of debate amongst the implementing team and the donors. SPRING is currently embarking on Cohort 4 and, through ongoing liaison with the M&E Partner, has already adapted its processes as a result of reflection on many of the lessons outlined in this report. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 0 4 Lessons and Recommendations SPRING is a dynamic and iterative programme which is constantly seeking to improve. This section outlines some the key areas of learning resulting from a reflection of the implementation of Cohort 2. The IP and the M&E Partner have a close working relationship, which while not impeding on the independence of the evaluation, adds to the constructive iterative nature of the programme, allowing for independent reflection on processes that might be improved. As a “critical friend”, the M&E Partner endeavours to provide regular and constant feedback to the IP. Recommendations have been provided informally throughout the cohort (and at key periods as the programme has begun implementing Cohort 3 activities), and articulated more formally in evaluation reports. The IP has taken on board many of the recommendations set out in the Cohort 1 PPE report, and continues to apply recommendations as they emerge. As a result, many of the lessons reported here have already been actioned and recommendations implemented both for Cohort 3 and ahead of Cohort 4. The degree of implementation is reported in the final column in Table 5 below. Some recommendations are more pertinent to future iterations of the programme, and these are reflected in bold in the table below. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 0 Table 5: Cohort 2 PPE lessons learned and recommendations Lesson Learned Recommendation Degree of Implementation Cross-cutting 1. The Theory of Change is a tool for implementation, learning and evaluation. The programme ToC can be used as a guiding tool for programme implementation and as a reflective tool for charting progress against the planned path of action. In an experimental programme, the ToC plays a more central role in the decision-making process, even if this role is only the recording of changes made. While a ToC cannot capture the granularity of all implementation details, it can capture key changes made to the programme, such as in the selection of businesses and the structure of support to businesses. Capturing this is crucial as these changes all speak to the DNA of the programme and its potential successful replicability,and should be reflected in the ToC. SPRING should more regularly record changes and details relating to the implementation on the programme level ToC. This will give SPRING opportunities to reflect on the planned route of implementation and the rationale for variations to this route. It also provides background for SPRING to test programme changes and iterations against the theory of change assumptions. As the programme heads towards its last planned cohort, SPRING should ensure that there is an opportunity to reflect on the ToC before the programme ends to capture learnings from all cohorts and to match evidence against it as well as identify evidence gaps. In future iterations of a programme it is recommended that the ToC is regularly unpacked and variations noted. Where possible, evidence for these variations and rationale for changes should be recorded. This recommendation has been implemented. SPRING has implemented this recommendation after the Cohort 2 PPE emerging findings workshop in July 2017. SPRING is in the process of revising the ToC following further changes to the logframe after the 2017 Annual Review. SPRING should continue to implement this recommendation before the programme end. This recommendation should also be taken up in the design of future programmes. Relevance 2. A multi-engagement and iterative selection process is foundational in the formulation of anticipated outcomes during the 9- month engagement process. The refinement of the selection process includes relying more heavily of local knowledge and switching to a series of interviews rather than a one-off pitch process. This allowed SPRING more opportunity to understand the context and requirements of the short-listed SPRING should continue to maintain its multi-engagement selection process. Future programmes should also utilise a series of engagements in the selection process, rather than relying on a one step application process. SPRING has continued to implement (with further iterations and improvements) these principles for the selection process for Cohorts 3 and 4. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 1 Lesson Learned Recommendation Degree of Implementation businesses and engage with the businesses’ decision makers regarding their needs. 3. Key implementation targets should be identified and acknowledged as programme drivers. Programmes face considerable pressure to meet logframe targets, even when they are largely on track to deliver. While clearly defined metrics have a role to play in determining whether programmes have achieved the desired level of success, funders need to be aware of inherent dangers of skewing the incentive for experimental, innovative programmes having to achieve numerical targets. These drivers can impact a number of aspects of the programme. In the case of SPRING, this has impacted on key elements such as business selection, business prototyping, and therefore depth of girl impact. When designing programmes of experimental nature, consideration should be given to the kind of, or range of metrics against which the programme implementation and success are assessed, rather than the use of definitive targets. This recommendation is not for implementation during the lifetime of SPRING and should be taken up in the design of future programmes. 4. With SPRING’s model of using HCD to help prototypes scale, there are limitations and challenges in working with large, mature businesses. There were evident differences in perceptions, anticipated goals, degrees of support, and decision-making process between larger, more complicated businesses and smaller entities. This can lead to different levels of commitment and mismatched expectations of funding support. In Cohort 2, two of the more established, mature businesses did not participate or benefit as fully as possible from the programme. In selecting and supporting mature businesses, SPRING should develop measures to manage expectations and ensure that all cohort participants receive (and are seen to receive) equitable levels of input and support. SPRING should ensure learning from what works and what does not in engaging with larger, more mature businesses. This should be reflected in the design of future cohorts. This recommendation has been implemented. SPRING has identified the challenges of working with large businesses and have amended their selection criteria to include ‘leadership’ to reflect the need for the SPRING participant to be influential to implement HCD in their business. Funders and future implementers should consider these challenges in the design of future programmes. While engaging with large, mature businesses can have rewards, they have different needs and expectations of smaller businesses, particularly in the capacity to uptake technical assistance and funding expectations. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 2 Lesson Learned Recommendation Degree of Implementation Effectiveness 5. Tight timeframes (e.g. between cohorts) are a constraint to in-depth learning. The tight cohort timelines have restricted the programme’s ability to reflect and learn as much as possible. While the programme has placed a high value on learning, and works to constantly adapt, these adaptations have not necessarily been tested against a hypotheses but have rather been reactions to circumstance. Future iterations of the programme might consider a longer timeframe between cohorts to allow for more effective programme learning, particularly to test new approaches or are experimental in nature. This recommendation is not for implementation during the lifetime of SPRING and should be taken up in the design of future programmes. 6. Country managers play a central role in the selection process, M&E process, and in providing 1-on-1 mentoring by bridging the gap between key SPRING touchpoints. The utilisation of local knowledge through the country managers, especially in the selection and support processes was an important component in these aspects of the SPRING programme. SPRING should ensure that the role of country managers is clarified, and that they are supported with adequate resources to support these areas of their role. Improved resources could include increased remuneration, increased hours, and an increased, weighted role in selection. Future programmes should ensure they have understood the importance of the role of local managers in providing local context to selection, providing technical assistance, and as facilitators for M&E data collection. This recommendation has been implemented. Learning from Cohort 2, SPRING has already clarified the role of the country manager for Cohort 3 and moving forward, and continues to resource the role accordingly. 7. Programmes with multi-faceted areas of input should acknowledge the multi￾faceted outcome areas that can result. SPRING provides three areas of support to its businesses: technical training on the use of HCD, business accelerator support including investment readiness, and a focus on girls. SPRING can provide support to its businesses in all 3 focal areas; however a danger lies in businesses being more biased in their ability to absorb support in areas with which they are All SPRING businesses receive a level of girl centred programme support. However, given the centrality of girl focus to the programme design, SPRING should bolster its girl-focussed support to businesses and that businesses receive this support post￾Bootcamp 2 period and in alumni support. This recommendation should be considered before the end of the lifetime of the programme. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 3 Lesson Learned Recommendation Degree of Implementation already familiar e.g. investment readiness rather than girl focus. 8. Using research within the HCD process can require specialist resources, and requires adequate time and resources dedicated to supporting businesses in this. For Cohort 2, the roll-out of HCD research for all businesses in the cohort involved a steep learning curve to coordinate the large-scale research across all 18 businesses and three countries. The challenges included ensuring that moderators are sufficiently trained and well versed in HCD research methodology, that fieldwork is conducted with the right respondents, and that findings are synthesised in time for Bootcamp 2. Well-coordinated research is crucial not only for the effectiveness of the HCD process for businesses, but also for the safety if the research participants. Girl safety remains a high priority for SPRING, but the risk to girls’ safety is likely to be higher in a poorly coordinated and managed research process. The difficulties mentioned above in coordinating this research process, magnify this risk. This is also an important learning for future iterations of the programme. SPRING should provide greater clarity on the goals and purpose of HCD research to participants, as well as its limitations. This applies to not only the businesses and research moderators delivering the research, but also to the research coordinators. Given the specificities involved in HCD research, this clarity is particularly important for the coordination of the research and the remit and capacity of the research coordinator. Context-specific learning on implementing HCD in South Asia should be applied for Cohort 4. Given the difficulty in implementing well￾coordinated but tailored research activities, future programme learning should account for this in the design and resource allocation. Future programme should ensure contracted third parties responsible for any deliverables are i) well-versed in and able to apply girl safety standards and ii) well-trained in unique methodologies (in this case, HCD), to ensure adequate delivery to the programme. SPRING continues to iterate on the best way to resource external logistical support for delivering HCD research. Learning from these experiences has been applied for Cohort 3 and ahead of Cohort 4. Part of this recommendation is not for implementation during the lifetime of SPRING and should be taken up in the design of future programmes. 9. Investment readiness support should be both tailored to the different level of needs of businesses, but also be scalable. SPRING has recognised that the entrepreneurs are all in different states of readiness regarding their potential for outside SPRING should continue with the provision of one on one support to businesses that are in a position to benefit at the time of the bootcamp. This recommendation has been partly implemented, and should continue to be considered before the end of the programme. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 4 Lesson Learned Recommendation Degree of Implementation investment, impacting on their ability to absorb and make use of investment readiness support. To address this, SPRING has provided both tailored one-to-one support and time from the Investment Director, but also a scalable solution by supplementing this with an online course. This also allowed businesses to implement steps towards investment readiness, at their own pace. SPRING should consider allowing all businesses to access the online course support offered by Duke, to be used when the businesses are in a position to benefit. To facilitate this process SPRING should look to providing some investment readiness support in its alumni programme / local network support. As the delivery of investment readiness support has evolved and given a more defined timeframe within the 9-month accelerator programme, the M&E system should ensure that it is documenting the investment readiness journey more clearly. Future programmes should consider the need to extend the timeframe for businesses to adequately improve their investor readiness. It is also suggested that future programs consider easily scalable solutions, such as online support. Part of this recommendation is not for implementation during the lifetime of SPRING and should be taken up in the design of future programmes. Sustainability 10. To embed HCD in SPRING businesses, HCD training should go beyond the one SPRING entrepreneur. SPRING invests considerable support, time, and resources into training business entrepreneurs in how to use the process of HCD in their businesses. As SPRING looks to contribute to the sustainability of its impact on businesses, it should recognise that businesses face challenges and require different types and levels of support to apply HCD within their own organisations. To embed HCD beyond the entrepreneur in SPRING businesses, SPRING should examine how best to provide additional resources and toolkits to entrepreneurs that allow them to share them with their teams. An optional module on how to ‘teach’ HCD could also be added to the bootcamp curriculum or as alumni support for those entrepreneurs who are interested. This recommendation should be considered before the end of the lifetime of the programme. COHORT 2 PPE REPORT SPRING MONITORING AND EVALUATION – MARCH 2018 5 Lesson Learned Recommendation Degree of Implementation 11. Sustaining the impact of SPRING on businesses will benefit from clearer plans for alumni support and networks. An alumni component was developed as part of the initial design of SPRING but has never been clearly articulated. There is a need and request for post -SPRING support for businesses. SPRING should finalise its post-SPRING alumni support process and communicate this to the businesses in all its cohorts. Objectives of the support could include building business capacity in HCD, reinforcing the centrality of girls within the businesses mind-set, and continuing to improve the investment readiness of the businesses. Future programmes might consider ensuring that post-programme support is designed and tested earlier in the programme, to allow iterations and adjustments to be made. Future programmes might also consider building the following elements as part of the effort to ensure that alumni networks are able to continue beyond the lifespan of SPRING support: • building the capacity of local business networks • drawing on their country managers to develop this alumni support network • support peer learning opportunities between SPRING businesses and businesses of similar programmes This recommendation should be considered before the end of the lifetime of the programme.