The Department for International Development SPRING Monitoring and Evaluation May 2018 Coffey International Development Ltd The Malthouse 1 Northfield Road Reading Berkshire RG1 8AH United Kingdom T (+44) (0) 1189 566 066 F (+44) (0) 1189 576 066 www.coffey.com Registered Office: 1 Northfield Road Reading Berkshire RG1 8AH United Kingdom Registered in England No. 3799145 Vat Number: GB 724 5309 45 This document has been approved for submission by Coffey’s Project Director, based on a review of satisfactory adherence to our policies on: • Quality management • HSSE and risk management • Financial management and Value for Money (VfM) • Personnel recruitment and management • Performance Management and Monitoring and Evaluation (M&E) Heidi Ober, Programme Director Signature: Cohort 2 Synthesis Report May 2018 COHORT 2 SYNTHESIS REPORT SPRING MONITORING AND EVALUATION – MAY 2018 3 What is SPRING? The SPRING programme supports the empowerment of adolescent girls in developing countries in four areas: helping girls to earn, learn, save, and stay safe and healthy. The programme uses businesses as “bridges” to reach girls and works with the businesses to view girls differently – to see their needs and develop business products and services to meet these needs. SPRING’s implementation team is made up of programme partners including fuseproject as the human-centred design (HCD), branding, and industrial design experts, a team of global thematic experts, and is led and managed by Palladium. The programme supports four cohorts of businesses, in multiple economic sectors to reach vulnerable girls in nine countries throughout East Africa and South Asia. Each cohort comprises a number of different programme components including a business selection process, a girl and business landscaping process, a nine-month accelerator programme consisting of two bootcamps, a business￾oriented Research-in-Context phase, and an end of cohort event. SPRING is funded by DFID, USAID and DFAT. Purpose of this report As the independent Evaluation Partner for SPRING, Coffey conducts an ongoing evaluation of SPRING through three interlinked processes: a Programme Performance Evaluation (PPE), a Business Performance Evaluation (BPE) and an Impact Evaluation. PPE and BPE have been completed for Cohorts 1 and 2, and the findings, lessons learned, and recommendations have been shared with SPRING’s donors and implementing partner. This synthesis report is a snapshot summary of the Cohort 2 PPE and BPE reports, giving insights into how, where, and why SPRING has evolved and changed. Midway through SPRING, what have we learned about supporting businesses? How does SPRING support businesses? As a business accelerator focussed on helping businesses empower girls, SPRING offers three categories of inputs: • Technical assistance focusing on the HCD process as part of Bootcamps 1 and 2 and Research-in-Context phase; investment readiness support; and in￾country support provided throughout the cohort; • Financial assistance, through the prototype development fund; and • Additional ad-hoc areas of support, including: in-kind, low, and pro-bono technical and advisory services, and mentoring. How has SPRING changed its support to businesses? Learning from its previous experience with Cohort 1, for Cohort 2, SPRING increased its touch points with businesses from one to two bootcamps. Prior to SPRING, most businesses had little or no understanding of HCD; this second bootcamp allowed for more effective delivery of technical assistance, particularly with sequencing the HCD process. It allowed businesses the time, space, and support to come back to iterate and refine their business prototypes. To further support this process, HCD research was expanded from five businesses to all Cohort 2 businesses. What is HCD in the context of SPRING? Human-centred design (HCD) is an innovation process, used to better understand the needs of end users, before designing and implementing solutions. HCD is core to the SPRING programme and is used to help businesses design, iterate, and refine their prototypes that ultimately serve to empower girls. The HCD process begins at Bootcamp 1 where businesses learn about HCD, begin to gather insights on girls, and first develop their prototype. Following this, they participate in the research-in-context phase to test their prototypes with real and potential users. They bring these insights back to Bootcamp 2, where they work with HCD specialists and technical experts to revisit and improve their prototypes. COHORT 2 SYNTHESIS REPORT SPRING MONITORING AND EVALUATION – MAY 2018 4 In terms of financial assistance, in Cohort 1 funding was provided in equal amounts to all Cohort 1 businesses. For Cohort 2, SPRING learned to encourage businesses to focus funding and businesses activities on prototype development and launch. Grant funding was changed to the Prototype Development Fund, (PDF) where businesses were awarded varying amounts of funding based on their applications and “pitches”. The investment readiness – a business’ ability to attract and manage investment - support underwent restructuring, with revised timelines for the support provided and the addition of an online course. This comes from learning that businesses come into SPRING with different degrees of investment readiness, and often need very different levels of support. In many cases, businesses need more than nine months to move from their pre-SPRING state to be investment ready. What type of businesses are the right “fit” for SPRING? In using businesses as their “bridges” to reach girls, SPRING select those that they thought would best benefit from SPRING’s technical assistance and expertise while still meeting programme targets. The selection process evolved between the first two cohorts, focussing not only on type of businesses selected, but also how the programme identified and interacted with these businesses. Some selection criteria remained from Cohort 1 (medium sized businesses, social component, expansion goals, etc.), while other criteria were added such as business longevity, profitability, team make-up, and history of innovation. Importantly, SPRING included the existence of a defined prototype as a selection criterion, ensuring that the business was beyond basic conceptualisation. It appeared that this criterion was important: businesses that appeared to gain the most from SPRING tended to be those that had already invested effort in their prototype. Cohort 2 included more mature businesses and businesses that were more likely to benefit from, or take up, HCD as a development process. As a result of revising the selection criteria, SPRING encountered challenges in meeting an extremely diverse set of business needs across Cohort 2. Given the range of levels of maturity of the Cohort 2 businesses, tailored support proved fruitful but challenging. SPRING recognises that not all the selected businesses are necessarily a right “fit” nor will they continue to target or impact girls, post-SPRING. And this might be acceptable as SPRING as an innovation programme, not all innovations can necessarily be successful. Have businesses found HCD useful? HCD is a core part of SPRING programming and is intended as a method to help businesses view end users differently and to assist them in understanding this group’s real-life behaviours, attitudes, and needs. SPRING’s aim with using HCD is to help businesses design prototypes to better impact girls, through increased reach or addressing unmet needs. The roll out of HCD research across all businesses, and the implementation of a second bootcamp focused on improving the effectiveness of businesses’ use of the HCD process to develop prototypes. In SPRING, businesses used HCD to both prove and disprove assumptions they held about their prototype end users and adolescent girls. Businesses were generally positive regarding HCD, explaining how the approach and the related support has helped them to think differently to understand the needs of the end user, to generate ideas, and solve design challenges. As a result of HCD, some businesses significantly redesigned their prototype during the accelerator programme. As HCD encourages businesses to constantly iterate on their prototype, in some cases this also meant that businesses learned to move away from girls as a potential end user market. It was found that HCD does not always result in adaptations that improve the effectiveness of girl impact. Application and usage of HCD amongst businesses was mixed. Some of the businesses have reported using HCD in wider application throughout their business processes, and not limiting it only to understanding and meeting girls’ needs. But at the same time, businesses reported the approach as being time and cost intensive, and only able to focus on small groups of end users. Most of the COHORT 2 SYNTHESIS REPORT SPRING MONITORING AND EVALUATION – MAY 2018 5 businesses from Cohort 1 reported an ongoing use of HCD for business and prototype learning, while some businesses stopped using HCD within the business because of internal constraints. Have there been any unexpected challenges in time and resource? SPRING thought that a 12-month window (the nine-month accelerator programme and three months following) would be sufficient for a business to engage with the programme and launch a prototype. But for one-third of the Cohort 2 businesses, this timeframe was too tight. Two of the Cohort 2 businesses are expected to launch prototypes outside of the 12-month window and four are not expected to launch at all. These same four received no prototype funding from SPRING, with two of them effectively disengaging from SPRING. While businesses responded positively to HCD, it is not a widely understood research and design methodology. Identifying additional support resources for businesses to adequately apply it has been expensive and time consuming, with HCD research in particular hampered by logistical and coordination difficulties. What have we learned from SPRING about programme design? Selecting the most appropriate businesses takes time. There are numerous factors to consider and the largest, or the most mature businesses, do not always appear to be the most appropriate for this type of engagement. We have learned that it is better to select businesses through a process of multiple interactions (such as a carefully curated pipeline that starts looking for businesses early in the process), than through a once off procedure (such as a selection ‘pitch’ event). Different programme stakeholders have different priorities. SPRING has three focuses, as a programme: it is a business accelerator, it is a programme that uses HCD, and it is a girl-focused programme. The three facets of SPRING implementation create some different programme expectations amongst the stakeholders. While businesses might be ready to absorb business or investment related learning from SPRING, and the implementing partner wanting to see proof-of￾concept of their programme design using HCD, the donors would often like to see a greater absorption of girl-focused learning. Having targets can impede an experimental programme. Targets are only one way of measuring process and progress and ensuring that the programme is accountable to reaching its potential. But when a programme that is tasked with exploring, experimenting, and iterating new approaches to development is tied to specific numerical targets, it can perversely incentivise the programme to “play safe” and lean towards a more conservative application. Businesses think differently to non-profits and development programmes. As part of their participation in SPRING, businesses are obliged to collect and report on data to contribute to programme reporting. While businesses are adept at collecting and interpreting business data, they are not always well equipped to, or see the benefit in, collecting data on impact or reach. In this sense, the expectations of businesses to collect data to contribute to programme targets should take into consideration both a.) the realities of the businesses’ capacities to collect data; and b.) their desires to do so. Programme learning also takes time. SPRING’s original timeframe was three 12-month cohorts, subsequently compressed to four 9-month cohorts. SPRING has implemented well within these timeframes, but the compression has restricted the programme’s ability to reflect and learn as much as possible; this is a weakness for a programme which values learning and works to constantly adapt and improve. HCD is a resource and time intensive process. Just as the businesses have reported that applying HCD within their own businesses requires time, resources and effort, the same hold true in training businesses to use HCD as a methodology, particularly in conducting prototype-linked research. Peer learning and networks has been a positive, unintended benefit of SPRING. Through the SPRING accelerator design as cohorts, businesses have been able to form peer networks and share learning. Although not an anticipated outcome, it has been an unintended way for SPRING to start to change the market for businesses to think about girls. COHORT 2 SYNTHESIS REPORT SPRING MONITORING AND EVALUATION – MAY 2018 6 Midway through SPRING, what have we learned about reaching girls? How has SPRING changed in its approach to reaching girls? Following Cohort 1, SPRING opted to move away from directly targeting Base of the Pyramid girls and to work with businesses to target vulnerable girls. Vulnerability covers a range of susceptible areas where SPRING businesses can positively impact on girls’ lives, and is still seen as valuable. SPRING has reduced an emphasis of targeting girls in the value chain. While the programme has learned that reaching girls in the value chain is not always feasible, it has also learned that focussing on girls as the end user eases the opportunity for businesses to achieve scale. In Cohort 2, seven businesses originally proposed reaching girls through their value chains, but with SPRING’s input and advice to refine their prototypes, only three implemented a value chain prototype. It is too early to tell whether reaching the girl as end user is as impactful as placing the girl within the value chain. These changes have been driven by the needs of businesses to launch prototypes, rather than a timely activity to address problems and needs of girls. This tension reflects some of the challenges that SPRING has to address – a need to be adaptive and innovative and still meet logframe targets, and to do so within very specific timeframes. What challenges has the programme faced in using business to reach girls? While SPRING has identified adolescent girls (aged 10-19) as a vulnerable and underserved demographic, for businesses to try and only reach girls, within specific age parameters, has been challenging. While these parameters might be suitable for a development programme, for businesses the parameters make less commercial sense. Businesses often want to reach girls beyond these age groups to target both younger and older girls, depending on their product or service offering. But in working towards programme targets, businesses have had to restrict their focus to these programme-defined parameters. In initially trying to target girls at the Base of the Pyramid, business found that girls’ access to economic resources to purchase the businesses’ products or services was a challenge. Only three businesses identified girls as purchasers while twelve saw girls as their end users. As mentioned above, reaching girls in the value chain has proved challenging for many businesses. Cohort 2 businesses that have looked to place girls in their value chains have faced a myriad of challenges. For example, unless the business was already in a position where girls in the value chain were central to their business model, it has proved difficult to add them. What have business learned about how to reach girls? Cohort 2 businesses have identified channels that impede and facilitate access to girls as a market. Their insights include: • Using personal networks and utilising girls as mobilisers and brand ambassadors works well to attract further girl customers; • Socio-cultural settings that prevented girls’ mobility made reaching girls more difficult, but can be eased through the use of ICT; • Trusted older female relatives often acted as gatekeepers and either made decisions on behalf of girls or held significant sway in their decision making; and • While in some cases they can be barriers, male relatives (such as fathers and brothers) can also be important allies to accessing girls. In particular, brothers can be important champions for girls at the household level. What happens to businesses after SPRING? As part of our evaluation, we followed up with seventeen of the eighteen Cohort 1 businesses. Eighteen months after the end of their engagement with SPRING, the majority of Cohort 1 businesses continue to tweak and iterate their SPRING-initiated prototype. Most businesses have continued their prototypes and continue to reach girls to at least some extent. Eight are in the COHORT 2 SYNTHESIS REPORT SPRING MONITORING AND EVALUATION – MAY 2018 7 process of scaling-up in other markets, regions, or countries. One business is no longer specifically targeting girls, but is trying to reach them as a subset, having expanded their target market to include boys as well. Another business is trying to instead reach older girls and women. Only two businesses have decided to discontinue their prototype, especially in working with girls. While one continues to work with girls, but is in the process of developing a new model after realising weaknesses in its former version. A number of businesses reported feeling more equipped to attract and facilitate outside investment as a result of SPRING, and the reputation of their businesses has benefitted as a result of participating in SPRING. Five businesses reported new partnerships since participating in SPRING and four businesses reported an increase in competition for their target markets reflecting an interest in the broader marketplace for products and services for adolescent girls. What have we learned from SPRING about reaching girls? Businesses have many reasons for reaching girls, but paramount amongst them is commercial viability. Businesses will not remain constrained to programme parameters if it does not make commercial sense. Business remain commercial entities and a development programme needs to engage with them on this level. SPRING provides the businesses with this flexibility, and as an example, recognises that to limit the business’ scope to adolescent girls can limit a prototypes’ commercial viability. SPRING only counts the girls reached as defined by the programme’s target of adolescent girls, but this means that the programme’s reach may be significantly larger than reported. Reaching girls is not a direct process, and often incorporates multiple potential routes. SPRING businesses often engage with “gatekeepers” or economic decision makers as end users to reach adolescent girls. These gatekeepers might include parents, older siblings or older family members. Importantly, men and boys play an important facilitative role in accessing adolescent girls and in promoting access to business’ prototypes. Therefore, the route to reach girls through prototypes is not always direct. In addition, the adolescent girl market overlaps with numerous other markets including older and younger girls, older women, female relatives, male relatives and male siblings. It is easier to reach girls as end users than to incorporate girls into a business’ value chain. At the prototype ideation and design stage, many businesses proposed to incorporate girls into their value chain, recognising the deep impact that meaningful employment can bring to older adolescent girls. However, this was not always a commercially-viable solution, if businesses were not previously involving girls in their value chains or there was not a commercial argument for this solution. Both SPRING and businesses also realised that the potential to reach girls in the value chain meant sacrificing breadth for depth by reaching less girls and not necessarily achieving scale. Further Reading Coffey, SPRING Cohort 2 Programme Performance Evaluation Report, March 2018. Coffey, SPRING Cohort 2 Business Performance Evaluation Report, March 2018.