1 Global LEAP Off-Grid Refrigerator Competition Evaluation Report 2 3 Executive Summary 4 Introduction 6 Structure 7 Section 1: Overview of the Competition 8 Why a Competition? 8 Why Uganda? 10 Theory of Change 14 Competition Design and Structure 16 Competition Timeline, Actions, Results and Learnings 26 Competition Winners 28 Innovation Prize Winners 30 Section 2: Findings 32 Impact Approach 33 Competition General Findings 36 Innovation 40 Capabilities 48 Ecosystem 56 Value for Money 64 Section 3: Lessons Learned 68 Learnings 68 Challenges and Opportunities 76 Recommendations 76 Conclusion 79 References 80 Acronyms 81 Appendices 82 Impact Framework 82 Sample Field Test Report 88 Partners 92 Contents 4 5 Key Findings Depending on the test, 12 to 13 Global LEAP Off-Grid Refrigerator Competition finalist devices outperformed off-grid refrigerators available in the market in four major tests (steady-state operation [10°C, 32°C, 43°C] and pull-down time). 96% of field testers were happy with the off-grid refrigerators and 100% found the products easy to use. Four nominees had never worked in the off-grid refrigeration market before the competition. Over 50% of finalists formulated partnerships to further develop or produce their refrigeration devices with a further 25% optimistic about building partnerships in the near-future. Over 1,000 Global LEAP finalist off-grid refrigerators were sold in East Africa as part of Global LEAP Off-Grid Appliance Procurement Incentives Programme. 84% of field testers would purchase the off-grid refrigerator they tested. 95% of field testers found that they had increased sales by using the off-grid refrigeration devices. 74% of field testers would be willing to pay between $200 and $700 for a solar home system-compatible off-grid refrigerator. There are now 17 independently verified, high quality off-grid refrigerators on the market. On average, 78% of competition partners, judges, and finalists felt that the competition had achieved its desired outcomes. Refrigeration holds the unique potential to unlock economic and social progress for billions of un- and under-electrified people. Refrigeration can prolong the nutritional value of food, diversify diets, enable income-generating activities, and reduce the time that households spend shopping or gathering food. That said, the marketplace for off-grid refrigeration is nascent and penetration of refrigerators in developing countries, particularly in rural areas, remains extremely low. Highly energy-efficient, appropriately designed and priced refrigerators are an essential step to the off-grid appliance market’s development and to its subsequent positive development impacts. The 2016-2017 Global LEAP Off-Grid Refrigerator Competition was a partnership of the U.S. Agency for International Development (USAID), UK aid, Power Africa, and the U.S. Department of Energy (DOE); implemented by CLASP, Energy 4 Impact (E4I), and DAI; and evaluated by Nesta and IMC. The competition was launched to stimulate the off-grid refrigeration market in developing countries—particularly sub-Saharan Africa. Refrigerators have yet to significantly penetrate the off-grid market in sub-Saharan Africa. As part of the competition research process, Global LEAP conducted desk research and customer surveys to identify what it would take to improve viability of off-grid refrigeration. The research indicated that refrigeration solutions must be significantly more efficient and lower cost than conventional products found in on-grid communities. The competition sought to address this challenge. This evaluation report presents an in-depth overview of the 2016-2017 Global LEAP Off-Grid Refrigerator Competition’s elements including the research and design process, theory of change, competition milestones such application intake, laboratory testing, field testing, and awards events. The report then profiles the winning and finalist refrigeration solutions, noting both their performance and key characteristics (e.g. volume). After profiling the winners, the report returns to the theory of change and evaluates whether the competition achieved each desired outcome and the decisions or factors that influenced the outcome. Generally, the evaluation finds that while the competition inspired some new technical performance from the devices and the competition’s field test process demonstrated both function and remunerative value to field test participants, more technical innovation, lower prices, policy change, market linkages, and customer awareness are needed for a thriving off-grid refrigeration market in sub-Saharan Africa. The report concludes by highlighting aspects of the competition that contributed to its success—namely dynamic partnerships, a thorough mixed methods research process, interlocking incentives, and a close relationship to market stakeholders. Finally, the report notes lessons learned and, based on these lessons, makes recommendations for both future iterations of the competition and pathways to strengthen the off-grid refrigeration market. Executive Summary 6 7 Structure Each section below provides an outline and analysis of the activities and process for that phase of the competition. The learning section aggregates the evaluation findings. Section 1 – Provides an overview of the competition, including the basis and intention for the competition, theory of change, the competition design and structure, timeline, and winners. Section 2 – Provides further details on the competition findings by first exploring impact through the framework of: • Innovation, or whether the competition incentivized an output that met the desired criteria, • Capabilities, or whether the competition improved the skills of competitors, and • Ecosystem, or whether the competition affected the broader ecosystem, market, or dialogue about off-grid refrigeration and SHS, and compiles the finalists’ product and professional developments. Section 3 – Provides competition lessons learned from the research, design, implementation, and evaluation phases, suggested actions that can be taken to improve future competitions in this sector, and other market-shaping activities. 7 The Global Lighting and Energy Access Partnership (“Global LEAP”) Awards were designed to transform the global market for off-grid energy products by recognizing and rewarding the most energy-efficient, highest quality off-grid appliances. In September 2016, Global LEAP launched the Off-Grid Refrigerator Competition. Off-grid refrigeration appliances were selected because refrigeration holds unique potential to unlock economic and social progress for the billions of un- and under-electrified people globally. This report uses Nesta’s Challenge Prize Centre’s theory of change and impact framework approach to evaluate and assess the impact of the competition. Though this work focuses on off-grid refrigeration devices, it is also about the organizational growth of the innovators who produced the refrigeration devices, and more broadly, about how this competition may have influenced the wider ecosystem for off￾grid appliances and solar home systems (SHS) in sub-Saharan Africa. The report evaluates the competition outcomes and process by assessing the impact of the competition at every stage— before, during, and after winners were selected. Evaluation activities were conducted on an ongoing basis during the competition period. The evaluation process sought to assess both whether the competition achieved the goals it sought to obtain as well as how the competition process may have contributed to those goals. The report is not without limitations. An impact assessment of the competition or the long-term performance of the off-grid refrigeration devices extends beyond this research’s remit. Furthermore, given competition delays which significantly impacted the timeline, this evaluation could not report some of its intended analyses, especially those pertaining to value for money. For those who would like to perform the analysis, this report outlines a methodology that can be used to calculate the competition’s value for money. This report is one of several other competition learning outputs produced by the partners; this includes a comprehensive learnings report by CLASP and another evaluation conducted by IMC’s Ideas2Impact (I2I) program, each complementing one another to provide a robust evaluation of this competition from different perspectives. Introduction 8 9 Why a competition? Refrigeration enables income-generating activities, prolongs the shelf life of fresh foods, reduces spoilage and waste, diversifies and enhances the nutritional value of diets, and reduces the time that households—particularly female members—spend shopping for, gathering, or preparing food. However, the market for off-grid refrigeration products is nascent and the prevalence of refrigerators in developing countries, particularly in rural areas, remains extremely low. Highly efficient, appropriately designed, and competitively priced refrigerators are essential to this market’s development and to its subsequent positive development impacts. Inducement competitions are a tried and tested method for stimulating innovation. They act as an incentive to achieve a specific result, which in this case is to transform the global market for off-grid energy. Rather than being a reward for past achievements, these competitions can also be used as a tool to encourage market, as well as to help create a positive social impact. This section explores the Global LEAP Off￾Grid Refrigerator Competition’s goals, design, structure, and the winners who emerged as a result. In Spring 2016, USAID (The United States Agency for International Development), through the Scaling Off￾Grid Energy: A Grand Challenge for Development (SOGE) and Power Africa, and the U.S. Department of Energy (DOE) partnered to sponsor the first round of the Global LEAP Awards Off-Grid Refrigerator Competition. 1 They were later joined by the Department for International Development (DFID/UK aid) via their Energy 4 Impact (E4I) program, designed to support research and development in climate technologies for developing countries. Together, implementing partners from each of the agencies, CLASP (with DOE and later USAID), DAI (with USAID), and E4I and IMC (with UK aid), supported the Global LEAP Awards to contribute to the design and launch of an open innovation prize competition. This competition focused on off-grid refrigeration technologies that both recognized and elevated the best existing technologies, and induced the creation of new technologies that surpassed the prevailing market standards in efficiency, value, and user appeal. 8 9 Section 1 Overview of the Competition 10 11 Why Uganda? As part of the Innovation Prize element of the 2016-2017 Global LEAP Off-Grid Refrigerator Competition, the partners supported a field testing component. While laboratory testing can effectively simulate varying temperatures, humidity, and customer wear-and-tear, it cannot measure a customer’s reaction to and valuation of the product. To select a field testing country, the partners looked among SOGE priority countries where they had a strong relationship with the USAID Mission, a favorable policy environment, and where the implementing partners had a significant footprint. Based on these criteria and market research, Uganda emerged as the best choice. In 2016-2017, the Ugandan market was characterized by a growing off-grid market and supportive policies. These are key factors for the emergence of the off-grid refrigeration market. In Uganda, approximately 18% of the rural population have electricity access . 2 Global LEAP Off-Grid Refrigerator Competition organizers hypothesized that off-grid shop owners would use refrigerators for vending purposes and somewhat limited storage of personal food items. 3 During initial scoping research conducted by USAID, it was found that while many people did know of off-grid refrigerators and solar home systems, interviewees from field research also identified various gaps in knowledge of this technology. To contribute to the competition design and evaluation baseline, in August 2016, USAID conducted field research in Uganda to understand gaps and determine the areas where impact is possible (see Table 1 on page 12). 12 13 FIELD RESEARCH INSIGHTS AREA OF IMPROVEMENT The USAID research synthesis noted that the market for off-grid refrigeration products is in its early beginnings, and the use of refrigerators is low. Market penetration In Kampala, fridges are quite large and heavy, and these were mainly used by small businesses, NGOs, or government institutions. They were not suitable for households. Product design Companies, such as SolarNow, produce 112L and 35L fridges, but they do not reflect the desired dimensions of most respondents of the CLASP survey: units between 50L and 100L. Product design Research shows that SHS solutions only reach 7% of the addressable off-grid market and are usually located in higher-density regions in Uganda. Additionally, most SHS providers are top international firms. This competition could help diversify the participants in the SHS market, potentially catalyzing domestic or regional innovations. Additionally, the competition may contribute to SHS reaching a wider landscape in rural Uganda. SHS market Off-grid refrigeration products range from $200 to $1000. USAID’s field research found a preference for cheaper options. This finding informed a prize award for “best value” which incorporated product costs, energy costs, and durability. Product costs While Uganda was the ultimate choice for field testing, the competition did not want to limit off-grid appropriate refrigeration devices’ success to Uganda. A broader goal was to nurture a nascent market for off-grid refrigeration solutions across sub-Saharan Africa and build demand for the SHS that could support them. This was illustrated by linking the competition to a results-based finance incentive scheme for East Africa, recognizing the strong SHS market in East Africa and the opportunity to incentivize SHS companies to sell finalist and winning refrigerators. Table 1 - USAID Research 13 14 15 The Theory of Change was the roadmap for the 2016-2017 Global LEAP Off-Grid Refrigerator Competition and details what the competition set out to achieve. Off-Grid Refrigerator Competition Theory Of Change 3b. Activities 3a. Impact Assumptions Research with experts and end users Launch and implementation There are few off-grid refrigeration products available on the market that are appropriate for developing country consumers The competition will signal to manufacturers/designers and innovators that the off-grid market has promising opportunities deserving of their skills and investment More efficient appliance including refrigerators will boost demand of solar home systems connections Engaging stakeholders Open call to action Launch and award event/campaign Scoping problem definition Recruit experts and judges in steps assessment Media and social media campaigns Define results statement Provide support for B2B partnerships Publicizing the prize award Design competition criteria and plan Laboratory & field testing Field testing Coordinating donor agency partnership Transform global market for off-grid energy products Innovation 2.Outcomes 1. Impact Goal GENERATE TECHNICAL AND MARKET INNOVATION to develop off-grid appropriate refrigeration for developing countries GENERATE SOLUTIONS FOR OFF-GRID CONNECTIVITY to give more people the benefits of a life with electrical goods MOBILIZE NEW TALENTS to generate new ideas for off-grid refrigerators GROW A DIVERSE MARKET FOR EFFICIENT OFF-GRID APPLIANCES to increase income, prosperity and quality of products BUILD B2B PARTNERSHIPS to accelerate distribution (partnership with distributors investors) BUILD VALUE OF SOLAR HOME SYSTEMS to increase demand and subscriptions BUILD CONSUMERS CAPACITY (Awareness) to increase the value they associate to solar systems (local business and final clients) RAISE MARKET AWARENESS on the existence of high quality off￾grid products Capabilities Ecosystem 14 Theory of Change 16 17 Competition Design and Structure Nominations 21 Sep 2016 by 10 Feb 2017 by 10 Feb 2017 20 Jan 2017 Nominations open Preliminary finalists selected by expert panel Nominators notified of preliminary selected Nominations due status Pre-Screening 05 06 07 08 09 10 11 12 by 24 Feb 2017 by 13 Mar 2017 by 10 Mar 2017 by 31 July 2017 by 31 Dec 2017 January 2018 by 31 July 2017 August 2017 Commercially-available preliminary finalist products available for random warehouse sampling Laboratory testing begins Field testing begins Competition winners & finalists and Energy Efficiency & Overall Value Innovation Prize winners named All samples received by test laboratory Laboratory testing ends & winners identified Field testing ends & winner identified Appropriate Design & User Experience Innovation Prize winner named Testing & Verification of Awards Finalists Awards Announcements & Promotions Timeline and Milestones After a rigorous research and design phase, the 2016-2017 Global LEAP Off-Grid Refrigerator Competition launched in September 2016. Between September 2016 and January 2017, nominations were received from innovators, product developers, manufacturers, and distributors. In early 2017, after evaluation and selection by an expert panel, the preliminary finalists were notified to send their products for laboratory testing. Testing took place September-December 2017. Expert judges reviewed the laboratory test results and identified Global LEAP finalists as well as the winners of the Energy Efficiency and Overall Value prizes. Prize winners were announced January 2018. Finalists with consistently successful laboratory test performance qualified for field testing, which took place at sites in Uganda during June-August 2018. In September 2018, expert judges reviewed field testing results and recommended the winner for the Appropriate Design and User Experience prize. The timeline to the right notes the original competition deadlines, however the deadlines and duration of activities shifted due to significant delays in preparations for both laboratory and field testing. These changes and their effects are detailed on pages 18-19. 18 19 August 2016 Research and Design Process USAID conducted a human-centered design-focused research and scoping exercise in Uganda. The goal of the research was to identify prospective customer habits and needs, perceptions around refrigeration, and what they might store in off-grid appropriate refrigerators were they to be available. September 2017 November 2018 August-November 2016 September 2016 November 2016 CLASP, with funding from I2I, conducted a laboratory test of existing off-grid refrigerators on the market to create a baseline for the performance of the competition’s products. The baseline testing allowed CLASP to create the eligibility criteria for the competition as well as build the laboratory test method. E4I conducted an in-depth study in off-grid refrigerator use cases in Uganda, this firstly provided context to how they would run the field test. Secondly, it provided a benchmark for the results from the field test. The off-grid refrigerator survey by E4I was used to create the field test method for the competition. Award nominations were open for four months, with nominees completing a nomination form detailing general applicant details, product information, product warehouse information, the declared product performance, and product market data. These details were used to determine whether the product met the competition eligibility requirements and whether it qualified for product shipping and testing subsidies. January 2017 February 2017 The nominations closed. An expert panel of judges (included technical, development, and off-grid industry and market experts, as well as representatives from donor agencies supporting the competition) were assembled in February 2017 to select the preliminary finalists on the basis of the nomination forms. The preliminary finalists had to ship product samples for laboratory testing. Commercially-ready competitors had to make a minimum of 50 products available for random warehouse sampling of two products. Late-stage prototypes had to furnish two products. October 2018 July 2018 April 2018 January 2018 The field testing results were released in October 2018 to finalists. E4I and CLASP conducted another formulaic analysis to identify the last innovation prize winners (see innovation prize winners on page 30 for further details). The products passed customs and E4I began coordinating installations of the products alongside the solar home systems with the 36 field testers across rural Uganda. Data was collected for six weeks to inform the field testing results. Initially field testing was meant to begin in April 2018, however, issues with shipping from the Netherlands to Uganda delayed the process. Upon arrival in the country, the products were further blocked from receipt by Uganda’s customs office for almost two months. This created further delays in field testing. The partners made the official announcement and awards event for the competition winners, finalists, and innovation prize award winners for Overall Value and Energy Efficiency at the Global Off-Grid Solar Forum and Expo in Hong Kong. December 2017 The laboratory test results were released in December 2017. An extensive results report was produced. As part of the calculation for competition winners and two of the innovation prizes, CLASP conducted a formulaic analysis to understand who the winners were (see competition winners on page 28 for further details). Due to significant delays in donor fund transfers to cover shipping costs and laboratory testing, laboratory testing was conducted on preliminary finalists’ products in September 2017 using the Global LEAP test method at an independent testing facility - Re/Gent in the Netherlands. Initial funding was supposed to come from DOE, however due to political constraints they were unable to fund the shipping costs, and due to a protracted funds transfer process, USAID was not able to cover the costs as intended either. DFID fronted the funds for shipping costs which USAID paid back at a later date. The competition closed with the announcement of the final innovation prize - Appropriate Design and User Experience at Unlocking Solar Capital in Kigali, Rwanda. 20 21 There were two key award categories to the Off-Grid Refrigerator Competition. Competition Winners - Based on laboratory testing and expert judging panel, with a determination of best in class. Innovation Prize Winners - Based, depending on the prize, on laboratory testing and an expert judging panel or field testing and an expert judging panel, which evaluated whether the refrigeration devices surpassed prevailing market performance. The competition recognized the most energy efficient and highest quality products in seven categories. The best performers in each category were declared the ‘Winner’ for that category. The competition nominations were only for commercially-available products. Table 2 - Competition Categories SMALL MEDIUM LARGE EXTRA LARGE REFRIGERATOR 5L-50L 51L-100L +101L REFRIGERATOR￾FREEZER COMBO 5L-100L 101L-150L 151L-200L +201L For the Innovation Prizes, commercially-available or late￾stage prototype products were evaluated for their energy performance, quality, design, and reliability according to rigorous international best practice. There were three $200,000 cash prizes for products that demonstrated market￾leading innovation in: • Overall value • Energy efficiency • Appropriate design and user experience There was a clear distinction between competition winners and Innovation Prize winners, as the former is recognized as the best overall in each category, while the latter on a number of characteristics that meet global standards, lead in innovation, and exceed existing market performance. Winners were decided by extensive and rigorous laboratory and field testing in which expert judges made the final judgment based on the results from testing. Competition Details 21 22 23 Laboratory Testing “Laboratory testing was a good idea. The criteria gave me guidance about what the market wants – it was a benefit for me to see what they valued.” —David Bergeron, SunDanzer - Finalist Laboratory testing sought to validate how the products performed vis-a-vis efficiency, cooling, durability, value for money, and truth in advertising. Testing took place at Re/Gent, an independent laboratory in the Netherlands. All preliminary finalists were expected to send product samples for testing. The timeline of laboratory testing, however, was significantly delayed (see timeline and milestones on page 16 for further details). Examined in more detail in Table 3, the Global LEAP Off-Grid Refrigerator Competition laboratory test method4 aimed to evaluate a combination of quantitative and qualitative performance factors: Quantitative: • Power consumption • Cooling and/or freezing capacity • Performance in under- and over-voltage conditions • Performance after exposure to high heat and humidity • Total cost of ownership Qualitative: • Design and usability • Innovative features/functions • Warranty (for commercially-available products only) • Truth in advertising5 TEST DESCRIPTION AMBIENT TESTING CONDITIONS TARGET TEMPERATURE Steady-State Operation Measure a product’s energy consumption (in kWh per day) at specified ambient temperature & humidity conditions • Low ambient (+10ºC / RH <75%) • Medium Ambient (+16ºC / RH <75%) • High Ambient (+32ºC / RH <75%) • Harsh Ambient (+43ºC / RH <75%) Stabilization at an average temperature of +4ºC Pull-down Measure the cooling time (in hours) of a product’s fresh food compartment from the ambient condition to a specified target temperature • High Ambient (+32ºC / RH <75%) Stabilization < +4ºC Autonomy Measure the time (in hours) which a product can maintain a specified target temperature after a product is disconnected from the power supply • High Ambient (+32ºC / RH <75%) Stabilization between +4ºC & 12ºC Voltage Fluctuation Measure the product performance at high (+20% of rated voltage) and low voltage (-10% of rated voltage) input • High Ambient (+32ºC / RH <75%) Stabilization < +4ºC Freezing Capacity Measure the time (in hours) a product freezes a specific amount of filling materials. Only applicable to refrigerator-freezer combination units • High Ambient (+32ºC / RH <75%) Stabilization < -15ºC Table 3 - Test Descriptions 24 25 Field testing occurred between June-August of 2018 to: • Establish how well products perform in a ‘real life’ situation. • Gather information from end users about their experiences with the products. • Inform the Appropriate Design and User Experience award. All products that satisfactorily completed the competition’s laboratory testing process were eligible for field testing. Two units per product were placed at field test sites chosen on the basis of four factors: • Business profile: Each small retail shop was, to the extent possible, of similar size with similar clientele and product offerings. • Climate and location: Test sites were in similar climatic zones and were reasonably close together. • Familiarity with solar: Shop owners had previous experience using SHS either at the test site or in their homes (e.g. use of a system for lighting and mobile phone charging). • New refrigerator users: Shop owners did not already own or use a refrigeration device to assess the product’s commercial benefits (e.g. new revenue generation) and whether products were perceived as affordable/viable by a target market segment. E4I identified 36 field testers across Uganda, including in Masaka, Mbarrara, and Mbale. Following the formal test period, functioning, commercially available products and the solar home system were left with the shop owners (test subjects) as an incentive to participate in the field test. 6 Given delays in shipping, data collection from the field was reduced to six weeks from twelve weeks. Concerns were raised as well as potential mitigation strategies to ensure the new timeline did not change (see Table 4). CONCERNS MITIGATION Participant/stakeholder queries over the contracted timelines. Early communication and amendment of field test protocols. Participants benefit from longer-term data collection, particularly in understanding issues of design, usability, and potentially durability. Data can continue to be collected for an additional 6 weeks and will be shared with participants following the award of the innovation prize. Whether a judgment could be made based on data from only half the collection period initially proposed. Technical performance in the field was not expected to vary much after the initial settling period. User preference may change over time, though it is difficult to know whether six weeks is significant. This portion of the judging criteria weighting could be reduced. 25 Field Testing “All us are interested in these solutions being applied to the real-world setting...feedback from users is critical to make solutions accessible.” —Makena, E4I - Partner Table 4 - Field Testing Concerns and Mitigations 26 27 RESULTS WHAT WE LEARNED Research 2015 Design 2015-2016 Launch 21 Sep 2016 - 20 Jan 2017 Pre-Screening Decisions 10 Feb 2017 Human centred design (HCD) research; Benchmark and baseline studies in Uganda. Design competition, two sets of consultations, design of laboratory and field-testing methods, competition documents. The launch was a digital push with strong communications activity via the Global LEAP brand. A panel of expert judges pre-screened nominations and selected ‘Preliminary Finalists’. Uganda was identified as the most viable country for a competition given how nascent the off-grid refrigeration market is. Having multiple partners allows extensive scoping research – HCD, benchmarking and baseline analysis on a variety of different aspects for the competition. RESULTS RESULTS RESULTS WHAT WE LEARNED WHAT WE LEARNED WHAT WE LEARNED 57 nominations in total from 28 organizations. Diverse group of nominations from 14 countries across the world. 24 nominations from a total of 15 organizations made it through the pre-screening. The communications push during the call to entry from implementers (CLASP and DAI) ensured a high amount of interest. Global LEAP was a key factor in attracting nominations. This process of prescreening nominations allowed for a filtering of solutions that did not adequately meet the set criteria of the competition. Developed implementation and communications plan, assessment criteria and competition evaluation framework. The design was done in continuous consultation with key USAID and UK aid stakeholders and this helped a quick roll out of activities and sign off. RESULTS RESULTS RESULTS RESULTS WHAT WE LEARNED WHAT WE LEARNED WHAT WE LEARNED WHAT WE LEARNED Product Sampling 24 Feb - 10 Mar 2017 Laboratory Testing Sep 2017 Field Testing Jun-Aug 2018 Award Jan 2018/Nov 2018 Preliminary Finalist products in both the Competition and Innovation Prize will be evaluated on qualitative and quantitative factors: Quant- power consumption, cooling and/or freezing capacity, performance in under and overvoltage conditions, performance after exposure to high heat and humidity, total cost of ownership. Qual - design, durability and usability, innovative feature/functions, warranty (for commercially-available products only), truth in advertising. This testing method was developed by Global LEAP (CLASP) and references test methods - IEC 62552 and WHO/PQS/E003/RF05-VP.4 All products that satisfactorily completed the laboratory testing process were eligible for field testing. Technical data – internal compartment temperature, ambient temperature and relative humidity, instantaneous power consumption, accumulated energy consumption per day, daily power available to product from SHS; and user experience data – ease of use, perceived technical performance and value, challenges were collected. Due to the delayed timeline of the competition. The competition partners decided to announce two of the Innovation Prize Award winners in Hong Kong at Global Off-Grid Solar Forum and Expo on January 22nd￾24th 2018 as laboratory testing had been completed and the decision could be made. The final Innovation Prize was announced at Unlocking Solar Capital in Kigali on November 7-8th 2018. Preliminary Finalists who have commercially-available products must make a minimum of 50 nominated products available for random warehouse testing. The administrator’s designated sampling agent will randomly select two nominated products for testing. There were 11 finalists selected overall with 17 nominations. 10 finalists participated with 15 of the nominations fully tested. 24 nominations from a total of 15 organizations provided product samples. 3 Innovation Prize Awards were announced with 7 finalists attending the first event at the start of 2018, and 0 attending the final event. Laboratory testing is expensive and prior to the Global LEAP Off-Grid Refrigerator competition, no uniform testing protocol existed. Finalists appreciated that by participating in the competition, they not only had this expensive process paid for, they also got standardized and vital feedback on their products’ performance. Field testing provided finalists with critical technical information regarding the performance of their product in the field, as well as user insights and experiences of using their product. There were significant delays in the products clearing Ugandan customs, as well as issues in installations and maintenance of products whilst in the field. USAID supporting finalists with product shipment costs of nominated products, saved finalists having to spend out-of pocket. Logistics of shipping need to be accounted for given the delays in processing products. For the first event, finalists found the event useful to network with other innovators and potential partners, distributors and investors. With the final event, though no finalists attended, Global LEAP promoted the winners and finalists throughout the event. Competition Timeline, Actions, Results and Learnings 28 29 The Global LEAP Awards identified winners for the best overall product for each size and form factor-based category. The competition’s finalists were awarded up to 110 points based on the following: • Laboratory testing: The product’s power consumption, cooling performance, and certain design and quality factors. • Expert assessment: An expert panel evaluated the quality, design, and usability factors based on data provided in nominations and the laboratory testing. Competition winners were identified by CLASP—awarding up to 80 points on total cost of ownership (which is based on laboratory testing results of power consumption and standardized Global LEAP Awards assumptions about retail price, $/kWh, product lifetime, and usage) and performance (which is based upon the products’ (i) time to cool a standardized load to a steady state and (ii) time holding a band of acceptable temperature once power has been disconnected). The formula for Total Cost of Ownership: (Product energy consumption x standardized lifetime usage assumptions [kWh] x $/kWh) + (wholesale price x standardized retail price modifier)) x Time to Cool to Steady State Modifier [where less time is better] x Time Holding Acceptable Temperature Modifier [where more time is better] = $XXX. In this assessment, Preliminary Finalists with a lower total dollar figure received a higher score. The remaining 20 points were based on the expert assessment. A finalist could have earned two points of ‘extra credit’ for each year of its warranty beyond the required two years, up to eight points in total. Finalists using exclusively natural refrigerants received three points of ‘extra credit.’ Competition Winners Table 5 - Competition Winners Specifications SunDanzer DCR50 PALFRIDGE LC86 PALFRIDGE LC221 METROPOLITAN MEFR48C1 BASIL DC170L Description Small refrigerator Medium refrigerator Large refrigerator Small refrigerator￾freezer combination Large refrigerator￾freezer combination Total Capacity 50L 86L 192L 46L 170L Rated Power Consumption 45W 49W 49W 50W 30W Daily Energy Consumption at 32°C 0.118 kWh/day 0.154 KWh/day 0.281 KWh/day 0.543 KWh/day 1.02 KWh/day Pull-Down Time 0.55 Hours 1.13 hours 0.76 hours 0.84 hours 0.9 hours Autonomy 1.35 Hours 1.22 hours 0.76 hours 0.63 hours 0.66 hours Freezer Capacity n/a n/a n/a 4L 11L Freezing Performance (hours to reach -6°C) n/a n/a n/a Not achieved 6.1 The Competition Winners are listed below: • SunDanzer DCR50 – Small refrigerator • Palfridge (The Fridge Factory) LC86 – Medium refrigerator • Palfridge (The Fridge Factory) LC221 – Large refrigerator • Metropolitan MEFR48C1 – Small refrigerator-freezer combination • Basil DC170L – Large refrigerator-freezer combination 30 31 “Being a part of the LEAP Global Off-Grid Refrigerator Competition was an eye-opening experience...it introduced us to so many players in a market we didn’t really know about” —David Bergeron, Founder and President - SunDanzer Three Innovation Prizes were awarded to products that clearly demonstrated market-leading innovation, as identified by the expert panel and by the baseline studies, as well as data and observations made during the laboratory testing and field testing. Innovation Prizes were awarded for each of the following product characteristics: • Energy Efficiency based on laboratory testing An expert panel assessed the data and observations from the laboratory testing. The SunDanzer DCR165 (large refrigerator) won the Energy Efficiency Innovation Prize. • Overall Value based on laboratory testing While a highly energy efficient refrigerator can have significant upfront purchase costs, a durable, energy efficient refrigerator will have lower energy and maintenance costs over time. This prize category sought to balance efficiency, durability, and purchase price to determine the best customer value for money. For the purposes of this competition, ‘overall value’ referred to the total score as determined by the laboratory testing and expert assessment process outlined herein. To allow for comparison of commercially-available products and late-stage prototypes, nominators of prototypes were required to provide an estimated wholesale cost along with a detailed justification of that cost. The winner of the innovation prize for overall value, SunDanzer DCR50 (small refrigerator), was the product with the highest total score across all competition categories. • Appropriate Design and User Experience based on field testing For the Appropriate Design and User Experience Innovation Prize, all finalist products were scored out of a possible 60 points. Up to 20 points were allocated through a quantitative assessment based on each finalist’s total cost of product, a formula-based calculation that included standardized assumptions for retail price, the cost of energy system installed to power the product alongside installation costs and any additional hardware. Up to 40 points were allocated based on the expert judges’ evaluation. The scores provided for each finalist was averaged together with the other judges scores then added to the total cost of product score. Palfridge trading as Fridge Factory LC86 (medium refrigerator) achieved the highest overall score and was named the Appropriate Design and User Experience Innovation Prize Winner. SunDanzer DCR50 SunDanzer DCR165 Palfridge LC86 “Best Value” Innovation Prize Winner “Energy Efficiency” Innovation Prize Winner “Appropriate Design and User Experience” Innovation Prize Winner Total Capacity (L) 50 163 86 Rated Power Consumption (W) 45 150 49 Daily Energy Consumption at 32°C (kWh/day) 0.118 0.191 0.154 Pull-Down Time (Hours) 0.55 0.64 1.13 Autonomy (Hours) 1.35 1.83 1.22 Innovation Prize Winners Table 6 – Innovation Prize Winners’ Specifications 32 33 Section 2 Findings This section outlines the approach to impact and details the findings for the 2016-2017 Global Off￾Grid Refrigerator Competition. The section begins with the findings from the entry nominations, which contextualizes the types of innovators and solutions that participated, then the findings are analyzed in relation to the impact categories, namely: Innovation which seeks to assess whether the competition attracted new innovators and inspired the development of new solutions in response to a defined problem or need. Capabilities which seeks to assess whether the competition fostered the development of new partnerships, new innovator capabilities and mobilized new and existing talents. Ecosystem which seeks to assess whether the competition has contributed to developing new markets, attracting new investors and raising awareness of off-grid refrigeration and solar home systems. WHAT FROM TOC SPECIFIC ITEM SPECIFIC INDICATOR WHEN WHO HOW QUESTION Assumptions What assumptions? What indicator(s)? When to collect the indicator(s) during competition? What stakeholders will inform on the indicator(s)? What evaluation technique? What question? Activities What activities? Outcomes What outcomes? These three impact headings organize the competition findings. This impact approach (and theory of change – see page 14) are the building blocks for the detailed impact framework (see full Global LEAP Off-Grid Refrigerator Competition impact framework in Appendix A). Impact Approach The impact framework supports the impact assessment through three steps. Estimate impact: This action includes understanding what impact the competition will generate and identifying key indicators and benchmarks to set the scene for future evaluation. This includes designing a theory of change, identifying indicators, collecting benchmarks – the evidence from the research which supported prize assumptions – and designing nomination form evaluation questions to collect baseline data from participants. Manage initial impact: Across the implementation of the competition, results are constantly monitored via the nomination form data, analyzing laboratory test results, activity surveys, and 1-2-1 sessions with partners and finalists. The competition collected all relevant baselines. Prior to field testing, a mid-term report was produced to assess the initial impact of the prize. Determine impact: After completion of field testing, a series of surveys and interviews were conducted with judges and finalists to complete the collection of findings and assess them against benchmarks and baselines collected during the lifetime of the prize. This enabled the team to determine the final impact under the terms of this assessment period. The competition’s impact will extend several years into the future and long term impact should be measured periodically. 32 Table 7: Impact Approach 34 35 To better understand the type of evidence the evaluation was looking to uncover throughout the competition, the indicator grid below provides an extensive list of the indicators used to benchmark, baseline, and then track key performance. OUTCOMES BENCHMARK BASELINES KPIS OUTCOME NARRATIVE Generate technical and market innovation to develop off-grid appropriate refrigeration for developing countries Current solutions (qualitative) Current solutions (quantitative) Performance in laboratory and field tests Judges feedback To measure overall technical advancement for off￾grid refrigerations solutions - meeting the needs of the Ugandan context and the eligibility criteria Generate off-grid solutions that improved user satisfaction, remuneration, and well-being Usability (access and cost) Field testing status cohort Field testing results and nominee feedback To measure the effectiveness of the solutions generated with a focus on end-users’ reaction to new products (focus on effectiveness) Mobilize talents to generate new ideas for off￾grid refrigerator # of nominations new to off-grid refrigeration % gender presence in sector # of nominations that passed screening # of nominations that never applied for donor funding % gender presence in nominations Motivations for participation % finalist new to donor funding % finalist transferred knowledge % gender presence in finalists To measure the extent to which the competition and prize awards mobilized talents to generate solutions Build business-to-business (B2B) partnerships to accelerate distribution (partnrship with distributors investors) Market research information. Type of support provided Level of finalists’ satisfaction with new partnerships To measure the role of the prize process in generating new partnerships. In particular, looking at B2B .partnerships that are generated by the competition Build consumers capacity (to increase the awareness and value associted with solar home systems in local businesses and final clients/ consumers). % solar power users in Uganda aware of off-grid refrigeration with SHS. % solar companies offering refrigerators as part of package Assumptions of cohort before field testing User experience from field testing Consumers feedback from field testing To measure the level to which both local business and consumers react to the innovative solutions (focuses on judgement and potential behavioral change) Grow a diverse market for efficient off-grid appliances to increase income-generation potential and the quality of products Current demand – in developing countries Current demand - in Uganda Nominated product market data Field testers’ records of sales and expenditures Market projections for competition finalists % of improvement in business sales vs expenditure To measure changes in the market in terms of market projections from the supply side and income improvement from the demand side Build value of solar home systems to increase demand for new systems and to increase SHS values USAID target for growing solar system value Proposed cost in nomination form Results from cost perception analysis. Attitudes towards using SHS To measure the extent to which bringing innovation in this sector will increase the value of SHS and how Raise market awareness of high quality off-grid products Current market perception Types of communications action Presence at events # of high-quality off-grid refrigeration products in market. Attitudes towards off-grid products To measure change in perception form the key players in the solar home system market for developing countries As the theory of change on page 14-15 details, the competition identified eight outcomes related to transforming the global market for off-grid energy products: 1. Generate technical and market innovation to develop off￾grid appropriate refrigeration for developing countries. 2. Generate off-grid solutions that improved user satisfaction, remuneration, and well-being. 3. Mobilize new talents to generate innovative off-grid refrigerators. 4. Build business-to-business (B2B) partnerships to accelerate distribution (partnerships with distributors and investors). 5. Build consumers’ capacity (to increase the awareness and value associated with solar home systems in local businesses and final clients/consumers). 6. Grow a diverse market for efficient off-grid appliances to increase users’ income-generation potential and the quality of products. 7. Build value of solar home systems to increase demand for new systems and to increase SHS values. 8. Raise market awareness of high quality off-grid products. The evaluation findings are comprised of work conducted by all partners and include pre-launch activities such as the prize scoping research, USAID’s human-centered design field research, CLASP’s stakeholder survey and request for information, CLASP’s off-grid refrigerator baseline testing data analysis, and E4I’s benchmarking survey of on-grid refrigerators. The findings also include the laboratory testing method and results conducted by CLASP and the field-testing method and results conducted by E4I and CLASP. Additionally, Nesta’s Challenge Prize Centre complemented these materials with desk research, nomination entry form data analysis, benchmark and baseline setting, laboratory and field testing results, surveys with finalists and judges, field research conducted alongside the field test team, interviews with finalists and partners, and a partner focus group. It should be noted, however, that there were difficulties with conducting some segments of the evaluation, namely due to low response rates on surveys and interviews as well as some challenges reaching finalists and judges. The analysis will trace the stages of the competition and the different ways that impact manifested in innovation, innovator capabilities, and influences on the ecosystem. The first stage of the competition was to analyze the entry nomination process. Table 8: Indicator Grid 36 37 Competition General Findings The Global LEAP Off-Grid Refrigerator Competition was able to engage with innovators from around the world to incentivize a diverse range of solutions. The competition received a total of 57 nominations (proposed off-grid refrigeration solutions) from 28 organizations. This exceeded expectations by 285% as the initial target was 20 nominations. The nominations originated from 14 countries – with the majority coming from China, the United Kingdom, and the United States. Chart 1: Applicants by Country The competition was open to commercially viable products and late-stage prototypes. The former were eligible for both the competition and innovation prize, whilst the latter were only eligible for the innovation prize. The vast majority of submissions were commercially viable products from enterprises ranging from small-scale organizations with 0-10 employees to larger firms with over 200 employees. This range was anticipated given that off-grid refrigeration is still a niche (medical cold chain, outdoor recreation refrigerators) and/or a nascent space (domestic and productive use). While rare in the manufacturing industry, the competition organizers valued gender diversity. In the nomination forms, they inquired what percentage of women were represented in the team submitting the nomination. Ten organizations had 0-20% of women working in their teams, 16 organizations had between 21-40%, and two had between 41-60%. Unfortunately, there are no existing figures relating to gender diversity in the off-grid refrigeration space and they remain spotty in the off-grid sector more generally. When asked how competitors learned about the competition, the majority of nominations noted that they learned about the prize from Global LEAP or their partner networks. The Global LEAP brand is recognized across the off-grid sector for being the leading authority on off-grid appliances, and their brand recognition and partner networks inspired nominations. The technical breakdown of proposed solutions from the nominations show that 17 nominations were battery￾based and 16 were solar-direct drive. Solar-direct drive refrigeration devices can be more reliable refrigerators, but the risk is that they build demand for refrigerators rather than demand for SHS. By category, 38 nominations were for a refrigerator and 19 were for the refrigerator-freezer combination unit. The refrigerator was the focal point. The competition did not research perceptions of freezer units and their value. 16 14 12 10 8 6 4 2 0 Australia India France Nigeria Bangladesh Japan Germany South Africa China Kenya Hong Kong Swaziland United Kingdom United States ALL COMMERCIALLY VIABLE PRODUCTS ALL PROTOTYPES Total nominations 48 9 Total organizations 26 8 Countries 14 7 Most common organizations 42 commercial enterprises 7 commercial enterprises Types of solutions 32 refrigerator 16 refrigerator-freezer combination 6 refrigerator 3 refrigerator-freezer combination Prior applicant for or recipient of donor funding 3 applied for USAID/UK aid 1 applied for USAID/UK aid Table 8 - Comparing Commercially Viable vs. Prototype Refrigerators 38 39 Chart 2: Competition Outcomes Achievement Strongly Agree Agree Neither Agree or Disagree Disagree Strongly Disagree Generate technical and market innovation to develop off￾grid appropiate refrigeration for developing countries Generate solutions for off-grid connectivity (to give more people the benefits of a life with electrical goods) Mobilize new talents to generate new innovations for off￾grid refrigerators Build consumers awareness of the value of off-grid solar home systems Grow a diverse market for eddicient off-grid appliances Build the value of solar home systems Raise market awareness 60% 40% 20% 0% 39 40 41 Innovation The innovation category assesses the extent to which the Global LEAP Off-Grid Refrigerator Competition had an impact on generating off-grid solutions that improved user satisfaction, remuneration, and well-being through technical and market innovation. Put another way, it assesses whether the competition induced the innovations it sought. “We have never produced a commercially marketable off-grid refrigerator. The competition acted as an impetus to try and create some kind of solution.” —Nessar, MAKS PowerTech Ltd - Finalist Outcome 1: Generate technical and market innovation to develop off-grid appropriate refrigeration for developing countries. The competition succeeded in incentivizing meaningful, if not wholesale, technical innovation. Depending on the test, 12-13 of the 24 tested off-grid refrigeration devices outperformed the market-available off-grid refrigerators in four major tests (steady-state operation [10°C, 32°C, 43°C] and pull-down time). In three other tests, the market available off-grid refrigerators outperformed the competing refrigerators, and in one test, they tied. All judges agreed that the competition helped generate technical and market innovation to develop off-grid appropriate refrigerators and considered the solutions produced via this competition to be innovative. Nonetheless, one of the judges acknowledged “there were certainly a few products that caused me some concerns in terms of quality or performance. However, the majority of products surpassed my expectations of efficiency (and, as a result, cost), while still delivering the required performance.” 82% of competition stakeholders felt this outcome was met during the competition, as all finalists demonstrated technical advancements in off-grid refrigeration. Indeed, a number of the finalists and winners noted the eligibility requirements and criteria guided them in what type of solution they should produce. To understand whether the products have achieved technical innovation, the nominated products were analyzed against those already in the market. The average laboratory test results of the nominated solutions and refrigerators from the baseline study shows AVERAGE LABORATORY TESTING RESULTS7 TESTS*** SOLUTIONS BASELINE # OF SOLUTIONS THAT EXCEEDED BASELINE Pull-Down Time 1.05 Hours 2.28 Hours* 7 Steady-State Operation (10°C) 13 solutions satisfactory 0.255 kWh p/day* 12 Steady-State Operation (32°C) 0.38 kWh p/day 0.902 kWh p/day* 12 Steady-State Operation (43°C) 12 solutions satisfactory 1.505 kWh/day* 12 Steady-State Operation (16°C) 0.13 kWh p/day 0.323 kWh p/day* 7 Autonomy Time 2.63 Hours 1 Hour* 7 Freezing Capacity (2kg to -18°C) 15 Hours 11.2 Hours** 0 Freezing Capacity (0.5kg to -6°C) 12 Hours 12 Hours** 0 Undervoltage 14 solutions satisfactory N/A N/A Overvoltage 17 solutions satisfactory N/A N/A Freezing Capacity (2kg to -12°C) 16.7 Hours N/A N/A Freezing Capacity (0.5kg to -13°C) 10.1 Hours N/A N/A Table 9 - Average Laboratory Testing Results (Solutions vs. Baselines) Expected Innovation Outcomes: 1. Generate technical and market innovation to develop off-grid appropriate refrigeration for developing countries. 2. Generate off-grid solutions that improved user satisfaction, remuneration, and well-being. * Based on the baseline testing data provided by CLASP and internal calculations. ** 23 units were tested, however only nine units at -6C and four units at -18C were able to reach specified target temperatures. The other products failed to operate due to malfunction or stopped operating after initial tests. These baselines do not account for weighting the unit while testing freezing capacity. *** See Table 3 on page 23 for description of tests. 42 43 71-76% of products outperformed the baseline testing in four tests: steady-state operation (10°C, 32°C, 43°C) and pull-down time. 8 The steady-state operation test means that those products are consuming far less energy on average, making them highly energy efficient in different ambient temperatures and humidity conditions (from cold to neutral to hot environments). In particular, the 32°C test is the common condition to test for daily energy consumption, and, on average, it outperformed the baselines by 0.522 kWh p/ day. The pull-down test time of 1.05 hours confirms that these products are much more faster and efficient at cooling their contents. Individually, many of these solutions produced better results than those on the market through various innovative technical specifications. For example, the SunDanzer DCR165 unit used energy most efficiently as determined by daily energy consumption and the time required to achieve and maintain a target temperature, thereby winning the Energy Efficiency Innovation Prize (see page 31). The majority of nominated refrigerators were able to outperform the baseline tests, however five nominated products - Phocos (FR170, FR240), SunDanzer (DCR165), SimuSolar (DCR165E) and Palfridge (LC86) outperformed the five key tests for refrigerator units (steady-state, autonomy and pull-down). These products went beyond the baselines to produce high-quality, energy efficient, cooling efficient and autonomous solutions. Though the competition spurred innovation in creating efficient off-grid solutions, results from the laboratory testing indicate that these solutions on average aren’t as low-cost as off-grid refrigerators currently on the market (CLASP baseline testing) with the average wholesale price per solution being $610.30, the average total retail price is $827.11 and the average total cost of production for these products is $2035.48. These prices and costs are still quite high, and their uptake is dependent upon meeting customer price points as discussed in Outcomes 5, 6, and 7. However, considering that this is an early instance of innovation in off-grid refrigeration technologies, innovators developed new solutions and wouldn’t necessarily have had the market impetus to further hone them if not for this competition. Field testing provided more practical information on the products’ performance in a real-world context. Normal breakdowns and poor weather and the ways in which the products affect performance inform customer perceptions more than detailed laboratory test results. Of the 36 refrigerators 9 tested, 27 (75%) worked well for the entire duration of field testing, 5 refrigerators didn’t work from the start, and as field testing progressed, 4 products suffered from technical issues—meter breakdowns, failure of cooling components, gas refrigerant leakages as well as one solution being placed on the grid. When breakdowns occurred, because these refrigerators were tested in rural areas, technicians were not able to immediately repair them.10 44 45 Table 10 - Field Test Performance Measures Category Average Daily Energy Consumption (kWh p/day) Daily Energy Consumption (kWh p/day) Declared Daily Energy Consumption (kWh p/day) External Temperature (C) Humidity (%) Small Refrigerator 0.31 0.53 0.17 26C 66 Small Refrigerator-freezer 0.65 0.57 0.35 28.5C 68.5 Medium Refrigerator 0.77 0.65 0.75 27.5C 60 Large Refrigerator 0.55 0.86 0.28 24C 49 Large Refrigerator-freezer 1.65 0.53 0.64 27C 62 Table 10 breaks down the product field testing performance by category type, daily energy consumption (average, daily and declared), and conditions tested in. The average daily energy consumption for all products was 0.69 kWh p/day to function in an average environment of 27°C with 63% humidity—the daily energy consumption is still relatively high to operate an off-grid refrigerator in the field, and represents an increase of 0.31 kWh p/day in comparison to the average daily energy consumption from laboratory testing. Field testing was invaluable because it revealed variations in device performance and energy consumption based on geographic conditions, refrigerator contents, and everyday use. Notably, three solutions (Basil DC170L, Palfridge LC86 and LC221) had a lower daily energy consumption in the field than what they originally stated, working better than they initially declared and performing better than advertised. It’s also important to recognize that the laboratory and field test methods themselves were innovative. Until the Global LEAP Awards Off-Grid Refrigerator Competition, there were no standardized or accepted means to test off-grid refrigerators for home and productive use. The development of these protocols allowed for comprehensive analyses of the solutions produced in this competition. Outcome 2: Generate off-grid solutions that improved user satisfaction, remuneration, and well-being. The field tests demonstrated that 75% of the competing refrigeration devices successfully functioned in off-grid environments. While 96% of field testers were at the very least happy with the off-grid refrigerators, and 100% found the products easy to use, many users wanted more cost-effective devices that had larger internal shelf-space to accommodate chilling more goods. CLASP, USAID, and E4I’s HCD research, benchmark, and baseline analyses allowed for a better, quantitative and qualitative understanding of off-grid refrigerators in a localized context. This research suggested different use cases for refrigerators, priority design elements and preferences vis-a￾vis volume, size, and average power consumption. To create products that were simultaneously competitive and valued by customers, innovators incorporated innovative features into their refrigeration products. All finalists identified a variety of innovations that improved their product performance such as thicker insulation, highly efficient AC/ DC compressors, and sensors to control temperature. The nominees also utilized innovative features to reduce product costs such as sensor-less controls and using low cost, but high-quality, materials such as metal casings and house parts. Lastly, some finalists sought to improve usability and accessibility by using different door models, not including locks, ensuring product height and width allows people with low mobility to use the product, and incorporating pay-as-you￾go electronics into the systems’ solar module. The field test involved 36 field representatives each testing a nominee’s product. The vast majority (over 90%) had never owned a refrigerator before, and after the testing period, 96% of field testers were either happy or very happy with the products, while 100% felt the products were easy to use and easy to open and close. Speaking to product design, 93% of field testers were either mostly or very satisfied, popularly noting that the “fridge cools contents very well and fast”, while others acknowledged “the compartment in the fridges provides many ways of using it” and “it’s spacious and can stock many items.” Field testers also offered some critiques of the products, some noted product-specific issues with 46 47 temperature knobs, freezing temperature, capacity, and door design. The most common criticism was size; the majority of field testers noted customer demand for chilled goods exceeded their refrigeration devices’ capacity. Field testers wanted to buy more drinks to place in their fridge and found small and medium refrigerators’ capacity too limited (even field testers with larger products struggled to meet increasing sales). The finalists valued receiving field testing results. They felt the independent field tests built credibility for both the product performance and the respective organizations. Finalists felt that product field performance on details like the daily energy consumption and user feedback was critical to shaping the future performance or product design. “We want to understand the valuable comments from the end user. It will give hints to us for future designs or modifications.” —Lotis, Metropolitan - Finalist Specifically knowing what design features work and didn’t work informed finalists’ product development and allowed them to consider different contexts and use cases for their off-grid refrigerators. Finalists did state that the field test results report (see Appendix B for sample) provided limited information on both technical performance and user experience. Of the technical information, only a graph and daily energy consumption were provided with an additional file of the monitoring data. Finalists noted they would have appreciated additional tests, monitoring, or observations on autonomy or pull-down time. Finalists also noted that though understanding how their products were utilized was useful, the full user testing results were difficult to understand without some additional context such as geography or the environment where these refrigerators were placed, the way in which the product was used, what was stored in the refrigerators and the quantity of contents. While the technical information provided via the monitoring data has contributed to understanding how field testers used the refrigerators, finalists noted that it would have helped to have this information from the outset. Finally, since the majority of field testers had never owned or used a refrigerator before, it’s reasonable to presume their feedback is informed by a positive bias towards the products. The refrigerators offered many field testers an opportunity to use a new technology which could also benefit their business—it’s hard not to feel warmly under those circumstances. Walyetoi Fazali from Mbale found the product had limited space, but faced with growing demand, he bought baskets and filled them with cold water (from the fridge), alternating between cooling drinks in the refrigerator and basket to continuously provide chilled drinks for customers. 48 49 MOTIVATIONS # OF RESPONSES The opportunity to win the competition 7 The opportunity to bring new and innovative products to market 6 The opportunity to win one of the innovation cash prizes 6 The potential to scale an existing product in a new global market 6 Learning from results of laboratory and/or field testing 6 The possibility of meeting new investors 5 The potential to a bring a product to the African market 5 The prestige and credibility of the Global LEAP brand 5 Applying the organization’s knowledge and skills for new purposes 3 The potential to bring a product to the Bangladeshi market 3 The opportunity to learn from the experience of individuals and organizations involved in the competition 3 Table 11 - Motivations for Finalists Capabilities The capabilities section assesses the extent to which the Global LEAP Off-Grid Refrigerator Competition has brought new talent into the off-grid refrigeration sector and/or improved the skills and networks of existing innovators. “Global LEAP gives you credibility as a supplier of solutions for the off-grid sector, and provides exposure to a wider market.” —Joseph Fernandez, Solageo - Finalist Outcome 3: Mobilize talents to generate new ideas for off-grid refrigeration. The competition marginally increased the number of new organizations working in off-grid refrigeration. Of the 28 organizations that nominated products, 4 nominees had never worked in the off-grid refrigeration market before the competition. Expected Capabilities Outcomes: 3. Mobilize talents to generate innovations for off-grid refrigeration. 4. Build B2B partnerships with distributors and investors to accelerate distribution. 5. Build consumers’ capacity (awareness to increase the value associated with solar systems—local businesses and final clients). The competition attracted four new organizations to the off-grid refrigeration space. That said, the off-grid applicant sector is a niche market, and the majority of finalists (71.4%) had already been working in the sector for 4+ years as seen in Chart 3. Among the competition’s 57 nominations, 54% of applicants passed the pre-screening to become finalists. Most of these finalists listed “winning” as their motivation to enter the competition. Other motivations include an eagerness to win a cash prize, bring new/ innovative products to market, and the potential to scale an existing product in a new global market (see Table 11). All finalists had worked in the off-grid or refrigeration sector, but only one had worked in the Ugandan market, while three were new to the East African market. This competition acted as a pathway for finalists to enter different markets across sub-Saharan Africa. Additionally, 57% of finalists stated that the competition pushed them to go beyond their expertise, as they sought to find ways to make their solutions more energy efficient and innovative. In particular, some finalists highlighted that the competition guidance and judging criteria inspired them to create new and innovative off-grid refrigeration solutions. Of the applicants, 86% had never applied for donor funding, while 14% of teams had applied for USAID funding. Among the finalists, 82% had never applied to donor funding and only two finalists—SunDanzer and SimuSolar had applied to USAID for funding. Until the competition, donors had not made a meaningful investment in this space (with the notable exception of vaccine 71.4% 14.3% 14.3% Chart 3: How long has your organization been producing off-grid compatible refrigeration products? 4 + years More than 12 months, but less than 2 years 0 - 6 months 50 51 refrigeration, which did and continues to receive sizeable donor funding). Private sector manufacturing companies in the domestic refrigeration sector rarely apply to USAID for donor funding. This competition allowed commercial enterprises to compete for financial and non-financial incentives, demonstrating the benefits that donors can offer the sector. It is challenging to contextualize the involvement of women in the solutions’ development, because there is minimal information available in the broader market literature. Within the nominations themselves, 25% of applicants had 21-30% of women working on developing their solution. For finalists, the distribution of women working on the solutions was spread across the range as seen on Chart 4 (0-10%, 11-20%, etc.). Given the low gender diversity among nominees, compared to the high likelihood that women will be the primary users of refrigerators, it is vital that solutions take a gendered perspective when developing their solutions to ensure that they are user-centered and innovative. Fenix International reported that women bring in more clients than men even though women only purchase 20% of SHS.11 More gender diversity may enable an organization to perform well in the rural electrification market than those that are more male￾oriented. 0-10% 31-40% 11-20% 41-50% 21-30% 51-60% Chart 4: Involvement of women in development of finalists’ solutions What percentage do women constitude of the team that develop this product? Outcome 4: Build B2B partnerships with distributors and investors to accelerate distribution. The Global LEAP Awards had a thoughtful, if somewhat aggregated, means of connecting finalists to partnership opportunities. The 2017 Buyer’s Guide for Outstanding Off-Grid Refrigerators and the Off-Grid Appliance Bulk Procurement Incentives program are both opportunities to highlight qualifying off-grid refrigerators to potential partners and distributors, but they occurred much later in the competition process and do not represent dedicated, individualized connections between competitors and prospective partners. Thus, before they became eligible for those programs, 50% of finalists had taken initiative to develop their own discrete partnerships in order to further develop or produce their solution, with a further 25% were optimistic about building partnerships in the near future. The off-grid sector needs effective and strategic partnerships in order to accelerate distribution of appropriate appliances, particularly refrigeration devices. During the initial stages of the competition, limited support was provided for partnership building. 88% of finalists highlighted that they mostly received support via introductions, subsidized product shipping costs, and the reports emanating from laboratory and field testing. While useful forms of acceleration, they do not necessarily result in partnerships. Fifty percent of finalists said they had been able to develop partnerships either to further develop and/or produce their solution or to distribute and scale their product with major pay-as-you-go solar companies, existing customers, importers, manufacturers, and distributors. Another 25% of finalists were in the midst of partnership discussions and confident of building new and valuable partners in the near future. The competition emphasized that partnerships would be crucial to the longevity of finalists’ solutions. SureChill was collaborating with a SOGE partner, the Shell Foundation, to do field testing in Kenya and India to identify user cases in semi-commercial environments for their off-grid refrigerators. Finalists have also exploited the synergy between the the off-grid appliance and solar home system industry to build partnerships in hopes of successfully developing and scaling their products. Two of the finalists, SunDanzer and SimuSolar, were collaborating with one another in order to produce an off-grid refrigerator and distribute it, as SimuSolar’s pay-as-you-go operating system and SunDanzer’s wide distribution networks would be mutually beneficial for global sales. Some finalists noted that the competition could have incorporated support mechanisms for partnership building. The CLASP team frequently met with distributors and investors to promote the competition finalists. One finalist noted that 52 53 they would have liked more notice of, and the opportunity to join, planned interactions with buyers and distributors (such as GOGLA events). The finalist would have liked to budget to join in these activities and envisioned that CLASP could potentially assist them in gaining direct access to contacts and explaining their products to potential partners. Regardless, the competition has connected finalists to a wider and more diverse network of potential collaborators, manufacturers, distributors and investors, as well giving them a platform and presence at many off-grid and solar events (see Outcome 8). Further, CLASP’s 2017 Buyer’s Guide for Outstanding Off-Grid Refrigerators displays all of the solutions with its rated product specifications, performance metrics based on laboratory testing and sales contact information, allowing key players in the market to see the quality of these solutions and potentially connect with finalists and winners. Outcome 5: Build consumers capacity (awareness to increase the value associated with solar systems - local business and final clients). The field test process dramatically and successfully increased customer perceptions of the value of off-grid appropriate refrigerators. Prior to the field test, Ugandans expressed a willingness to pay around $100 for an off-grid appropriate refrigerator. After experiencing the remunerative value of the test refrigerator, 84% of field testers would purchase the off-grid refrigerator they tested and were willing to pay between $200- $500 for a refrigerator. For this competition, it became imperative to improve consumers’ knowledge of off-grid refrigeration devices. Prior to the competition, USAID’s HCD research found that most people were aware of off-grid refrigerators and SHS, but not that they can be closely associated with each other. Potential users expressed disbelief this could exist, but were very interested. Despite the interest, when asked what they might refrigerate, some users proposed dried fish, tomatoes, and onions. As none of these foods require refrigeration, consumer education was needed. Field testing offered not only a chance to test with end￾users, but it also presented the opportunity to engage with a variety of users—domestic and commercial. Prior to field testing, the 36 field testers expressed different assumptions about solar/off-grid refrigerators, such as “it works efficiently without electricity,” “they work in areas without power,” “they work well,” “it cools drinks,” “it works like other fridges,” or they had “never heard about solar/off-grid refrigerators.” Some users had previously been connected to the grid, but dropped the service when they found that the costs associated with being on the grid—approximately 18 cents per kWh—impacted their profits. After field testing, 84% of the users claimed they would purchase the off-grid refrigerator they had been testing. There is an interesting contrast in what value they imagined for the refrigerators prior to receipt compared to having and valuing the product in-person. All users assumed that they would somewhat gain from having a refrigerator, be it in sales or customers, but that process wasn’t initially clear to them. Earlier competition research indicated that users would only be willing to pay ~$100 for a refrigeration device, however, as Outcome 7 notes, after being placed with a refrigerator, the majority of field testers became willing to pay a range of between $200 and $500 for a refrigerator. Users’ willingness to pay more after they have used the products shows how perceptions change once they have the refrigerator. Nabukenya Resty told us, “before having the refrigerator, I was making losses on pineapple and passion fruit juices, but now no more losses as I can store the juices in the fridge for much longer” – the changes were immediate. Critically, users noted that having an off-grid refrigerator connected to a SHS allowed them to accumulate savings, which then facilitated their ability to buy more items to store in the off-grid refrigerator. During the field testing period, overall, the 36 field testers, in total, had a 268% increase in customers from 494 customers as a baseline to 1817 in total from the field testing period. They note that having the refrigerator immediately attracted customers. Users also stated the importance of the position of the refrigerator, it must always be toward the front of the shop, so that potential customers can see it, as a refrigerator is seen as a high-value appliance, and it acts as a tool to convert potential customers into actual customers. Users also felt they understood solar and SHS much better once they were using the off-grid refrigerator. E4I, with Village Energy (a local Ugandan solar company that provided technical support), also taught users how to connect and disconnect refrigerators after activity and how to read the meter levels (understand what the red, amber, and green signals mean on some products). Many field testers were also keen to follow what they perceived were rules for using the products, namely, only storing the Alfred, a butcher in Gulu, Uganda had never seen solar power nor refrigerators; he was scared to even ask for the price since he knew he couldn’t afford it. There needs to be a better marketing system in local areas. 54 5555 products they intended to sell. This misperception was understandable, as major beverage distributors often require that only their products be stored in the refrigerators they provide to businesses for free as part of the beverage sales program. In this case, however, the refrigeration devices were furnished by the competition and no such restrictions existed. Once field testers were reassured by the E4I field team that there were no restrictions on the refrigerators’ contents, users began to store meats, milk, fruit, and fish. This guidance should have been made clear to users before field testing started, as it may have affected the results. Some of the field tested products were not best suited for domestic and small and microenterprise use. SureChill entered a vaccine refrigerator into the competition. The features needed to be compliant for the vaccine cold chain actively compromised the product’s utility for small and microenterprises. For example, SureChill’s side lock and limited interior space (compared to the larger outside form) was not a fit for commercial or domestic use; the features made it difficult for users to store and sell sufficient items to justify the refrigerator’s costs. Finally, while there was a marked increase in customer willingness to pay more for an off-grid appropriate refrigerator, this analysis did not not measure customer ability to pay or what sales volume would enable an ability to pay. While a subsequent section discusses financing options for refrigerator purchase, this analysis cannot claim if the willingness to pay $200-500 is feasible.12 The field testing period created value for users that extended beyond refrigerated products. Field testers saw other domestic and commercial uses for solar home systems —phone charging, lighting, opening a canteen, or starting a barbershop. In one community in Budaka, Mbale, four small and micro enterprises agreed to field test solar home systems and competing refrigerators. Not only did the refrigerators increase sales of cold drinks and perishable goods as hypothesized, the stores were able to power a light bulb and could stay open later, so overall store revenue increased. Itinerant food vendors positioned themselves near the stores with refrigeration and they also reported increased revenue as people bought more food over longer hours to accompany their cold drinks. Perhaps most surprisingly, the store owners became concerned that their valuable refrigerators would be stolen. They alone couldn’t guard the refrigerators, so they organized a neighborhood watch. Now, as a result of the neighborhood watch, the town as a whole is reporting fewer safety incidents. 56 57 Ecosystem The ecosystem category assesses the extent to which the Global LEAP Off-Grid Refrigerator Competition has influenced the off-grid refrigeration sector in Uganda (and sub￾Saharan Africa), policy and, more generally, knowledge and awareness of off-grid refrigeration devices and solar home systems. “Change the perception of the solar-powered system— some people assume that a solar power system is worse than on-grid power because it’s coming from the sun. They tend to underestimate the quality and the power of SHS. They also assume that the products should be free or less expensive as it’s using solar power. In this sense, since using the solar power and its products are less expensive than the regular on-grid power products, they assume that on-grid products are better since they are more expensive.” - Interviewee, USAID HCD Research Outcome 6: Grow a diverse market for efficient off-grid appliances to increase income, prosperity, and product quality. While the field test process found that 95% of field testers had increased sales by using the off-grid refrigerators, most of the competing refrigerators were not produced at a size or a scale and available to small and microenterprises (SMEs) at a price that justifies their purchase. Significant changes are needed in both the volume and costs of producing appropriate refrigerators, as well as Uganda’s regulatory environment to foster the conditions to allow refrigerators to increase customer income and prosperity on a wider scale. On the whole, the current demand for off-grid refrigeration solutions across developing countries (including Uganda) suggests low market penetration worldwide. Product prices range from $70 to $1,000. Interviewees recommended factoring in other appliances drawing power on a SHS and integrating a payment plan or financing options to overcome users’ income constraints. E4I found that the typical cost of a solar refrigerator ranged between approximately $600 and $15,000 (not including the costs of SHS and energy to power the products). For interested consumers, there are few refrigeration options that are available, accessible, and affordable. This competition hypothesized that off-grid refrigeration devices have a business use which may enable some individuals and communities to grow their income. Results from field testing found that this was the case, as 95% of users gained more sales by utilizing the off-grid refrigerators. In terms of monetary gain, business sales across the six-week field testing period increased by 255.49%, and daily shop income also rose by 255.42%. Users, however, noted difficulty in meeting the exponential demand that arose from them having an off-grid refrigerator. With growing demand, their business expenditures grew as well (value of sales on cold drinks grew by 403% in order to meet a customer base that grew by 238%). In particular cases such as with Agnes Nansubuga, she had the only refrigerator in the entire village and therefore felt pressured to continually ensure her shop is fully stocked with drinks. As mentioned in Outcome 5, field testers had to find ways of trying to meet that increase in demand. The growing demand, though difficult in terms of managing stock, was generally seen as very positive and enabled users to diversify and add new chilled or cold products to sell. 40% of users added popular product lines such as drinking water, passion fruit, passion fruit and pineapple juice, while a few also added milk and yogurt. A user noted, “these are new business lines which I started because of the fridge.” Some users also identified a growing market for popsicles and ice, but only a small subset of users were placed with a unit able to produce ice. This was due to both a limited number of combination refrigerator-freezer units and some refrigerators having restricted temperature setting options. On the supply side, market projections for competition finalists are very low (in the low thousands per year), as the majority Expected Ecosystem Outcomes: 6. Grow a diverse market for efficient off-grid appliances to increase income, prosperity, and product quality. 7. Build the value of solar home systems to increase demand for unique systems and increase the value of existing SHS. 8. Raise market awareness on the existence of high quality off-grid products. 58 59 SMALL MEDIUM LARGE AVERAGE OF TYPE REFRIGERATORS $459.90 $601.20 $820.56 $627.22 REFRIGERATORS￾FREEZER $316.80 N/A $1245 $780.90 Table 12 - Estimated Average Retail Price of Products by Category of them note that the off-grid market is slow to adopt the new and innovative solutions. SolarNow noted policy issues affecting their ability to sell, as the Ugandan government introduced a tax on consumer electronics and mobile money transactions (affecting both customer financing and PAYGO). Although prize winners Palfridge notes that though they have been commercially producing their winning product, they have faced issues getting products to different Southern African countries. This is due to the small amount of stock they have, and they have not been able to produce them in larger quantities as it is too costly. All finalists acknowledged they struggled to sell their products. The issues are twofold: There are issues at the supply level as manufacturers and distributors negotiated to ensure the costs of production and distribution were not too high, then demand issues in being able to engage with customers to demonstrate feasibility and remunerative productive use. The competition brought attention to the sector, but it did not facilitate a transformation among atomized manufacturers, distributors, and consumers. In terms of pricing, the average retail price of the off-grid refrigerators is $827.16, Table 12 provides a further break down of the average prices according to the category type of refrigerators available. Given what the user cost-perception analysis reveals in Outcome 7, users speculate they would be willing and able to afford the small and medium refrigerators (between $200-$700).13 Based on some of the pain points identified during the field testing, a majority of field testers, however, wanted larger sized refrigerators to support their business needs, and only a fraction of users were willing to pay $700+. Table 13 also displays the performance and price of the competition and innovation prize winners, with all the products sitting on the customer price spectrum on $200 to $700. Consumers are also dependent on whether distributors include financing mechanisms—whether as separate refrigeration products or included as part of an SHS package. The choice in the product, package, and financing offerings will largely determine whether users will be able to access and afford them. Finalists have struggled to position their products and establish the price points for their products, as there is minimal available Ugandan market information. However, there are successful SHS companies that may be interested in strategic partnerships and it is recommended the finalists consider these type of opportunities to leverage successful distribution networks. While user willingness to pay could provide them with a target to meet, the high costs of production (on average $2,035.48), made it unlikely the current range of solutions would achieve a price point that consumers consider affordable. Lowering the appliances’ cost is contingent on working in collaboration with key market players—manufacturers, distributors, government agencies, and other stakeholders—to produce reliable products at scale, identify demand and distribute the products efficiently, and to lower customs and tariffs that prevent the easy import of appropriate devices. Finalists argued for better facilitation with market players and sought more invitations to events, meetings, and platforms for direct (or face-to-face) involvement with solar and off-grid actors to ensure they could pitch and position their solutions appropriately. Outcome 7: Build value of SHS to increase demand for unique and existing SHS. The field test process found that 74% of field testers would be willing to pay between $200 and $700 for an off-grid refrigerator bundled with SHS. While this represents a positive new association between off-grid appropriate refrigerators and solar home systems, it perhaps says as much about the known financing options available via solar home systems as it does a willingness to increase an existing SHSs value. There was no demonstrated increase in unique solar home systems as a result of the competition, but there were new off-grid refrigerator sales. Off-grid appropriate refrigerators are one appliance type among many (lights, chargers, TVs, fans, clippers) which, with adaptation for efficiency, work via SHS. Building awareness that refrigerators can be solar home system-appropriate will help develop demand for new solar home systems and existing customers increasing the value of their SHS. 60 61 Winner Type Category Type Category Average Daily Energy Consumption Daily Energy Consumption Declared Daily Energy Consumption Deviation from Declared Daily Energy Consumption External Temperature (C) Humidity (%) Estimated Retail Price ($) Metropolitan Elect MEFR-48C1 Competition Winner Small refrigerator￾freezer 0.65 0.57 0.33 71.8% higher tested energy consumption than declared value 31 71 167.4 SunDanzer DCR50 “Overall Value” Innovation Prize Winner + Competition Winner Small refrigerator 0.31 0.28 0.11 152.4% higher tested energy consumption than declared value 26 66 448.2 Fridge Star LC86 “Appropriate Design and User Experience” Innovation Prize Winner Medium refrigerator 0.77 0.22 0.39 42.4% lower energy consumption than declared value 24 49 601.2 Basil iFreez DC170L Competition Winner Large refrigerator￾freezer 1.65 0.5 0.72 31% lower energy consumption than declared value 27 62 675 Fridge star LC221 Competition Winner Large refrigerator 0.55 0.38 0.39 3% lower energy consumption than declared value 26 66 572.4 SunDanzer DCR165E “Energy Efficiency” Innovation Prize Winner Large refrigerator 0.55 0.34 0.14 139.5% higher tested energy consumption than declared value 24 49 718.2 Table 13 - Performance vs. Price of Competition and Innovation Prize Winners The research also addressed the perceived value of SHS and solar power. During the field testing, field research was conducted with 12 of the 36 field testers. The twelve users provided feedback on their experiences using the off-grid refrigerator and the value of SHS. All of the field testers interviewed had varying degrees of knowledge about solar products and of refrigerators. The perception of solar power and SHSs is that “it is cheaper, once you have it you save” and “it serves all the time, once you get it, you won’t lose power.” Users also associated safety with SHS, based on past experiences fearing faulty wiring with AC/grid power could cause a fire. Testers also felt reassured that SHS wasn’t susceptible to electrical shocks. Finally, field testers valued that solar power produces cleaner, safer energy than wax candles or kerosene, which smoke and can be unsafe. Rashida Nasungo from Bufombo, Mbale said that once she told her friends about the refrigerator and how great it was, her friends wanted more off-grid appropriate appliances. Her friends wanted solar and off-grid appropriate appliances because they doubted the national grid would arrive in their communities in the near future and even once it did, they doubted their ability to afford it. Outcome 8: Raising market awareness of existence of high quality off-grid products The competition made a mixed contribution to market awareness of available off-grid appropriate refrigeration devices. While everyday Ugandans may not demonstrate brand awareness of most available, off-grid appropriate refrigerators, distributors are now aware of the 17 high quality off-grid refrigerators available on the market via the 2017 Buyer’s Guide for Outstanding Off￾Grid Refrigerators and the ten refrigerators participating in the Off-Grid Appliance Procurement Incentives Programme. The Global LEAP Awards Off-Grid Refrigerator Competition sought to improve market awareness off-grid appropriate refrigerators through multiple modalities: Field testing, the Off￾Grid Applicance Procurement Incentives Programme, and via the competition itself. Awareness of off-grid refrigerators and SHS was very low, and Lighting Africa found that brand awareness was extremely low in Central and Western Uganda. 83% and 86% of people could not recognize SHS brands or companies, highlighting the low level of market penetration.14 Among field testers, most only knew of only a few solar companies—SolarConnect and Village Energy. Across the field testing sites, brand penetration was low. Perhaps this can be attributed to the costs associated with marketing and expanding into rural areas. 62 63 Relatedly, only SolarNow had brand recognition, mainly in Northern and Eastern regions of Uganda at 34% and 38%. SolarNow is the only finalist operating in Uganda providing small (35L) and large (112L) off-grid refrigerators, and they have reported issues in reaching rural areas in Uganda. The expenses associated with trying to target rural communities are such that strategically it makes more sense for them to target cities to grow their customer base. Table 14 compares the off-grid refrigerator markets in several developing countries. In particular, Uganda has extremely low market penetration and refrigerator prices vary across the largest range—a finding supported by the field testing results which have the same range (see Outcome 5, 6, and 7). There was a lack of awareness of not only solar home system companies and off-grid appliance companies, but there was also limited customer linkages between them. That is, customers did not readily connect off-grid appropriate appliances and appliance distributors with how they may relate to solar home systems and their plans. Following testing, the field testers demonstrated improved understanding of the relationship between off-grid products and solar systems, with many wanting to buy additional appliances from phone chargers to TVs. Nonetheless, brand recognition is still low and users are not aware of which companies provide high-quality products. The Global LEAP Awards Off-Grid Refrigerator Competition sought to improve market awareness at multiple levels. Moving from Uganda, to an ecosystem level, the competition supported field testing 17 high-quality off￾grid refrigeration products, now available on the market, of which 16 participated in the Global LEAP Off-Grid Appliance Procurement Incentives Programme. Solar manufacturers and distributors regularly search for new innovations at cost￾effective prices. UltraTec, a Ugandan based solar supplier and distributor which supplies customers ranging from restaurants to the agricultural cold chain, uses the Global LEAP Off￾Market Penetration Refrigeration Capacity (L) Price Range ($) Bangladesh Very low 60-80 300-500 India Low 45-55 70-625 Kenya Very low 50-55 -300 Nigeria Low to medium 50-55 -300 Uganda Extremely low 50-80* 200-1000* Table 14 - Developing Countries Off-Grid Market15 Grid Appliance Procurement Incentives Programme. In an interview, UltraTec noted how valuable the competition has been in discovering new and innovative off-grid refrigerators in the different product categories. UltraTec posited that the competition’s off-grid refrigerators could provide more energy￾efficient systems for their clients to reduce milk spoilage, particularly from rural to urban areas. In addition to the field testing and Off-Grid Applicance Procurement Incentives Programme, the Global LEAP Awards Off-Grid Refrigerator Competition’s communications and outreach tactics also sought to improve broader market awareness of off-grid refrigeration. Global LEAP adapted its communications campaign to publicize the announcement of the Innovation Prizes, the competition’s winners and finalists, and the award-winning, high-quality products. Ultimately, 85% of finalists felt that Global LEAP offered their off-grid products and organizations credibility in the off-grid market that elevated their products to investors and customers. The competition also leveraged industry events and conferences to promote the competition, its finalists, and winners. The competition was launched at the Global Entrepreneurship Summit in September 2016. The finalists, winners, and the efficiency and value innovation prize winners were announced at the Global Off-Grid Solar Forum and Expo hosted by GOGLA in Hong Kong in January 2018. The Forum and Expo event was attended by seven finalists including the winners, and helped them build partnerships and network with stakeholders. Finalists were also able to demonstrate their solutions and network with distributors and manufacturers.16 The appropriate design and user experience innovation prize was announced at Unlocking Solar Capital event hosted by GOGLA and SolarPlaza in November 2018. Though no competitors attended, Global LEAP promoted the winner of the prize, Palfridge, to industry investors and distributors. 64 65 Value for Money Value for money assessments seek to answer whether the competition provided a return on its investment. Due to the aforementioned delays in the competition, there was a shorter than anticipated amount of time to compile final information and conduct this analysis. In order to complete a meaningful value for money calculation, the following information is needed: • All financial information broken down in a consistent way to correlate the types of activities undertaken (i.e., events, communications outputs, materials produced). This would be required from all partners, and should equal the total operational budget. • If possible, any in-kind contributions that were made or could be assigned, and then, • A breakdown of the activities undertaken throughout the competition and the number of times that activity occurred. With this information, it would be possible to calculate the operational efficiency of the competition, by breaking down the number of activities conducted and the cost associated with those activities across the competition. Operational efficiency is equal to: • Total operational cost / no. of activity carried out = cost per activity • (Activity/ months) x cost of activity = total value per months – predicted cost per months = Efficiency We were, however, able to calculate a basic level of programmatic ROI of the Global LEAP Off-Grid Refrigerator Competition, as it maps out the extent to which the outcomes have been achieved relative to the program costs. This only presents a cost per outcome and does not take into account the more detailed level of analysis which requires a more granular level of budget breakdown. The calculation is as follows: Total ROI: Total program costs to impact metrics = Total program cost / 8 outcomes = cost per outcome x no. of outcomes in impact category = cost per category vs results and progression on all impact metrics (progression are results vs expectations) Market and beneficiaries proxy = progression x time. Forecast (- drop out %) = market impact vs beneficiaries potential. The total program cost is $1,893,049.1517 which is equal to an average cost of $236,631.14 cost per outcome (total program cost/# of outcomes). In Table 15, we provide a list of the level of investment for each impact category (outcome investment x # of outcomes in the category) and for each category it assesses the quantitative results which are possible to observe so far. It is difficult to tell if this level of impact would have been achieved under normal market conditions with this timeframe, primarily due to there being insufficient research, funding, and a lack of programs supporting the sector. However, this does provide a benchmark for future programs, funding streams and competitions in the off-grid refrigeration sector as to the level of impact that can be achieved within a certain timeframe. To look at the ROI for each impact area, the budget has been broken down to reflect the proportion of budget that correlates most appropriately to those particular indicators. Innovation - $473,262.29 / 3 indicators = Average $157,754 per indicator To properly assess the ROI, comparative assessments need to be made against: • The number of new innovations introduced to the market and associated R&D costs which might have been incurred under normal market circumstances for their development • The number of units tested in the field and the comparative value of equivalent testing time under regular conditions of testing • Social value attributed to satisfaction of use from the field testers Capabilities - $709,893.43 / 4 indicators= Average $177,473.36 per indicator Proportional calculations can be assigned against: • The anticipated number of nominations versus the received number of nominations • The number of new innovators that entered the field due to the competition • The number and nature of the new partnerships and the value of any additional investment, be it financial or in kind • Proportion of users that express an increase in knowledge due to the competition and the type of knowledge they have attained which can be quantified against training values in these areas; also the intrinsic learning values 66 67 KEY PERFORMANCE INDICATOR AS IMPACT BASE MAIN RESULTS EXPECTED RESULTS ACHIEVED VALUE Innovation Investment of $473,262.29 Performance of products compared to market Most finalists will outperform the baseline refrigerators 12 to 13 finalist off-grid refrigerators outperformed current off-grid refrigerators in the market in four major tests (steady-state operation [10C, 32C, 43C] and pull-down time) Achieved Performance of products in field Products will work in the field 27 products worked in the field with the average daily energy consumption being 0.69 kWh p/day. 3 performed better than expected Achieved User satisfaction Users will be satisfied with products 96% of field testers were at the very least happy with the off-grid refrigerators, and a further 100% overall found the product easy to use Achieved Capabilities Investment of $709,893.43 # of nominations 20 nominations 57 nominations Achieved % of new talents Attract new talents to the off-grid refrigeration space 4 nominees were new to the off-grid refrigeration space Achieved # of new partnerships Generate new partnerships 50% of finalists created new partnerships. 25% are optimistic about creating partnerships in near future Achieved % off-grid solar knowledge Users will improve off-grid solar knowledge 33% of users better understood off-grid refrigerators and SHS Neutral Ecosystem Investment of $709,893.43 % improvement in user business sales vs expenditure Users will improve their business sales 95% of field testers increased their sales, overall sales across the testing period increased by 255.49% Achieved Market projections of finalists and winners Finalists and winners will have strong sales projections Sales for finalists and winners ranged from zero to in the low thousands, with prize winners reporting increases from the previous year’s orders Negative Cost perception analysis Calculate the price points of users The majority of users would pay $200 to $500 in cash by installments paying $10-20 per month over a period of 1 to 3 years for an off-grid refrigerator with SHS Achieved # of brands known to provide high quality off-grid products The products produced in this competition will be of high-quality 17 high-quality off-grid refrigerators were produced by this competition and are on the market Achieved Ecosystem - $709,893.43 / 4 indicators = Average $177,473.36 per indicator The most challenging of areas to calculate ROI as should be based on broader projects and attribution is more complex to assign. However, some premise for the calculations: • Increased sales attributed to new products in the field testing process • Market projections need to be proxied against a specific timeframe under new commercial conditions • Based on the market projects of how many units are anticipated to be sold at the price points that users are willing to pay and the duration of time for payments ($200 to $500 in cash by installments paying $10-20 per month over a period of 1 to 3 years for an off-grid refrigerator with SHS) • Attributional value of facilitating the introduction of 17 new products on the market To determine the value for money of the competition most effectively, these ROI calculations should be combined with the operational budget values derived from the activity calculations to provide a comprehensive analysis of the value that the competition presents against what normal market parameters would have achieved within the same timeframe. Table 15 - Total Program Costs to Impact Metrics 68 69 Section 3 Lessons Learned Learnings The 2016-2017 Global LEAP Off-Grid Refrigerator Competition sought to incentivize technical innovation, support innovators, and influence the broader ecosystem about the value of off-grid-appropriate refrigeration devices and SHS. This section focuses on the competition’s key lessons learned, namely, the use of mixed methods research, the competition’s incentives, dynamic partnerships, and evaluating perceived and actual market impact of the competition and its solutions. The section will also address the challenges the competition confronted, as well as offer practical suggestions to improve future competitions. Research - Mixed Methods Using mixed methods research in the competition scoping and design process enabled the prize team to develop a multifaceted understanding of the competitors and market. The mixed method approach to the Global Off-Grid Refrigerator Competition provided an evidence base to build strong benchmark and baseline indicators and support for the competition. • DAI, CLASP, and USAID conducted a scoping process involving key informant interviews, desk research, and facilitated discussions at conferences which contributed to the prize design and incentives. • DAI and CLASP concurrently ran a survey and request for information process to solicit stakeholder feedback on existing refrigeration products, customer needs, and key elements of the proposed competition design. • CLASP conducted a baseline testing analysis using 36 off-grid appropriate refrigerators to understand what an innovative solution might look like. This process became the foundation of the laboratory test method. While the laboratory test method drew from other sources (such as testing for vaccine refrigerators), this was the first test method exclusively created for testing off-grid refrigerators for domestic and SME use. • Additionally, CLASP shaped the eligibility criteria and guidance through the baseline testing as they had to consider ‘what innovation looked like in off-grid refrigerators’, which offered innovators help in developing their solutions. • E4I conducted a study on 173 micro-enterprises in Uganda using on-grid refrigerators to understand the use and application of off-grid refrigeration, providing a benchmark. • USAID supported HCD research to gain a rich qualitative understanding to the Ugandan context. The research identified items people want to cool, ideal run time and size of refrigerators, price point, and the most preferred features. • E4I and CLASP conducted field testing in Uganda, and created a specific field-test method to not only capture the technical information, but also the more meaningful context￾specific information - the first of its kind. • In addition to standard qualitative features (e.g. pull-down time), qualitative judging enabled more focus on the end￾user and the way in which the products would be utilized, meaning that elements of design and usability, innovative features and functions, and truth in advertising (products tested performance versus specified/claimed performance) were investigated as well. These activities ensured the competition had a multifaceted understanding of existing off-grid technologies, customer, distributor, and solar home system demand and constraints, and This section provides competition lessons learned from the research, design, implementation, and evaluation phases, suggested actions that can be taken to improve future competitions in this sector, and other market-shaping activities. 70 71 the off-grid refrigeration market in Uganda. The thorough mixed methods approach created a solid foundation for the program design and implementation. Incentives - Design The competition offered both financial and non-financial incentives to stimulate innovation. Efforts to identify compelling incentives began during the research and design phase, in hopes of addressing both competitors’ motivations to apply and to use incentives strategically to fill gaps in the market. The competition’s incentives included: Cash Prizes helped the competition attract both innovators known to the off-grid refrigeration space, as well as to motivate new actors to enter this space. The majority of finalists were motivated by the opportunity to win one of the innovation cash purses. Compared to traditional grant or procurement programs, innovation inducement prizes often attract mass supplier interest more quickly, easily and efficiently. In the off-grid refrigeration market, an area that is technology and innovation-driven, cash purses incentivized innovators to create new solutions. When asked, all finalists said they found the $200,000 Innovation Prizes sufficient for what was being asked of them in the competition. Some finalists noted that they would have liked to see more prizes available (e.g. five prizes worth $100,000) to reflect the work that went into creating their solutions and to have more opportunities to signal to the market that their products demonstrated “prize￾winning” value. Testing was an important incentive for finalists; 100% of respondents appreciated the testing elements of the competition. Under normal circumstances, product developers and manufacturers have to pay for testing costs themselves—but the competition subsidized these expenses to remove barriers to entry. Even if refrigeration developers or manufacturers were to pay for testing, as noted earlier in the report, until the Global LEAP awards, there was no standard laboratory or field testing protocol for off-grid appropriate refrigerators. The Global LEAP testing process filled a needed gap by introducing a standardized process. Moreover, the laboratory test design allowed for a robust, technical assessment of the refrigerators, while still recognizing critical qualitative elements. The test results were captured in an extensive report for innovators to use and improve their solutions. The Global LEAP Awards also designed and sponsored field testing. Field testing provided innovators with a real understanding of how their solution was received and utilized by customers. This customer feedback was relayed to competitors. All finalists who participated in field testing were generally satisfied with the results. They did, however, note (as discussed at length in Outcome 1) that they would have appreciated more detailed field reports. Competitors would have liked more information on their product’s technical performance, the environmental conditions that their solutions were placed in, and user price points and feedback. Instead, the field testing report provided a high-level of field performance and user response. The Global LEAP field test process was a first step, but not a substitute for a longer, more detailed field test process that logs user performance and geographic information in service of honing solutions to meet end-user needs. Global LEAP reputation was another reason why innovators applied for the competition. Finalists recognized that their organizations would gain brand exposure through Global LEAP’s network and partners. Additionally, the Global LEAP imprimatur is the quality standard for off-grid appliances: throughout the competition, all finalists consistently noted that they valued the prestige afforded to them as Global LEAP finalists and winners. Complementing the prizes, the Global LEAP Off-Grid Appliance Procurement Incentives Programme (RBF) aimed to accelerate the uptake of off-grid solar technologies such as solar home systems through the development of a complementary market for high-quality and energy efficient off-grid refrigerators (please see Market Impact on page 74 for further details). By coordinating purchases in bulk, this program reduces the costs of appliances by enabling manufacturers to produce them at scale and helps collectivize resource-intensive challenges such as shipping and customs. Qualifying to participate in this program reduced some of the friction competitors would have experienced entering weak markets alone. 72 73 Though all these incentives can easily work in isolation, the key is that they worked in combination with one another to incentivize, motivate, attract, support, and develop innovators and their off-grid refrigeration products. Dynamic Partnerships - Implementation “The partnerships [...] ended up yielding unbelievable results...the other things that the partners brought to the table is what made it so successful, without question.” - Jeff Stottlemyer, CLASP - Partner Partnerships were integral to the Global LEAP Off-Grid Refrigerator Competition. The competition had multiple partners: three major agencies including USAID (and the Power Africa US Government initiative), DOE, and UK aid/I2I; three implementing partner organizations—CLASP, DAI, and E4I; and two evaluation teams in Nesta and IMC. A large partnership has its pros and cons. The donor coordination between USAID, DOE, and UK aid efficiently pooled resources and set an agenda around a topic of common interest. Unfortunately, however, there was also a learning curve regarding the process and speed of funds transfers between USAID/DOE/CLASP, which created delays in shipping the products for laboratory testing, and, in turn, delayed the competition timeline.18 Fortunately, as a function of the partnership, when the delays became untenable, UK aid/I2I stepped in to support shipping of solutions for laboratory testing (and was later “reimbursed” by USAID). This case is a prime example of what the partners mentioned as ‘pinch points’ at various stages of the competition. These ‘pinch points’ vary from issues in providing funds in a timely manner, to a complicated clearance process, to delays in the laboratory and field tests due to challenges with importing products into the country. Ultimately, however, the negatives were outweighed by the positives. The partners cited the collective skills and resources dedicated to deliver this program as contributing to positive outcomes measured in the innovation, capabilities and the ecosystem categories. In a focus group, after the first innovation prizes were awarded, partners acknowledged that each organization brought something unique to the table. For example, CLASP has extensive expertise of working in the off-grid space, and were able to utilize their skills and knowledge for the competition. E4I are experts in field testing and created the first test protocol for off-grid refrigerators. DAI was able to provide insight and knowledge with regard to prize design and brought expertise through their management of various innovation competitions. The partners attested that resulting competition design and implementation plan representing their collective expertise was better than the sum of its parts. From a donor perspective, this competition called attention to a new donor role in signalling and stimulating markets. Katrina Pielli of Power Africa called the competition “an example of an appropriate role for donors—recognizing what markets need when private sector is not present, and thinking about how we push to help develop the markets.” Going forward, donors should keep in mind this work is most successful when donors are engaged, collaborative, and mindfully stimulate—rather than distort—markets. In addition to E4I, CLASP, and DAI’s competition’s design and implementation support, the Global LEAP Awards also leveraged support from other partners, namely, Energising Development (EnDev) and Power Africa to fund the Off-Grid Appliance Procurement Incentives Programme and create a clear path to market for finalists’ solutions. By collectively pooling financing for various competition activities, the Global LEAP Awards were able to offer an array of incentives which have assisted innovators in bringing their solutions to emerging markets. 74 75 Market Impact - Evaluation “I just had no idea of these players, this market...I had [kind of] heard things about it. But I’m so glad I am here meeting the right people who potentially need thousands and tens of thousands of these fridges.” —David Bergeron, SunDanzer - Overall Value and Energy Efficiency Innovation Prizes Winner A key part of this competition was to spur market impact at the innovation and wider ecosystem level. Being able to attribute market impact to the competition helps assess the competition’s value. Global LEAP positioned competition finalists and winners the market in two distinct ways. First, Global LEAP published the 2017 Buyer’s Guide for Outstanding Off-Grid Refrigerators. The guide serves as a procurement tool for off-grid solar companies and off-grid solar product distributors. The document provides market intelligence to interested stakeholders and alerts them to high quality, vetted, tested products. This was a key tool in ensuring finalists got exposure to a wider pool of players in the off-grid solar ecosystem. Another incentive to finalists and winners is the aforementioned RBF Global LEAP Off-Grid Appliance Procurement Incentives Programme. The RBF program sought to encourage off-grid appliance suppliers and solar distributors to place bulk orders of selected finalist appliances and market them to their customers. In return, Global LEAP offered an incentive structure that covered a percentage of the FOB (Free on Board/Shipping) price for eligible products. RBF sought to further reduce barriers to market participation as well as foster productive long-term business relationships. Ten of the 17 qualifying finalists participated and benefited from market facilitation between manufacturers and distributors and investors. In fact, the program received three confirmed orders from companies operating in East Africa to purchase 1,025 highly energy-efficient, off-grid appropriate refrigerators. These orders represent some of the world’s first large-scale, off-grid refrigerator procurements. Winners and finalists also have access to the Efficiency for Access Coalition and Acumen investor network which supports debt and equity investors to gain market intelligence, builds an investable pipeline, and allows investors to ask more technical questions about specific off-grid appropriate technologies during their due diligence. This is critical considering the role that impact investing plays in the solar and off-grid market, as potential investors are seeking to invest in technologies and organizations with proven results. The Efficiency for Access and Acumen network provides a platform that immediately connects investors with productive use companies such as the competition finalists and winners. This network offers winners and finalists an immediate opportunity to gauge investors’ perspectives on investment, and pending mutual interest, is a source of necessary capital to scale solutions to reach low￾income users, grow core operations, and finance inventory. In sum, the competition signaled to the market that off-grid refrigeration worth additional investments from innovators, distributors, manufacturers, solar companies, governments, and international institutions. Collectively, they have the potential to unlock a substantial market in sub-Saharan Africa. 76 77 Reach: Despite the competition’s aspirations of inspiring appropriate, locally￾designed technologies for sub-Saharan Africa, the competition struggled to deliver. Most competitors came from the United States, United Kingdom, or China and most devices had price points that exceeded customer willingness to pay. New Technical Assistance: The competition can continue to experiment with new forms of technical assistance to competitors such enterprise support, market analyses, or more individualized forms advisory services. Stronger Methodologies: The competition created new and innovative laboratory and field test methods. Future rounds of the competition should draw on the ensuing lessons learned continue to improve on them. Communications: Some finalists stated that the competition failed to relay important information, such as the opportunity to participate in the GOGLA Off-Grid Solar Forum and Expo or detailed testing results and feedback in a timely manner. Timing: The delays associated with funds transfers, shipping, testing, and customs impacted the competition length and what could be measured. The competition was also not always forthcoming about the length and reasons for delays causing confusion among competitors. Ongoing Opportunities for Innovation: While the competition incentivized the development of innovative, new off-grid refrigeration solutions, field test feedback demonstrated room for additional innovation—particularly to increase device size, lower costs, and other geographies with different climate characteristics. Opportunities Challenges Challenges and Opportunities The competition experienced many challenges, especially reach, communications to participants, and timing. The competition process also exposed several key opportunities which could allow the off-grid refrigeration market to prosper, namely, new technical assistance, stronger methodologies, and ongoing opportunities for innovation. Recommendations Future iterations of the competition may be improved by instituting the following recommendations: • Deliberately facilitate additional exchanges between innovators and off-grid/solar stakeholders—users, local solar companies, local and global distributors, and investors so that innovators might respond to their needs. • Conduct extended field testing. Although the competition initially envisaged twelve weeks of field testing, this became untenable under the competition’s timeline. In future iterations of the competition, a minimum of twelve weeks of field testing will make the ensuing data more reliable. • Relay more detailed data from field testing to competitors. Competitors requested additional data on field test environmental conditions, user behaviors and user perspectives so they might incorporate those insights in their next iteration of the products. • Consider field testing with households. Field testing with households would help better understand the factors associated with domestic and productive uses and can add to a body of knowledge on household refrigeration in developing country contexts. • Develop and maintain clearer communication channels between partners and competition stakeholders. Finalists mentioned feeling unaware of competition developments. Judges were, at times, difficult to reach. Consider creating an online forum via Slack or Google to ensure all participants, from finalists to judges, are well aware of the competition process, as well as any news and updates. • Clarify roles. Consider being more explicit about partners’ roles at the start of the competition to determine roles, responsibilities, and goals, but ensure capacity for flexibility and overlap, if another dynamic partnership is formed. • Require a business plan. Consider requesting a business plan from competitors that explains the solution in 78 79 comprehensive detail, how and why the solution is innovative, what potential plans to market and scale are, what knowledge they have of users and developing contexts, and so forth. This business plan could be evaluated by judges as part of the decision-making process for the competition and innovation prize awards. It also provides a better understanding of innovators’ perspectives on potential use cases of their solutions. • Level the playing field. Explore ways to support innovators working in developing country contexts so they are able to compete alongside already established organizations from developed countries. The shipping and testing subsidies were a significant step toward lowering barriers to compete, but other considerations such as accepting applications in languages other than English or lowering warehousing requirements would enable more early stage and developing country competitors to compete. • Explore new tactics to improve innovator participation in evaluation activities. To mitigate low response rates from finalists to participate in interviews and surveys, consider either strategically pairing interviews and surveys with key milestones such as laboratory test shipping forms or field test results or offering incentives to finalists to participate. • Invest in the resources, both with respect to technology and staff, to continue to measure and monitor : • The refrigeration devices’ technical performance in the field, and • Evaluate the price points for refrigeration devices, relative to energy consumption and remuneration to determine both the right price for the products as well as when average owners can recoup their investment in them. • Consider placing more emphasis on the competition being a means to encourage adoption of SHS. If innovators see a clear overarching goal in not just supplying off-grid refrigerators to end-users in developing contexts, but also see the competition is about building value and trust in SHS and solar, their innovations will reflect that. Conclusion Many stakeholders are avidly monitoring the off-grid appliance sector, and as a shortcut, they have been following the market signaling and shaping activities of the Global LEAP competitions. The Global LEAP competition has spurred and signaled to the market energy efficient and high quality lights, fans, TVs and, for the first time in 2016-2017, off-grid refrigerators. This is vital information for market stakeholders as they respond to the findings from a recent Efficiency for Access Coalition survey on off-grid appliances.19 The survey found that demand for refrigerators continues to grow and represents a huge commercial opportunity. The survey featured five productive use cases for refrigerators: Household refrigerators, refrigerators/freezer units for commercial use, agricultural cold chain technologies, milk chilling units, and vaccine refrigerators. These use cases were consistently cited as both sources of potential demand and potential positive impact on socioeconomic development and poverty reduction. The diversity of productive use opportunities may also attract more innovators, distributors, manufacturers, investors, governments, and international organizations to develop the off-grid refrigerator market, through product research, development and manufacturing, policy reform, financial and capital mechanisms, and distribution network models. In response to the domestic and SME use case, this competition has produced new and innovative off-grid refrigeration devices that offer better energy efficiency, cooling efficiency, and autonomy than those previously active in the market. In addition, the finalists and winners have adopted high quality innovative designs to meet users’ needs. That said, while the competition has focused attention, innovation, time, money, and resources in the space, the Global LEAP Off-Grid Refrigerator Competition alone cannot achieve systemic change. More needs to be done at all levels—further investment, donor-development projects, pilot grant funding, technical assistance, and other open innovation competitions—to transform the off-grid refrigeration market to ensure access and affordability to low-income users. While this competition cannot claim to have built a robust off-grid refrigerator market in sub-Saharan Africa, it has indelibly shaped the market by creating formally established tests and benchmarks to guide future activities in the off￾grid refrigeration sector. We look forward to seeing both individual products’ performance in the market as well as the competition’s broader ecosystem impacts reverberate through this dynamic sector. 80 81 5. References 1. Page 9 - Following changes in 2016 and 2017, the Department of Energy’s participation became limited to coordination via Power Africa. 2. Page 10 - http://documents.worldbank.org/curated/en/102741561002617590/text/ Concept-Project-Information-Document-PID-Uganda-Energy-Access-Scale-up-Project￾EASP-P166685.txt 3. Page 10 - One of the findings from the field testing process is that, in Uganda, stores that have sufficient foot traffic to sustain an off-grid refrigerator are rare. Most of the stores that have the foot traffic to afford the costs of energy and a refrigeration device are on the grid or on the verge of being connected to the grid. Many off-grid stores identified for field testing struggled to have sufficient customers and revenue that may influence decision-making about the expenses associated with buying a refrigerator and paying for its energy use. 4. Page 22 - A method developed referencing internationally-accepted test standards. Referenced test methods include IEC62552 (househ5old refrigerating appliances – characteristics and test methods) and WHO/PQS/E003/RF05-VP.4 (refrigerator or combined refrigerator and water-pack freezer: solar direct drive without battery storage). 5. Page 22 - Truth in advertising refers to a product’s tested performance versus specified or claimed performance on key performance metrics (e.g. power consumption). 6. Page 24 - Unless there was a product warranty, field testers were obliged to cover subsequent costs and repairs following the testing period. 7. Page 41 - Please see CLASP Off-Grid Refrigerator Baseline Testing Data Analysis here - https://storage.googleapis.com/e4a-website-assets/Market-Baseline-Report-Off￾Grid-Refrigerators-May2018.pdf 8. Page 42 – It should be noted that some products results could not be determined or it was not possible to determine from the laboratory testing. 9. Page 43 – Two solutions were tested per finalist in field testing . 10. Page 43 - Despite delays, the refrigerators that could be repaired, were repaired and left with testers. The refrigerators that did not work were collected by local E4I and CLASP staff and disposed. 11. Page 50 - https://www.greentechmedia.com/articles/read/why-gender-is-key-to￾african-off-grid-energy-sales#gs.sX37bGU 12. Page 54 - Nor can it claim that original willingness to pay $100 is feasible. 13. Page 59 - But this is speculation as ability to pay was not measured. 14. Page 62 - Market Assessment of Modern Off-Grid Lighting Systems in Uganda. Lighting Africa, 2016. (https://www.lightingafrica.org/wp-content/ uploads/2016/12/Uganda-2.pdf) 15. Page 62 - The State of the Off-Grid Appliance Market. CLASP, 2016. (https:// storage.googleapis.com/clasp-siteattachments/The-State-of-the-Global-Off-Grid￾Appliance-Market-Report.pdf) 16. Page 63 - Conference attendance was made by teams on individual basis and the competition did not subsidize their participation. As a result, some finalists decided not to demonstrate their products. 17. Page 64 - We are unable to account for how much of the program cost was for operations. 18. Page 72 - The reverberations of this delay extended to an abbreviated field test process, and less time for impact to occur and then be feasibly be measured and reported in this framework and the other partner learning documents. 19. Page 79 - Off-Grid Appliance Market Survey: Perceived Demand and Impact Potential of Household, Productive Use and Healthcare Technologies, Third Edition, Efficiency for Access Coalition, September 2018. (https://storage. googleapis.com/e4a-website-assets/Market-Survey-2018.pdf) 6. Acronyms 80 AC/DC Alternating current and direct current B2B Business-to-business C Celsius DFID UK Department for International Development DOE United States Department of Energy E4I Energy 4 Impact, a not-for-profit organization working towards energy access in Africa, with offices in Kenya, Uganda, Tanzania, Rwanda, Senegal, and the United Kingdom EnDev Energising Development, an energy access partnership currently financed by six donor countries: the Netherlands, Germany, Norway, United Kingdom, Switzerland, and Sweden FOB Free on board/shipping Global LEAP Awards GOGLA Formally Global Off-Grid Lighting Association HCD Human-centered design I2I Ideas 2 Impact, a DFID-funded program testing new technologies and innovative approaches to address development challenges kWh Kilowatt hour L Liters N/A Not applicable NGO Non-governmental organization PAYGO Pay-as-you-go RBF Results-based financing, and specifically, Global LEAP Off-Grid Appliance Bulk Procurement Incentives program RH Relative humidity ROI Return on investment SHS Solar home system or solar home systems SOGE Scaling Off-Grid Energy, a Grand Challenge for Development and partnership between USAID, Power Africa, DFID, African Development Bank, and the Shell Foundation SME Small and microenterprise or microenterprises TOC Theory of Change UKAid United Kingdom Aid US United States USAID United States Agency for International Development W Watts WHO World Health Organization Global Lighting and Energy Access Partnership Awards, a competition to identify and promote the world’s best off-grid appliances, accelerating market development and innovation 82 83 ToC What Indicators Who When How Question Results Target Value Innovation Assumptipn There are few off-grid refriger￾ation products available on the market that are appropriate for developing country consumers Technicalities of current solutions available in the market (qualitative) USAID Scoping field research in Uganda What are the technical specific details of the current solutions available? Volume requirements ranged between 10L-188L refrigerator capacity in a sur￾vey by CLASP, but majority of response fell between 50L-100L. Support for smaller units. N/A Usability - accessibility and cost of current solutions USAID Scoping field research in Uganda The threshold for demand begins at households who make over approx. $2500/year. At the high end, units peak at $1000 and at the other end units costs below $200 but are unreliable. Interviewees suggested that $100 for a reliable product would be reasona￾ble, but they recommend factoring in other appliances drawing power on an SHS and integrating a payment plan or financing options to acters to users’ income constraints. N/A N/A N/A Researcher Field visit Jan17 consultations What is the perceptions of open data and ICT in Nepal? Yet 70% of them would like to receive more training and see more demon￾strations as well as more qualified local business to address their request and needs.” (CIMMYT research) N/A N/A Capability Assumption The prize will signal to manufac￾turers/designers and innova￾tors that the off-grid market has promising opportunities deserving of their skills and investment. # of nominee who are new in producing off-grid refrigeration system Nominee Before laborato￾ry testing Feb17 Entry Survey 4 organisations are new to the off-grid refigeration industry, and 1 of these is already planning on selling its product in the Bangladesh market N/A % of gender presence in the nominee Nomineew Before laborato￾ry testing Feb17 Entry nomination form What percentage do women consitute of the team that developed this product? All competition: Out of 28 organisation, 10 had a gender presence between 0-20% in their teams. 16 had between 21-50% of women in thei teams. And just 2 organisations had 51-60% of women. N/A Information on part￾nerships from market research Nesta Scoping desk research N/A There is limited information aviailable on partnerships in the sector, but there appears to be very few partnerships between off-grid refrigerator manufac￾turers and distributors. Partnerships occur at an institutional level and more information is available on this N/A % of solar power users aware of possibility to use off grid refrigerators with solar system (in test country) Nesta Scoping desk research N/A Most regions in Uganda are on grid, but there is a lack of knowledge about SHS. SHS also difficult to install due to the housing infrastrucutre in these parts of Uganda, as well as a high theft rate. N/A % of solar companies offering refrigerators as part of their solar system package Nesta Scoping desk research N/A In Uganda, only SolarNow produce small fridges 35L and big fridges 112L as part of their solar systems package. N/A Ecosystem Assumption More efficient appliance including refrig￾erators will boost demands of solar home systems connections Current demand in developing countries USAID Scoping desk research N/A The demand in other developing countries: Bangladesh - very low, 60-80L,$300-500. India - low, 45-55L,$70-625. Kenya - verylow, 50-55L, ~$300. Nigeria - low-to-medium, 50-55L, ~$300 N/A Current demand in Uganda USAID Scoping field research in Uganda N/A Market is nascent, penetrain is extremely low. Market tends to favour small refrigerators with an average preference between 50L-80L size. In terms of its uses, people need fridges to prevent food spoilage, cooling water and drinks, to sell chilled drinks and to store medicine. N/A ToC What Indicators Who When How Question Results Target Value Ecosystem Assumption More efficient appliance including refrig￾erators will boost demands of solar home systems connections Current demand in developing coun￾tries USAID Scoping desk re￾search N/A The demand in other developing countries: Bangladesh - very low, 60-80L,$300-500. India - low, 45-55L,$70-625. Kenya - verylow, 50-55L, ~$300. Nigeria - low-to-medium, 50-55L, ~$300 N/A Current demand in Uganda USAID Scoping field research in Uganda N/A Market is nascent, penetrain is extremely low. Market tends to favour small refrigerators with an average preference between 50L-80L size. In terms of its uses, people need fridges to prevent food spoilage, cooling water and drinks, to sell chilled drinks and to store medicine. N/A Current market perception USAID Scoping field research in Uganda N/A In term of brand awareness of SHS (solar brands), the World Bank found it was lowest in Central and Western regions of Uganda, where 89% and 83% of households couldn’t mention any brand. SolarNow are the most recognised particularly in Northern and Eastern region where 24% and 38% of households knew the brand N/A USAID target for growing solar sys￾tem value USAID Scoping field research in Uganda N/A The SHS target is to increase the number of SHS connections by 20 million N/A Appendix A: Impact Framework Ecosystem Assumption More efficient appliance including refrigerators will boost demands of solar home systems connections Current demand in developing countries USAID Scoping desk research N/A The demand in other developing countries: Bangladesh - very low, 60-80L,$300-500. India - low, 45-55L,$70- 625. Kenya - verylow, 50-55L, ~$300. Nigeria - low-to-medium, 50-55L, ~$300 N/A Current demand in Uganda USAID Scoping field research in Uganda N/A Market is nascent, penetrain is extremely low. Market tends to favour small refrigerators with an average preference between 50L-80L size. In terms of its uses, people need fridges to prevent food spoilage, cooling water and drinks, to sell chilled drinks and to store medicine. N/A Current market perception USAID Scoping field research in Uganda N/A In term of brand awareness of SHS (solar brands), the World Bank found it was lowest in Central and Western regions of Uganda, where 89% and 83% of households couldn’t mention any brand. SolarNow are the most recognised particularly in Northern and Eastern region where 24% and 38% of households knew the brand N/A USAID target for growing solar system value USAID Scoping field research in Uganda N/A The SHS target is to increase the number of SHS connections by 20 million N/A 84 85 ToC What Indicators Who When How Question Results Target Value Impact Activities Research with experts, innovators, and end users; Engaging stakeholders; Scoping problem definition; Define competition and prize statement; Design prize criteria and plan; Technical performance of the product available on the market (quantitative) CLASP Before laboratory testing Feb17 Clasp research The average refigerator consumes from 1 to 2.5 kWh energy per day. While an off-grid solar system would need atleast 2 kWp of capacity to support such a product and even then they would only be able to power the refigerator for a few hours each day N/A Qualitative Information provided by the nominee (on innovation) Nominee via CLASP Before laboratory testing Feb17 Entry nomination form All nominees noted that their products were innovative through their energy efficiency, but they also had different innovative features to improve their product performance such as thicker insulation, higher efficient AC/DC compressors, sensors to control temperature. As well as innovative features to reduce product cost such as sensor-less controls and using low cost but high-quality material e.g. metal casings, house parts. Lastly, to improve its usability and accessibility using different door models, no locks, incorporating pay-as-you-go electronics into the systems solar module, ensuring height and width allows people with low mobility to use the product amongst many other features. For nominees to create innovative features as part of their solutions Positive Status of cohort for field testing E4I Before field testing May18 Field research in Uganda Field testing of the OGRE solutions will be tested on the following characteristics: business profile (small retails shops), those familiar with solar (people who have used SHS), new refrigerator users (have not owned/used a refrigerator) and climate and location (test sites with similar climatics zones) N/A Feedback on judging and eligibility criteria Judges and nominee via CLASP Competition award Jan18 Competition survey (Participants, Judges) Was there a clear link between the competition's criteria and what the competition was trying to achieve? Al judges agreed that there was a clear link between the criteria and the competition aims. Positive Feedback on statement and problem definition Judges and nominee via CLASP Competition award Jan18 Competition survey (Participants, Judges) Was the competition statement clear and well-framed? On average 94% of judges agreed at the very least that the competition statement was well￾framed Positive Capabilities Activities Launch and implementation; Open call to actions; Recruit experts and judges for assessment; Provide support for B2B partnerships Laboratory testing; Field Testing; Award event/ campaign; # of nominee that pass the screening CLASP Before laboratory Feb17 results from screening Current Status Out of the 28 organisations that applied - 6 after the initial screen by CLASP went out of competition either due to their product being ineligible or of a low quality. Another 6 nominations made it through the preliminary screening/round, but withrew from competition. While 15 organisations passed the screening and went on to the next stage of the competition via laboratory testing Positive # of nominee who never applied for USAID/UKAID before Nominee via CLASP Before laboratory Feb17 Entry nomination form Have you applied for funding from USAID or UKAID in the past? Out of the 28 organisations that applied - 22 had never applied for USAID funding, whilse 4 organisations had, while 2 did not provide an answer. Additionally, 25 had never applied to donor sunding to support the development of their product, 1 had applied and that was to the MacArthur Foundation, while 2 again did not respond. Positive Motivations for entry into the competition Nominee After lab testing Dec17 Finalist survey What motivated you to enter the Off-Grid Refrigerator competition? All of the finalists were motivated to enter the competition as they wanted the opportunity to win the competition. 85.7% either wanted the opportunity to win on of the innovation prizes, the opportunity to bring new/innovative solutions to market, learning from results iof the laboratory and/or field testing or the potential to scale an existing product. Positive % gender presence in the nominee Nominee via CLASP Before laboratory Feb17 Entry nomination form What percentage do women consitute of the team that developed this product? Still in competition: 6 organisations have between 0-20% of women in their team, while 8 teams have between of 21-50% of females in their organisations. Negative Assumptions of cohort before field testing E4I Before field testing May18 Field testing pre￾form What are you looking forward to the most from having a new fridge?/ What are you worried about the most about having a new fridge? The 36 field testers expressed different assumptions about solar/off-grid refrigerators, as “it works efficiently without electricity, they work in areas without power, they work well, it cools drinks, it works like other fridges or they had never heard about solar/off-grid refrigerators”. 75% of finalists associated solar/off-grid refrigerators with positive outcomes Neutral Type of support provided for partnership CLASP Competition award Nov18 Interview What support was provided to finalists to engage in partnerships? CLASP did not provide direct support for partnerships, events were held in which finalists were invited to meet key stakeholders within the off-grid refrigeration market, however finalists felt the only support provided was in reference to shipping costs, laboratory and field testing. Negative satisfaction info received All partners Competition award Jan18 Focus Group All partners were satsfied with the award events in Hong Kong and Kigali. It was noted that this competition brought together a dynamic partnership to create new and innovative off-grid refrigerators. Positive Satisfaction from award event CLASP Competition award Jan/Nov18 interview/survey How satisfied were you with the award event(s)? Out of 11 finalists, 6 of them were able to attend the announcement of the first two innovation prizes in Hong Kong. The majority were satisfied, however they did note that the timeline of when the announcement would be made was very unclear. For the last innovation prize awarded in Kigali, none of the finalists were able to attend. Neutral ToC What Indicators Who When How Question Results Target Value Ecosystem Activities Launch and award event/ campaign; Media and social media campaign; Publicizing the prize winners (Advocacy) Field Testing; Coordinating donor agency partnership and Results Based Financing; Product market data or expectations from nominee Nominee via CLASP Before laboratory Feb17 Entry nomination form How do you think your product will perform at market? Is your product ready to go to market? The majority of finalists had the expectations of increasing brand awareness, gaining validation of their product and attracting more sales. Positive Field tester basic record of sales and expenditure E4I Before field testing May17 User survey How much money do you make in the shop (in totla) per day?/How much monehy do you get from the sales of soft dinks (or other products intended for refrigeration) The aggregated sales made in the 36 field testers shop is UGX5.215 million ($1398.76) of which 32% comes from the sale of soft drinks or other products used for refrigeration Neutral Proposed cost in nominee form Nominee via CLASP Before laboratory Feb17 Entry nomination form Total/Estimated total wholesale price of 500 units ($USD) - Internal calculation On average across all the finalists, the average wholesale price of one refrigerator unit is $610.30 USD N/A Type of communication actions CLASP/DAI Ongoing Communication plan CLASP conducted a nominations outreach campaign by leveraging their expansive and exisiting network, as well as facilitating engagement with the innovators. DAI supported SOGE in raising awareness of the competition and prize awards Attract a wide variety of innovators and interest in the off￾grid sector Positive 86 87 ToC What Indicators Who When How Question Results Target Value Innovation Outcomes GENERATE TECHNICAL AND MARKET INNOVATION to develop off-grid appropriate refrigeration for developing countries Extent to which nominee products perform better to those present in the market Nominee via CLASP After lab testing Dec17 Laboratory testing results 12 to 13 finalist off-grid refrigerators outperformed current off-grid refrigerators in the market in four major tests (steady-state operation [10C, 32C, 43C] and pull￾down time) Positive Judges feedback Judges Via CLASP Competition award Jan18 survey/interview To what extent do you agree with each of the following statements: 'The Global LEAP Off-Grid Refrigeration Competition has contributed toward generating technical and market innovation? All judges strongly agreed that the Global LEAP Off-Grid Rrefrigeration Competition contributed to generating technical and market innovation. Positive Performance of product on the field Local business via E4I After field testing Nov18 Field testing results Out of the 36 refrigerators tested in the field, 32 worked in the field though after installation, 9 products had technical issues with 1 being connected to the grid. The average daily energy consumption was 0.69 kWh p/day across all categories at an average external temperature of 27C and humidity of 63%. Only three products produced less energy than their original daily enerrgy consumption value. For the majority of products to work and be tested in the field Positive GENERATING SOLUTIONS TO INCREASE THE VALUE PROPOSITION OF OFF￾GRID CONNECTIONS to give more people the benefits of a life with electric goods Nominee feedback from the field test (qualitative) Nominee via CLASP After field testing Nov19 survey/interview What are your reflections on the feedback provided in the field test report? The majority of finalists were satisfied with the field test report as it allowed their solution to be independetly tested and therefore validated in Uganda, where the majority had n.ever worked before However, many noted there was critical information missing in both technical aspects and user experience and field testing conditions. Neutral Field testing results Local business via E4I After field testing Nov20 User experience form 100% of field testers thought at the very least the products were easy to use and easy to open and close. 98% believed at the very least it was easy to fit these products into their business. Overall 96% of testers were happy with the products as it cooled their drinks well and fast. Positive Capabilities Outcomes MOBILISING TALENTS to generate solutions for off-grid refrigerator % of successful nominee who are new to donor funding Nominee via CLASP Before laboratory Feb17 Entry nomination form Have you applied for funding from USAID or UKAID in the past? 82% had never applied to donor funding and only two finalists - SunDanzer and SimuSolar had applied to USAID for funding. Positive % of nominee who have transferred knowledge Nominee via CLASP Before laboratory Feb17 Entry Survey Did this prize challenge your organization to go beyond your expertise? If yes, how? If no, why? 57% of finalists felt they had transferred knowledge by going beyond their expertise highlighting the additional considerations into ways of improving energy efficieny and producing a low-cost solution Neutral Feedback from gender presence in the finalists and winners Nominee via CLASP Competition award Jan/Nov18 survey/interview What percentage do women consitute of the team that developed this product? 36% of finalists had 0-20% of women working on their team, another 36% had 21-40% of women and 27% had 41-60% of women working on their off-grid refrigeration solution. Neutral BUILD B2B PARTNERSHIPS to accelerate distributions (partnership with distribution investors) BUILD CONSUMERS CAPACITY (Awareness) level of finalist satisfaction on new partnerships Nominee via CLASPw Competition award Jan/Nov18 After lab test Dec17 survey/interview Have you been able to build any partnerships over the course of the competition? 50% of finalists were able to build new partnerships or leverage existing partnerships to gain more customers. 25% were optimistic that through validation of the competition they would be able to build partnerships. 25% had not been able to realise any partnerships yet. Neutral To increase the value they associate to solar systems (local business and final clients) User experience from field testing Local business/ via E41 During field testing Jul18 User experience form User attitudes and reflection on solar/off-grid Of the 36 field testers, 84% could purchase the off-grid refrigerator they tested, noting that the products supported an increase in sales and customers, as well as offering new additional business lines which they could sell. Positive Consumer feedback from field testing Local business/ Consumers During field testing Jul18 User experience form/interview User judgement and perception change on solar/ off-grid All users (100%) assumed that they would somewhat gain from having a refrigerator, be it in sales or customers, but that process wasn’t initially clear to them. Earlier competition research indicated that users would only pay ~$100, but it was found that the most popular range was between $200 and $500. The fact, users are willing to pay more after they have used the products shows how perceptions change once they have the refrigerator. Positive ToC What Indicators Who When How Question Results Target Value Ecosystem Outcomes GROWING A DIVERSE MARKET FOR EFFICIENT OFF GRID APPLIANCES to increase income, prosperity and the quality of products Market projections for competition finalist Nominee via CLASP/Nesta Competition award Jan/ Nov 18 survey/ interview How do you intend to bring your product(s) to market in developing countries? /What are your sales forecasts for this product? The market projections of competition finalists is either zero or relatively low (in the low 000s p/year), as they note the off-grid market is slow to adopt the new and innovatie solutions. Negative Market projection for prize finalists Local business via E4I/Nesta After field testing Dec17 survey/ interview How do you intend to bring your product(s) to market in developing countries? /What are your sales forecasts for this product? For prize finalists (winners), SunDanzer (2) noted global sales for the products in this comptition had generally been in the low 000s p/year. While Palfridge (1) is still a small scale operation, but their winning product is a commercial product noting sales across South Africa, however it is still early for them to offer any projections on this specific product Negative % of improvement in business sales vs expenditure Local business via E4I After field testing Dec17 results form field testing monitoring What is the value of the cold drinks sold?/How much money (total) has the business taken in the period?/How much money do you make in the shop (in total) per day?/How much money do you get from the sale of cold/ refrigerated drinks per day? All 36 field testers found that their business sales increased exponentially at a rate of 21% across the testing period. While expenditure also increased by 42.2% in order to meet growing demand of customers coming from other villages for mainly cold drinks or ice. Positive BUILD VALUE OF SOLAR HOME SYSTEMS to increase demand and subscriptions results from cost perception analysis Local business via E4I After field testing Nov18 results form field testing monitoring How much would you be willing to pay for a solar refrigerator? The results from the cost perception analysis found that the majority are willing to pay $200 to $500 in cash by installments paying $10-20 per month over a period of 1 to 3 years. 57% of users would pay between $200 and $500, 21% would pay less than $200, 16% would pay $500 to $700, 1% would pay $700- $1000 and 4% did not respond as they felt they could not afford the refrigeration and system. Neutral Attitudes towards using SHS Local business via E41/Nesta After field testing Nov19 User experience/ interview/focus groups What value do you think there is in SHS, and solar in general? Out of the 36 field testers all of them found value in SHS as it had no significant cost implications, it was able to stay on continuously without any issues, it was safe. However all of them noted that they knew very few solar companies as they were all in different parts of rural Uganda, and didn't travel to the towns as frequently. Positive RAISING MARKET AWARENESS on the existence of high quality off-grid products results from cost perception analysis Local business via E4I After field testing Nov18 results form field testing monitoring How much would you be willing to pay for a solar refrigerator? The results from the cost perception analysis found that the majority are willing to pay $200 to $500 in cash by installments paying $10-20 per month over a period of 1 to 3 years. 57% of users would pay between $200 and $500, 21% would pay less than $200, 16% would pay $500 to $700, 1% would pay $700- $1000 and 4% did not respond as they felt they could not afford the refrigeration and system. Neutral Attitudes towards using SHS Local business via E41/Nesta After field testing Nov19 User experience/ interview/focus groups What value do you think there is in SHS, and solar in general? Out of the 36 field testers all of them found value in SHS as it had no significant cost implications, it was able to stay on continuously without any issues, it was safe. However all of them noted that they knew very few solar companies as they were all in different parts of rural Uganda, and didn't travel to the towns as frequently. Positive Satisfaction with Global LEAP market facilitation Nominee via CLASP Competition award Nov18 survey/ interview How satisfied are you with market engagement over the course of this competition? 85% of finalists felts the competition and Global LEAP had offered their products credibility towards the market. While Global LEAP facilitated and promoted the off-grid refrigeration solutions at award events, off-griwd and solar events, in-person meeting with distriubtors, manufacturers, investors to ensure the market was fully aware of these new and innovatibe solutions Neutralw # of brands known to provide high quality off-grid products Nominee via CLASP Competition award Nov18 Laboratory/ field testing results There are now 17 high quality off-grid refrigerators available on the market that have undergone Global LEAP's rigorous laboratory and field testing methods. w Positive 88 89 Appendix B: Sample Field Test Report G-LEAP Field Test- Business Data Form: Bi-weekly responses (4) DEMOGRAPHIC DATA Entrepreneur ID ___________________________________________________________________________________________ Product ID ___________________________________________________________________________________________ Geolocalization GPS coordinates can only be collected when outside. latitude (x.y °) ___________________________________________________________________________________________ longitude (x.y °) ___________________________________________________________________________________________ altitude (m) ___________________________________________________________________________________________ accuracy (m) ___________________________________________________________________________________________ How many people are employed at this business? ___________________________________________________________________________________________ How many women? ___________________________________________________________________________________________ How many men? ___________________________________________________________________________________________ If there are any difference in employment from the previous interview, what was the reason? ___________________________________________________________________________________________ BUSINESS/SALES INFORMATION How is the fridge / fridge-freezer being used by the entrepreneur Observation by Field Researcher? ¡ Soft drinks - (Juice + Soda) ¡ Water ¡ Beer ¡ Raw food (fruits and vegetables) ¡ Raw meat ¡ Cooked food ¡ Milk ¡ Other If other, specify ___________________________________________________________________________________________ Description of Items placed in freezer (if available in the unit): ¡ Water for ice ¡ Ice lollies ¡ Soft drinks ¡ Other If other, specify ___________________________________________________________________________________________ How full is the fridge? ¡ Practically empty (0-25% full) ¡ Less than half full (25%-50% full) ¡ About half full (50% full) ¡ More than half full (50%-75% full) ¡ Completely full (100% full) Please take a picture of the inside of the fridge ( with contents) Click here to upload file. (< 5MB) Do you place items for household use in the unit? ¡ Yes ¡ No What household items are you placing in the unit? ___________________________________________________________________________________________ How much space do the household items occupy in the unit? ¡ Less than half the space ¡ About half the space ¡ More than half the space Have you noticed a variation in sales since you received the fridge/freezer? ¡ 1. Yes- I am making far fewer sales ¡ 2. Yes- I have fewer sales, but not by much ¡ 3. No- I have not noticed any difference in sales ¡ 4. Yes- I have some more sales than before ¡ 5. Yes- I making many more sales than before Explain you answer above ___________________________________________________________________________________________ How much money do you get from the sale of cold/ refrigerated drinks per day? ___________________________________________________________________________________________ How much money do you make in the shop (in total) per day? ___________________________________________________________________________________________ Have you noticed any change in the number of customers since using the product? ¡ 1. Yes- I have far fewer customers ¡ 2. Yes- I have fewer customers, but not by much ¡ 3. No- I have not noticed any difference in customer numbers ¡ 4. Yes- I have a few more customers than before ¡ 5. Yes- I have many more customers than before Explain you answer above ___________________________________________________________________________________________ Have you started a new business line because of the product? ¡ Yes ¡ No What additional business lines have you started? ___________________________________________________________________________________________ How else has the product changed your business? ___________________________________________________________________________________________ OBSERVATIONS BY RESEARCHER OVER 1HR PERIOD Enter date and time at start of observation yyyy-mm-dd hh:mm ____________________________________________ ____________________________________________ How many drinks in total have been sold? ___________________________________________________________________________________________ How many of those were cold drinks? ___________________________________________________________________________________________ What is the value of the cold drinks sold? ___________________________________________________________________________________________ How much money (total) has the business taken in the period? ___________________________________________________________________________________________ How many customers came to the shop (in total)? ___________________________________________________________________________________________ 90 91 G-LEAP Field Test- User Experience Form- Bi-weekly responses DEMOGRAPHIC DATA Entrepreneur ID ___________________________________________________________________________________________ Respondent’s name ___________________________________________________________________________________________ Geolocalization GPS coordinates can only be collected when outside. latitude (x.y °) ___________________________________________________________________________________________ longitude (x.y °) ___________________________________________________________________________________________ altitude (m) ___________________________________________________________________________________________ accuracy (m) ___________________________________________________________________________________________ Gender ¡ Male ¡ Female Product ID ___________________________________________________________________________________________ USE OF PRODUCT What are you currently using the product for? (e.g. also for domestic use or not)? ¡ Only business uses- only placing business related items ¡ Only domestic uses- placing personal use items in the fridge ¡ Both business and domestic uses Overall, how easy to use is the product? ¡ Very easy ¡ Easy ¡ Hard ¡ Very hard How easy was it to open and close? ¡ Very easy ¡ Easy ¡ Hard ¡ Very hard How easily does the product fit in your business space? ¡ Very easy ¡ Easy ¡ Hard - I had to rearrange my shop to find space for it ¡ Very hard - I had to completely change my shop or build an extra space for it How happy are you with this product? ¡ Very happy ¡ Happy ¡ Neither happy nor unhappy ¡ Unhappy ¡ Very unhappy What do you like BEST about the product? ___________________________________________________________________________________________ What do you like LEAST about the product? ___________________________________________________________________________________________ How much did you like the look of this product? ¡ 1. It is ugly I really do not like how it looks ¡ 2. It doesn't look very nice ¡ 3. It is ok i neither like nor dislike how it looks ¡ 4. It looks good ¡ 5. It looks very good What would you change about this product? ___________________________________________________________________________________________ Do you think this product has been good for your business? ¡ 1. No- It has been VERY bad for business ¡ 2. No- I has been bad for business ¡ 3. I have not noticed any difference in the business ¡ 4. Yes- It has been good for business ¡ 5. Yes- It has been VERY good for business Explain you answer above ___________________________________________________________________________________________ Anything else you would like to add regarding usability? ___________________________________________________________________________________________ COST EXPECTATIONS Would you purchase this product? ¡ Yes ¡ No Explain your answer above ___________________________________________________________________________________________ Would you purchase a different solar refrigeration product? ¡ Yes ¡ No Explain your answer above ___________________________________________________________________________________________ How much would you be willing to pay for a product like this? ¡ Less than UGX 720K ($200) ¡ UGX 720K- 1.8M / ($200-$500) ¡ UGX 1.8M-2.7M / ($500-$700) ¡ UGX 2.7M -3.6M/ ($700-$1000) ¡ More than UGX 3.6M ($1000) How would you pay for a product like this? ¡ Cash at once ¡ Cash by installments ¡ Get a loan Based on you answer above, how much deposit would you expect to pay? ¡ Less than UGX 180K ($50) ¡ UGX 180K ($50) ¡ UGX 180K-360K ($50-$100) ¡ UGX 360K-720K ($100-$200) ¡ More than UGX 720K (More than $200) How much would you be willing to pay in monthly installments? ¡ Less than UGX 36K ($10) ¡ UGX 36K-72K ($10-$20) ¡ UGX 72K-180K ($20-$50) ¡ > UGX 180K (More than $50) How long would you expect the loan to last? ¡ Less than 1 year ¡ Between 1-3 years ¡ Between 3-5 years ¡ More than 5 years Anything else you would like to add regarding cost? ___________________________________________________________________________________________ Anything else you would like to add? ___________________________________________________________________________________________ A picture of the fridge and installation Click here to upload file. (< 5MB) 92 Expert Judges Steven Hunt (UK Department for International Development) Dr. Arne Jacobson (Schatz Energy Research Center, Humboldt State University) John Keane (SolarAid) Dr. Rezwan Khan (United International University) Charles Muchunku (former Chairman, Kenya Renewable Energy Association) Gopal Nadadur (Clinton Health Access Initiative) Arthur Itotia Njagi (IFC Lighting Africa) David O’Connor (Solar Sister) Himanshu Pokharna (Inficold) Clare Boland Ross (Rockefeller Foundation) Kate Steel (USAID/Power Africa) Written by: Bhavik Doshi Research and Impact Coordinator, Challenge Prize Centre, Nesta bhavik.doshi@nesta.org.uk Partners Donors Implementors